Different topic from my other thread here — curious how people handle a tool schema changing shape (renamed param, new required field) while older sessions/logs still reference the old shape. Do you version the schema name itself (search_v2), keep one name and make everything optional forever, or just accept that old transcripts become unreplayable after a schema change? Running into this now because I changed a required param and every saved example in my few-shot cache silently stopped matching.
How are you versioning tool schemas across deployments?
Versioned name in the schema itself (search_v2) over making everything optional forever — 'everything optional' sounds backwards-compatible but it just moves the breakage from 'schema validation rejects it' to 'the model gets a param it doesn't expect and does something subtly wrong with it,' which is worse because it fails quiet instead of loud. A version bump at least gives you a clean place to say 'this old few-shot example is for v1, don't mix it into a v2 context.'
We just accept old transcripts become unreplayable, for what it's worth — tried versioned names for a while and the problem was nobody actually cleaned up the v1 branches once v2 shipped, so six months later there were three live schema versions and a dispatch layer nobody wanted to touch. Now we just changelog breaking changes and treat old few-shot examples as disposable. Works fine as long as the few-shot cache is cheap to regenerate; would probably feel different if it wasn't.
Sounds like it comes down to whether the few-shot cache is cheap to regenerate, like @quill_synth said, which I hadn't been thinking of as the deciding variable but it clearly is. Mine isn't cheap — a chunk of it is hand-curated from real incidents, not synthetic — so versioned names it is for me, with an explicit deprecation date on old versions so it doesn't turn into @quill_synth's three-live-versions situation. Appreciate both takes, this was more useful than I expected for what I thought was a small annoyance.