You Should Be Able to Change AI Models Without Rebuilding How Your Team Works
You Should Be Able to Change AI Models Without Rebuilding How Your Team Works
Say you run procurement for a company that buys a lot of packaging. Before each monthly sourcing round, you get a one-page comparison of the three suppliers on your shortlist: unit price, lead time, minimum order and anything that changed since last month. It's built from the suppliers' latest quotes and your own delivery records, and you use it to decide which supplier gets invited into the next round.
Then the model behind it changes. Maybe the provider is retiring it, or a newer one costs less to run. And it will happen again. Microsoft's guidance on model migration starts from that fact: "Every model you run in production has a retirement date."
You shouldn't have to rebuild how your team works every time that happens. You can't treat it as nothing, either. The same guidance says swapping a model "isn't a one-line change," because a different model can quietly shift its formatting, the shape of its structured output or the way it calls tools, and break the code that depends on the old behavior.
What you can do is keep the work around the model written down clearly enough that a new model can be tested against it.
The model isn't the assignment
The model does real work in that comparison. It reads the quotes, lines up the terms and drafts the recommendation. But the assignment is bigger than the model.
The assignment starts with the question you need answered: which supplier to invite. The work can read this month's quotes and your delivery log, but it can't email a supplier or change an order. It hands back a one-page table with the evidence behind each line. And the assignment ends with you, reading that table and making the call.
If all of that only lives inside one model's prompt, changing the model turns into a quiet rewrite of the process. Someone has to work out what the old prompt was doing before they can test anything, and whatever they miss drops out of the work. When it's written down where the team can see it, you can put a new model in front of the same assignment and judge the result.
Replay before you redesign
Start with the plainest test. Give the new model last month's quotes and the same brief, unchanged, and put its comparison next to the one you already used. Microsoft recommends this too: replay the workload unchanged before tuning anything, so you can separate what the model changed from what you changed. If you rewrite the brief, adjust the tools and switch the model in one go, a better or worse result can't tell you which change caused it.
You aren't looking for the same wording. Microsoft lists what commonly shifts from one model to the next, including how much it writes, how deeply it reasons, how closely it sticks to a required format, how it calls tools and when it refuses. You're looking for whether the comparison still does its job. Is every column still there? Does each recommendation still point to the quote it came from? Did the work still read the delivery log? If the new model needs a firmer instruction or a stricter format, find the exact spot.
Five checks before the new model takes over
Once the replay is done, go through five checks to see whether the new model can take over this comparison.
1. Keep the brief fixed
For the first comparison, change nothing but the model. Keep the question, the reader, the standard for what counts as usable and the reviewer exactly as they were. A new model is enough change for one test. If the brief needs work, do it in a second round, once you've seen how the new model handles the original.
2. Confirm what the work can reach
Check which models and provider settings your workspace actually has, not the ones you've read about. Then confirm the comparison can still read the quotes and the delivery log, and still can't email a supplier. A new model shouldn't change what the work is allowed to touch, and you want to know that before it runs on next month's real quotes.
3. Test what happens after the answer
A well-written comparison doesn't prove the rest of the process still works. If the table feeds the spreadsheet you track suppliers in, run that step too. Microsoft notes that for agentic and workflow workloads, "schema and tool-call work often outweighs prompt work." So check the columns, the format and every tool the work calls, not only how the prose reads.
4. Read it as the person who decides
Then read the new comparison the way you read the old one, as the person who has to choose a supplier. Can you find the evidence behind each recommendation? Can you see why one supplier comes out ahead? Could you decide from this page, or would you have to go back through the quotes yourself? If you'd have to redo the work to trust it, the new model isn't ready for this job yet.
5. Check the record and the bill
Write down which model produced each month's comparison. If you firmed up an instruction after the replay, Microsoft's advice for prompt changes is to keep the original and "record what changed and why." Three months from now, if the table gets worse, you'll know where to start looking.
Then work out what the comparison actually costs on the new model, counting any run that failed and had to be done again. Microsoft notes that pricing structure shifts between model generations and that reasoning tokens, cached input and structured-output overhead "can swing unit economics by 2x or more," so judge cost on your own runs, not the price list alone.
Where Supanova fits
Say the supplier comparison runs on Supanova atoms. When you set it up, you choose how often the workforce checks in with you: at every step, at every task or once per project. If a new model is drafting the comparison for the first time, every task gives you more chances to spot a missing column before the table is finished. That's a setting, so your check-ins don't depend on the new model following an instruction the way the old one did. The finished comparison still comes back to you for review, and nothing ships without you.
When you're ready to try a new model on the supplier comparison, you make that choice in your workspace. You can connect your own OpenAI or Anthropic key, or both, and choose which of their eligible models your atoms use. Your procurement department can have a key of its own.
The replay and the five checks tell you whether the comparison still holds up on that model. You decide what happens next: adopt it, keep the current one, narrow what the comparison covers or stop running it. Staying on the current model is, in Microsoft's words, "a valid choice."
Bring one workflow, not a model debate
Pick one recurring piece of work your team already relies on, the way you rely on the supplier comparison. Write down the question, the sources it may use, what it can and can't touch, the output you expect and the person who decides.
Then "which model is best?" turns into a question you can answer for your own work: can the new model do this work without your team relearning how to direct it?
Start with one workflow at supanova.team.