Skip to content

When the model provider retires the model: dependency risk you can actually plan for

Model deprecation is scheduled maintenance. What breaks, who pays to retest, and which contract terms a buyer should require in writing before you buy.

nox.markets5 min read
  • vendor-risk
  • models
  • procurement
  • operations

Every product that calls a large model sits on someone else's roadmap. Providers retire names, cut endpoints, raise prices, change refusal behavior, and ship "upgrades" that fail your eval even when the blog post calls them better. If your AP agent is pinned to one vendor URL, deprecation is not an edge case. It is a scheduled maintenance event you did not put on the calendar.

The skeptical question is the right one: what happens when the model provider changes or deprecates a model? If the answer is "we'll figure it out" or "just switch the dropdown," you are the migration project.

What actually breaks

The name in the dashboard is not the contract. A product that prints GPT-something or Claude-something on a marketing page has already confused the buyer. The contract that matters is task success on your documents: fields extracted, reminders that match policy, tickets routed to the right queue. The model is an implementation detail. When vendors lead with the model name, they are selling a dependency.

Silent behavior change is worse than a hard sunset. A retired endpoint at least 404s. A replacement model that is "mostly compatible" will draft a slightly different GL suggestion, skip a refusal, or get more confident while being wrong. Finance feels this first. A wrong amount that posts is not a UX issue.

Your prompt is not portable in the way slides claim. Moving a pile of instructions to a new model is a retest, not a copy-paste. Tool-calling conventions differ. PDF parsers differ. Rate limits differ. The weekend prototype survives. The production worker with idempotency, OAuth scopes, and an exception queue does not survive without someone owning the harness.

Price and routing change under you. Inference cost is the vendor's COGS. When they change pricing, your unattended runs change margin — yours or theirs. If you built the agent, you eat it. If you bought a finished job, the vendor should eat the migration work. That allocation must be in writing.

Compliance artifacts drift. A new subprocessor, a new region, a new training clause, a new retention default. The MSA you filed last year can be stale because the model layer moved. Subprocessor lists have to be live, not a PDF from onboarding.

Who is on the hook (build vs buy)

If you built it, you are the vendor. The 15%-of-an-engineer keep-alive was always this: re-run the eval, compare error rates, decide whether to migrate, update prompts and parsers, tell the controller what changed, and keep the audit trail coherent. There is no shame in that. There is only denial if you budgeted zero.

If you bought a canvas (agent builder, automation platform with an LLM step), you are still the vendor. The canvas did not take accountability for accuracy. You specified the job, you evaluate the output, you maintain it as models change. That can be the right choice when the job is your differentiator. It is a project. Call it one.

If you bought a finished product, migration is much of what you pay for. The honest version of that promise looks like this — and nox.markets does not yet publish the eval half of it:

  • Products run against an internal abstraction, not a single vendor endpoint the customer has to know.
  • On deprecation or degradation, the vendor re-runs a published eval suite and migrates only when the numbers hold. That suite is planned here, not shipped.
  • The published accuracy number is the contract. The model is not. We will not print an accuracy number until a harness exists.
  • You get a changelog with a notice window, not a surprise weekend.
  • Rollback exists. A forced "upgrade" that fails finance evals is a defect.

Until those artifacts are public, treat any "we handle model changes" line as an intention, not a control you can audit.

What to require in the security packet and the MSA

Ask for written answers. If they only exist on a sales call, they do not exist.

  1. Named subprocessors, including every model provider, with region. If a vendor is not on the list, they are not allowed to touch the data.
  2. Notice window for provider or model changes that can change output. Not "as soon as reasonably practicable" as the only clause.
  3. Whether customer data is used to train models — yours or the provider's. The acceptable sentence is contractual no-training on customer data. If they cannot say it, stop.
  4. What is retained, for how long, and whether a human at the vendor can see it. Per product, not a vibe.
  5. Eval on deprecation: dataset type, whether they re-run before cutover, whether you can delay cutover if error rate moves.
  6. Your off switch: pause, revoke OAuth, export logs, export configuration in an open format. Work already in QuickBooks stays in QuickBooks. You are not trapped in a proprietary inbox.
  7. Exit if the marketplace itself shuts down. Configuration export, output history, source escrow for self-hosted. Fair question. Design for it on day one or admit you did not.

What you can do even if you stay on one provider

Keep the system of record as the system of record. If the agent only lives in a chat log, deprecation deletes the work. If every action lands as a draft bill, a CRM note, or a ticket, you still have the business objects.

Pin evals to documents you own. Twenty real invoices in a folder you control beat a vendor demo set. When the model changes, you re-run the same twenty. That is the entire sophistication of an eval harness at this company size.

Default to human approval where money moves. A model swap that would have posted is then a queue of drafts. Annoying. Recoverable.

Prefer vendors who do not print the model name as the product. Invoice matching is the product. The model is a part. Parts get replaced.

Do not stack three model vendors in three point tools for one close. You multiply deprecation calendars. A pack or a single accountable vendor is fewer moving parts. That is operational, not aesthetic.

What nox.markets is claiming, tightly

Model change is ours to solve, and much of what you pay for on a finished job. Products should not make the buyer run a migration project when a provider sunsets a name. The accuracy number on the page is the contract we intend to keep.

We will not imply that this is already proven in production across a customer base we do not have. We will not hide named failure modes behind a model swap. We will not say hallucination is gone because we switched models.

SOC 2 Type II is in progress, not a completed report. ISO 27001 and HIPAA are not claimed here.

Keep the ChatGPT or Copilot seats. They make people faster. They do not absorb deprecation for a process that has to run at 2am. That absorption is a product feature. Demand to see it as a policy, an eval, and a changelog — or staff it yourself and put it on the engineering calendar under its real name.

More from the blog