GPT-4o and GPT-4.1 leave the OpenAI API on 14 January: how to test the switch

On 14 October OpenAI told developers that gpt-4o, gpt-4o-mini, gpt-4.1 and gpt-4.1-mini will be removed from the API on 14 January 2027. The suggested replacement for all four is gpt-6-luna. Fine-tuned models built on them go on the same day.

Plenty of production code still sends one of those four names, because they were the cheap, dependable default for a long time. If yours does, you have about eleven weeks. Changing the model name is the easy part. Making sure nothing else changes along with it is the part that needs testing, and that's what this post covers.

What's going, and when

The 14 January date isn't the only one coming up. These are the shutdowns OpenAI has announced for the next six months, from its deprecations page as of 25 October 2026:

Look closely at the third row. gpt-4o-mini-ttssounds like part of the 14 January group, but it's a speech model and it goes eight days earlier. Similar names, different dates. That's why the search below matches exact model names and not a prefix.

Step 1: find every place you name a model

Model names end up in more places than the one API call you remember: environment files, a config JSON, a fallback list, a Terraform variable, a prompt-testing notebook nobody has opened in a year. Search the code for them. Don't rely on memory.

This script walks a folder and prints every exact match for the four retiring names, with or without a date snapshot such as -2024-08-06. It exits with code 1 when it finds any, so you can run it in CI and keep the names from coming back.

// List every place a project names a model that OpenAI retires on 14 January 2027.
// Usage: node find-retiring-models.mjs <folder>
import { readdir, readFile } from "node:fs/promises";
import { join } from "node:path";

const retiring = ["gpt-4o", "gpt-4o-mini", "gpt-4.1", "gpt-4.1-mini"];
// Longest names first, so "gpt-4o-mini" isn't reported as "gpt-4o". An optional date snapshot may follow.
const names = [...retiring].sort((a, b) => b.length - a.length).map((m) => m.replaceAll(".", "\\.")).join("|");
const pattern = new RegExp(String.raw`(?<![\w.-])(${names})(-\d{4}-\d{2}-\d{2})?(?![\w.-])`, "g");
const skip = new Set(["node_modules", ".git", "dist", "build", ".next"]);

async function* files(dir) {
  for (const entry of await readdir(dir, { withFileTypes: true })) {
    if (skip.has(entry.name)) continue;
    const path = join(dir, entry.name);
    if (entry.isDirectory()) yield* files(path);
    else yield path;
  }
}

let hits = 0;
for await (const path of files(process.argv[2] ?? ".")) {
  const text = await readFile(path, "utf8").catch(() => "");
  text.split("\n").forEach((line, i) => {
    for (const m of line.matchAll(pattern)) {
      hits++;
      console.log(`${path}:${i + 1}  ${m[0]}`);
    }
  });
}
console.log(hits ? `${hits} reference(s) to retiring models` : "no retiring models found");
process.exitCode = hits ? 1 : 0;

We ran it on a small sample project with names in a TypeScript file, a JSON config, an .env.example and a Python file. The Python file also mentions gpt-4o-mini-tts. It isn't reported, because it isn't on the 14 January list:

$ node find-retiring-models.mjs sample
sample/.env.example:1  gpt-4o
sample/config/models.json:1  gpt-4.1
sample/config/models.json:1  gpt-4o-2024-08-06
sample/src/summarise.ts:1  gpt-4o-mini
sample/src/voice.py:2  gpt-4.1-mini-2025-04-14
5 reference(s) to retiring models
$ echo $?
1

The search can't see names that live outside the repository: in a database row, a feature flag, a secrets manager, or a dashboard setting in a third-party tool. Check those by hand. A quick way is to log the model field from every API response for a day. OpenAI sends back the model that actually answered, and that tells you what production is really calling.

Why changing the name isn't enough

A new model takes the same request and returns a 200, so a smoke test passes. The things that break are quieter:

• Output shape.Your code parses the reply. If the new model wraps JSON in a code fence, adds a sentence before it or renames a field, the parse fails. Structured outputs with a schema guard against most of this. Free-text prompts that say "reply in JSON" give no such guarantee.
• Length. Newer models often answer at a different length. A summary that fitted a card in your UI may now be cut off, or a reply that was too long may now be too short.
• Cost and tokens. Price per token changes, and so can the number of tokens. A reasoning model can use many tokens you never see in the reply. Compare total tokens per request, not only the price list.
• Latency.Time to first token and total time both move. A timeout that sat comfortably above the old model's worst case may now be too tight.
• Parameters. Some models reject request parameters older ones accepted. The o-series models, for example, refused a custom temperature. Read the new model's page and remove what it doesn't take, otherwise every request returns a 400.
• Behaviour. Refusals, tone, how strictly it follows a system prompt. These matter most when the model talks to your users directly.

Step 2: test properties, not wording

You can't assert on the exact text an LLM returns, because it varies between runs even on the same model. What you can check is the set of things your application relies on: the reply parses, it mentions the facts it has to, it fits the space you have, and it arrives in time.

Take 20 to 50 real inputs from your logs, with personal data removed. Pick the ordinary cases and a handful of awkward ones: an empty field, a very long input, a request in another language. Write down for each what must be true of the answer:

[
  {
    "id": "refund-policy",
    "input": "Customer asks: can I get a refund after 40 days? Reply as JSON with fields answer and policyLink.",
    "mustMention": ["30 days"],
    "maxChars": 600
  },
  {
    "id": "order-summary",
    "input": "Summarise this order as JSON {items, total}: 2x USB-C hub at 39, 1x keyboard at 89.",
    "mustMention": ["167"]
  }
]

Then run every case through the old model and the new one and compare. The harness below doesn't depend on an SDK. You pass it a callModel function that wraps whatever call your app already makes, so the test sends exactly the requests production sends:

// Run the same inputs through two models and compare properties, not wording.
// callModel(model, input) is yours: wrap whatever SDK or HTTP call your app already makes,
// and return { text, ms, inputTokens, outputTokens }.
import { readFile } from "node:fs/promises";

const cases = JSON.parse(await readFile("golden.json", "utf8"));

const checks = {
  // Your app parses this as JSON, so the reply has to parse.
  json: (c, r) => { try { JSON.parse(r.text); return true; } catch { return false; } },
  // Facts the answer must contain, whatever the phrasing.
  mustMention: (c, r) => (c.mustMention ?? []).every((s) => r.text.toLowerCase().includes(s.toLowerCase())),
  // Your UI has a box this big.
  length: (c, r) => r.text.length <= (c.maxChars ?? 2000),
};

async function run(model) {
  const results = [];
  for (const c of cases) {
    const r = await callModel(model, c.input);
    const failed = Object.entries(checks).filter(([, check]) => !check(c, r)).map(([name]) => name);
    results.push({ id: c.id, failed, ms: r.ms, tokens: r.inputTokens + r.outputTokens });
  }
  return results;
}

const [before, after] = [await run(process.env.OLD_MODEL), await run(process.env.NEW_MODEL)];
const sum = (rs, key) => rs.reduce((n, r) => n + r[key], 0);
for (let i = 0; i < cases.length; i++) {
  const [b, a] = [before[i], after[i]];
  if (a.failed.length > b.failed.length) console.log(`REGRESSION ${a.id}: ${a.failed.join(", ")}`);
}
console.log("failed checks", before.flatMap((r) => r.failed).length, "->", after.flatMap((r) => r.failed).length);
console.log("total ms     ", sum(before, "ms"), "->", sum(after, "ms"));
console.log("total tokens ", sum(before, "tokens"), "->", sum(after, "tokens"));

A regression means a case that passed on the old model and fails on the new one. Fix those by adjusting the prompt or the parsing, and run it again. Look at the total time and tokens too. A model that passes every check but costs 40% more is a decision to take on purpose, not something to find on next month's invoice.

Step 3: switch gradually, and before the deadline

• Keep the model name in config, not in code. If the new model misbehaves, switching back should be a config change, not a deploy. Until 14 January you still have somewhere to switch back to.
• Send a share of traffic first. 5% for a day, then 50%, then all of it. Watch your parse-error rate, the share of replies cut off for length, latency at the 95th percentile, and cost per request.
• Pin a dated snapshot when you can. An alias can start pointing at a newer model without you changing anything. A dated snapshot only changes when you change it.
• Test the day after.Once a model is removed, requests to it fail with an error instead of a reply. Make sure that error reaches your alerts. A user shouldn't see a blank reply or a stuck spinner because a fallback list still names a model that no longer exists.

And put the next date in your calendar now. Going by the table above, a model you choose today is likely to be retired within about two years. If you keep the scanner in CI and the golden set next to your code, the next migration takes an afternoon.

The same advice holds for any API you depend on

LLM providers just retire things more often than most. Any API you call will eventually rename a field, drop a version or change a limit. The pattern is the same each time: find every place you depend on it, write down what you rely on as checks, compare old and new, then switch gradually. Our contract testing track applies that idea to ordinary REST APIs.