Home › Guides › How to switch AI models in your app (without breaking it)

How to switch AI models in your app (without breaking it)

General information, not legal, security, or financial advice. Written for small business owners and solo builders making their own apps with AI coding help. Products, prices, and terms change often; check each provider's current docs.

You built an app for your business with help from Claude Code, Cursor, ChatGPT, or similar, and it uses AI to draft emails, sort leads, or summarize notes. Now a new model looks cheaper, faster, or smarter. Can you just switch? Usually yes, if the app is set up for it. This guide shows how, using a made-up example: a plumbing company's quoting and sales pipeline app.

On this page

1. Two different "AI engines"

People mix these up, and they carry very different risks:

The rest of this guide is about the second kind.

2. Pros and cons of switching or mixing models

Why people switch or mix:

What it costs you:

3. Design your app for switching

One AI service layer

The single most useful habit: all AI calls go through one file or module. The rest of the app asks for a task ("draft an estimate email"), not a specific company's model. Each provider gets a small "adapter" that translates. Switching then means changing config, not hunting through 40 files. A short TypeScript sketch (Python looks much the same):

// ai.ts - the ONLY file in your app that talks to AI providers
import config from "./ai-config.json";
import { anthropicAdapter } from "./adapters/anthropic";
import { openaiAdapter } from "./adapters/openai";
import { loadPrompt } from "./prompts";

const adapters = { anthropic: anthropicAdapter, openai: openaiAdapter };

export async function complete(task: string, input: object) {
  const { models, timeoutMs } = config.tasks[task];   // e.g. "draft_estimate_email"
  const prompt = await loadPrompt(task);              // prompts/draft_estimate_email.v3.md
  for (const m of models) {                           // first choice, then fallbacks
    const start = Date.now();
    try {
      const out = await adapters[m.provider].call(m.model, prompt, input, timeoutMs);
      log({ task, model: m.model, ms: Date.now() - start, tokens: out.usage });
      return out.text;
    } catch (err) {
      log({ task, model: m.model, error: String(err) });  // try the next model
    }
  }
  throw new Error(`All models failed for ${task}`);
}

// Elsewhere in the app, nobody knows or cares which company answers:
// const email = await complete("draft_estimate_email", { customer, quote });

You can ask your coding assistant: "Refactor all AI calls into a single service module with provider adapters, per-task config, timeouts, and fallbacks."

Or use a common API or gateway

Instead of writing every adapter yourself, you can route calls through a service or library that speaks to many models in one format. Options include:

Cloud platforms make sense if you already use that cloud. A gateway adds another company between you and the model, so check its pricing, data terms, and reliability too. Model names and features on a gateway don't always match the provider's own API exactly.

Choose the model per task, in config

Keep model names, fallback order, and timeouts in one config file. Then a switch is a one-line change you can roll back. Use pinned model versions rather than "latest," so behavior doesn't change under you.

{
  "tasks": {
    "draft_estimate_email": {
      "models": [
        { "provider": "anthropic", "model": "[pinned model version]" },
        { "provider": "openai",    "model": "[pinned model version]" }
      ],
      "timeoutMs": 20000,
      "promptVersion": "v3"
    },
    "classify_service_call": {
      "models": [ { "provider": "openai", "model": "[small, cheap model]" } ],
      "timeoutMs": 8000,
      "promptVersion": "v1"
    }
  }
}

Keep prompts as versioned files in Git

Don't bury prompts inside code strings. Put each one in its own file, with a version number in the name, so you can see what changed, compare versions, and roll back.

prompts/
  draft_estimate_email.v3.md     <- the instructions, in plain text
  classify_service_call.v1.md
evals/
  draft_estimate_email.cases.json
  classify_service_call.cases.json

Build a small test set before you switch (an "eval")

An eval is a list of example inputs plus what a good answer must (and must not) contain. Run it on the old and new model and compare. Twenty to fifty real-ish cases catch a lot. Include tricky ones: missing data, odd requests, angry customers.

[
  {
    "name": "basic estimate follow-up",
    "input": { "customer": "Dana", "estimate_total": 2150, "product": "50-gallon water heater install" },
    "must_include": ["Dana", "2,150"],
    "must_not_include": ["discount", "guarantee"],
    "max_words": 150
  },
  {
    "name": "missing price - should not invent one",
    "input": { "customer": "Lee", "estimate_total": null, "product": "leak repair under kitchen sink" },
    "must_not_include": ["$"],
    "expect": "asks for the missing price or leaves a clear placeholder"
  },
  {
    "name": "service call classification",
    "task": "classify_service_call",
    "input": { "message": "Water heater is leaking all over the garage, can someone come today?" },
    "expect_json": { "category": "emergency", "urgency": "high" }
  }
]

Your coding assistant can write a small script that runs every case against any model and prints pass/fail, cost, and speed.

Keep business logic in code, not prompts

Prices, discounts, tax, approvals, and "who gets what" belong in regular code, where they're predictable and testable. Let the AI handle the language around them. Then switching models can't change your prices.

Risky:

Prompt: "If the job needs more than 4 hours of labor, give a 5% discount, unless it's
commercial, in which case add a 15% after-hours fee for emergency service calls..."

Better:

// Code decides the numbers. The AI only writes the friendly email around them.
const price = calculateEstimate(job);          // tested, predictable, the same every time
const email = await complete("draft_estimate_email", { customer, price });

Check every structured answer with a schema

When the AI returns data (a service-call category, a list of line items), check it against a schema before your app uses it. In TypeScript, zod is popular; in Python, pydantic. Several providers offer "structured output" modes that help, but checking it yourself works the same no matter which model answered.

import { z } from "zod";

const ServiceCallSchema = z.object({
  category: z.enum(["emergency", "repair", "install", "spam"]),
  urgency: z.enum(["low", "medium", "high"]),
  summary: z.string().max(200),
});

const raw = await complete("classify_service_call", { message });
const parsed = ServiceCallSchema.safeParse(JSON.parse(raw));
if (!parsed.success) {
  // Don't trust it. Retry once, try the fallback model, or send to a person.
}

Keys, logs, fallbacks, and rollout

4. Subscriptions vs. API billing

A common surprise: a chat subscription doesn't pay for your app's AI calls.

Estimating API cost simply:

Rough monthly cost for one AI feature =
  (calls per month) x (input tokens per call x input price per token
                       + output tokens per call x output price per token)

Example (made-up round numbers - use your provider's current price page):
  40 quote emails a day x 30 days      = 1,200 calls/month
  ~1,500 input tokens + ~300 output tokens per call
  1,200 x 1,500 = 1.8M input tokens;  1,200 x 300 = 0.36M output tokens
  Multiply each by the per-million-token price for your model, then add both.
  Rule of thumb: 1 token is roughly 3/4 of an English word.

Then add a safety margin, set a spending limit, and compare against your logs after the first week. Cheaper models can cost many times less per token than top models, which is why mixing models per task can matter.

5. Questions people don't think to ask

6. Checklists

Switching day

[ ] 1. Pick one task to switch (not the whole app at once)
[ ] 2. Read the new model's docs: pricing, rate limits, data terms, JSON/tool support
[ ] 3. Add the API key to your secrets manager (never in code or Git)
[ ] 4. Add or update the adapter for the new provider
[ ] 5. Copy the prompt to a new version file; adjust wording if needed
[ ] 6. Run your eval cases on old AND new model; compare quality, cost, speed
[ ] 7. Fix the prompt until the new model passes; re-run all cases
[ ] 8. Point a small slice (e.g., 5-10%) of real traffic at the new model, behind a flag
[ ] 9. Watch logs for a few days: errors, schema failures, cost, speed, complaints
[ ] 10. Increase gradually; keep the old model as the fallback
[ ] 11. Note the change (date, model version, prompt version) in your changelog
[ ] 12. Know how to roll back: flip the flag or config back to the old model

Portability checklist

[ ] All AI calls go through one service file/module (no provider SDKs scattered around)
[ ] Model names and fallback order live in config, not hard-coded
[ ] Model versions are pinned, not "latest"
[ ] Prompts are files in Git with version numbers
[ ] Business rules (prices, discounts, approvals) are in code, not prompts
[ ] Every structured output is checked against a schema
[ ] There's an eval set you can run against any model in minutes
[ ] Keys are in a secrets manager; each provider has spending limits set
[ ] Logs record model, prompt version, tokens, cost, speed, and errors per call
[ ] Timeouts and a fallback model are set for every task
[ ] You know which tasks use embeddings, and which model made them
[ ] You've checked each provider's data-use and retention terms for customer data

FAQ

Is it risky to switch from one AI coding assistant to another?

Usually low risk. Your code is ordinary files you own; keep them in Git and you can open the same project in a different tool. The bigger decisions are about the AI models your app calls while it runs.

Does my ChatGPT or Claude subscription cover my app's AI usage?

Generally no. App calls go through the provider's API, which is billed separately, usually per token. Anthropic's Max and Team plans include some monthly API credits; check each provider's current billing pages.

What's the easiest way to make my app model-independent?

Route every AI call through one service module, keep model names in config, keep prompts as versioned files, and validate structured answers with a schema. A gateway or cloud AI platform can reduce how many adapters you write.

Do I need to retest after switching models?

Yes. Prompts, formats, and accuracy can change between models. Run a saved set of test cases on both models, then roll out gradually.

Can I reuse my embeddings or fine-tuned model with a new provider?

No. Embeddings only work with the model that created them, so re-embed your data. Fine-tuned models stay with the provider where you trained them.

Rather skip the plumbing? BotFit's done-for-you bots handle specific chores with you approving every step: Get Paid drafts invoice reminders from QuickBooks Online (early access waitlist below), and Florida Sales Tax, Done For You prepares your DR-15 for your approval. For prompt tips, see How to train your bot.

More guides

BotFit Get Paid: early access

Polite invoice reminders drafted from your QuickBooks Online overdue invoices. You approve the messages before anything is sent. $19/mo, or $14/mo as an add-on for Florida sales tax customers. Join the list and we'll email you when your spot opens. Free to join, no payment now. Unsubscribe anytime.