Choosing an AI model: cost, speed, quality and privacy
No best model, only the right one per task. Four dimensions to weigh, why most workflows need two models, and how to stay reversible.
The short answer
There is no best AI model. There is the right model for each step of a task, chosen on four dimensions: how well it performs on that step, measured on your own cases; how fast it responds; what it costs per call at your volume; and where your data goes, under what contractual terms. Most real workflows end up with two models: a small, fast, cheap one for routine steps and a stronger one for the step that needs judgement. The choice should be reversible, which means prompts and retrieval that are not tied to one provider and a test set you can rerun on any model in an afternoon.
The four dimensions
| Dimension | What to check | Common mistake |
|---|---|---|
| Quality | Score on your own test set for this specific step | Trusting general benchmarks or a demo |
| Speed | Time to first response and to completion, at your typical input size | Choosing a slow model for a step a person waits on |
| Cost | Price per unit of input and output, multiplied by your real volume and input sizes | Estimating from a few test calls rather than a month of traffic |
| Privacy and terms | Training on your data excluded; region; retention; data-processing agreement; who can access logs | Using a consumer plan for business data |
Why two models
- Map the steps of the workflow: classify, extract, retrieve, draft, check.
- Score candidate models per step on the test set.
- Notice the pattern: routine steps score the same on small and large models; one or two steps benefit from the strong one.
- Assign accordingly: small and fast for the routine steps, strong for the hard step.
- Measure the total cost at real volume. The difference between all-strong and mixed is usually large.
Keeping the choice reversible
Models improve monthly and prices fall. A workflow built so that switching is an afternoon’s work captures those improvements; one built around a single provider’s specific features does not. Keep prompts in your own repository, retrieval in your own index, and evaluation in your own test set. Use provider-specific features only where the gain is clear and the fallback is known. Then the question “should we switch models?” is answered by rerunning the set, not by a rewrite.
What this means for you
Choose per step, not per project. Measure quality on your own cases, check speed against who is waiting, compute cost at real volume, and read the privacy terms of the exact plan. Expect two models, not one. And keep the whole thing portable, so that next quarter’s better, cheaper model is an afternoon’s switch rather than a new project.
Frequently asked questions
Should we just use the most capable model for everything?
It is the simplest start and often the wrong steady state. The most capable model is slower and more expensive, and for routine steps such as classification, extraction and short summaries a smaller model is as accurate and many times cheaper. Measure per step on your own cases, and use the strong model where the measurement says it earns its cost.
How do we compare models fairly?
With a test set: fifty to a few hundred real cases from your own work, each with a known good outcome. Run every candidate model on the same set and score the results. Vendor benchmarks measure general ability; your test set measures your task. It is the only comparison that matters, and it takes an afternoon to build.
Which questions should we ask about privacy?
Whether your data is used to train the provider's models, where it is processed and stored, how long it is retained, whether a data-processing agreement is available, and whether a regional option exists. The answers differ by provider and by plan; the business tier is usually where the acceptable answers start.