To choose an LLM for a real project, you need to narrow the field by four things in order: task fit, cost at your expected volume, latency tolerance, and context window requirements. Benchmarks matter far less than this. Most builders pick a model based on hype or habit, run into a wall three months later, and then switch. Working through these four filters before you write a line of integration code will save you that pain.
Why Benchmarks Are the Wrong Starting Point
Leaderboards measure models on standardised tests. Your project is not a standardised test. A model that scores best on coding benchmarks might still hallucinate confidently on the specific domain your product operates in. A model that ranks lower on reasoning tests might be exactly right for a structured extraction task where it's cheaper, faster, and consistent enough. The benchmark habit comes from researchers and journalists. Builders have different constraints.
The question to ask instead is: what does failure look like in my product? If failure means a wrong answer that a user silently accepts, accuracy is paramount. If failure means a slow response that breaks a user's flow, latency is paramount. If failure means a runaway API bill, cost predictability is paramount. Different answers point to different models, and no benchmark will tell you which failure mode you're optimising against.
The Four Filters: Work Through These in Order
- Task fit: Classify your use case honestly. Is it generative (drafting, summarising, rewriting)? Extractive (pulling structured data from unstructured text)? Conversational (multi-turn dialogue with memory)? Reasoning-heavy (multi-step problems, code generation, analysis)? Each category has models that handle it better at a given cost tier. Running a heavy frontier model on a simple extraction task is like using a CNC router to cut cardboard.
- Cost at volume: Don't test at zero scale and assume the pricing will hold. Model out your realistic monthly token usage: average prompt size, average completion length, number of calls per user per day, expected user count at 3 months and 12 months. Then price it against your two or three candidate models. The difference between providers on a high-volume task can be substantial, and pricing structures change frequently, so always use the provider's live pricing page rather than a cached comparison.
- Latency tolerance: A synchronous, user-facing feature in a web app has a very different latency budget to a background job that runs overnight. Frontier models tend to be slower than smaller, distilled models. If your use case is conversational or real-time, test time-to-first-token, not just total generation time, because that's what users actually feel. If it's an async pipeline, latency barely matters and you can optimise purely on quality and cost.
- Context window requirements: If you're processing long documents, managing long conversations, or doing retrieval-augmented generation over large corpora, you need to know whether your typical payload fits comfortably inside the model's context window. 'Comfortably' means well within the limit, not right at the edge, because performance degrades near context limits on most models. If your workload routinely exceeds what a good mid-tier model offers, that might be a reason to choose a frontier model despite the cost, or to redesign the pipeline so it doesn't need the full context at once.
Before you test anything, write down your failure modes in priority order. Pin that list to the top of your evaluation doc. Every model decision should be checked against it explicitly.
The Model Landscape in 2026: What the Tiers Actually Mean
The market has stratified into three rough tiers, and where you should build depends on which tier actually matches your requirements rather than which tier sounds most impressive.
| Tier | Typical fit | Trade-offs |
|---|---|---|
| Frontier (e.g. the largest GPT, Claude, Gemini variants) | Complex reasoning, ambiguous tasks, high-stakes generation where accuracy outweighs cost | Higher cost per token, often slower, vendor lock-in risk is real |
| Mid-tier (e.g. smaller variants of the same families, Mistral-class models) | Most product use cases: summarisation, classification, structured extraction, light coding tasks | Very strong quality-to-cost ratio; often the right default for builders |
| Small / local models (e.g. open-weight models running on your own infra) | High-volume, low-complexity tasks; latency-critical applications; data-sensitive use cases where you can't send data to a third party | Requires more engineering; quality ceiling lower for complex tasks; worth it when privacy or cost at scale demands it |
A common and expensive mistake is starting with a frontier model because it's easiest to get working, shipping to production, and then finding the bill untenable at scale. A smarter sequence is to prototype with the frontier model, measure what quality level you actually need, then ask whether a mid-tier model can meet that bar. Often it can, especially once you've written a decent system prompt.
Evaluation: How to Actually Test a Model for Your Use Case
- Build a golden dataset: Collect 30 to 50 real examples of the input your product will handle and the output you'd consider correct or good. These should come from actual usage or from domain experts, not from you guessing. This dataset becomes your repeatable test.
- Run every candidate model against it: Use the same prompt structure for each. Score the outputs: you can use a rubric you grade manually, or use another LLM as a grader (with the caveat that LLM-as-judge has known biases and should not be your only signal).
- Measure latency and cost in your actual stack: Don't rely on provider latency claims. Run your test set through the API, time the calls, and calculate the cost. Do this from the region where your app will actually run, because inter-region latency is real.
- Test edge cases and failure modes specifically: Don't just test the happy path. Feed the model the inputs most likely to cause the failure modes you defined earlier. A model that scores 90% on clean inputs but fails badly on edge cases is a worse choice than a model that scores 85% consistently.
- Re-run your evaluation after prompt changes: A lot of apparent model differences evaporate with better prompting. Before you conclude that Model A is genuinely better than Model B, make sure you've given both a fair, well-engineered prompt. You'd be surprised how often a careful system prompt closes a gap that looked decisive.
Factors That Often Get Ignored (But Shouldn't)
- Data residency and privacy: If you're handling data from UK or EU users, you need to know where the provider processes and stores data. This is not optional and it's not just a legal formality. Some providers offer UK or EU data residency; others don't. Check the data processing agreement before you build, not after.
- Rate limits at your expected scale: Free tier and even paid tier rate limits can become a constraint faster than you expect. Check what limits apply to the tier you plan to use, and what the upgrade path looks like.
- API stability and versioning: Models get updated, deprecated, or replaced. If you're building a production product rather than a prototype, understand the provider's version pinning policy. Unexpected model updates can break carefully tuned prompts overnight.
- Multimodal requirements: If your product needs to handle images, audio, or documents as inputs, not all models in a provider's family support this, and the ones that do may differ in quality and cost from the text-only variants.
- Fine-tuning availability: For some use cases, especially domain-specific ones, fine-tuning on your own data can close a quality gap that no amount of prompt engineering will fix. Not every model or provider makes this available at a reasonable cost or complexity level.
If your product handles personal data from UK or EU users, establish your legal basis for using a third-party AI API before you write any integration code. The ICO's guidance on AI and data protection is worth reading. Retrofitting compliance is significantly harder than building it in from the start.
A Practical Decision Sequence for Solo Founders and Small Teams
- Write down the failure modes for your specific feature, in priority order.
- Classify your task type (generative, extractive, conversational, reasoning-heavy).
- Model your token economics at realistic volume for month 3 and month 12.
- Shortlist two or three candidate models based on task fit and cost profile.
- Build a golden dataset of 30 to 50 real examples.
- Run a structured evaluation: quality score, latency, cost, edge case behaviour.
- Check data residency and API terms for your user base.
- Choose the lowest-tier model that meets your quality bar. You can always move up.
- Pin the model version and monitor quality after any provider update.
The goal is not to find the best model. It is to find the model that is good enough for your constraints and the cheapest to operate at your scale. Those are different questions, and conflating them is where most early-stage product decisions go wrong.
Should I just use the most capable model available to be safe?
Not usually. The most capable models are the most expensive and often the slowest. Starting there makes sense for prototyping because it sets a quality ceiling, but shipping to production on a frontier model without testing whether a mid-tier model can meet your bar is a habit that tends to produce nasty billing surprises. Use the frontier model to define what 'good enough' looks like, then test whether a cheaper alternative hits that mark.
How much does context window size actually matter in practice?
It matters a lot for specific use cases: processing long documents, analysing large codebases, maintaining long conversation histories. For most user-facing features, a generous but not extreme context window is fine. The important thing is knowing your typical payload size before you choose, not assuming you'll always be well within limits. Performance on most models degrades when you approach the context ceiling.
Is it worth building on open-weight models to avoid vendor lock-in?
Sometimes, and for specific reasons: data privacy requirements, high-volume tasks where inference cost is a major concern, or use cases where you need to run models on your own hardware. The trade-off is engineering overhead. Running open-weight models in production requires infrastructure that most small teams would rather not own. It's worth it when the constraints genuinely demand it; it's usually not worth it just to avoid hypothetical lock-in.
How do I know if my prompts are good enough before blaming the model?
Test the same task against a clearly superior (and more expensive) model with the same prompt. If the better model doesn't do substantially better, the problem is likely in how you've defined the task, not in the model's capability. Most prompt engineering gains come from being more specific about the desired output format, adding concrete examples of good and bad outputs, and explicitly naming the failure modes you want the model to avoid.
What should UK-based builders know about data residency when using LLM APIs?
UK GDPR requires that personal data processed by a third party (including an AI API provider) is covered by a valid data processing agreement, and that any transfer to countries outside the UK or EEA is done under an appropriate legal mechanism. Some providers offer UK or EU data residency as a paid option; others process data in the US by default. Check the provider's DPA and their data processing locations before you send any user data through the API. The ICO has published guidance specifically on using AI tools with personal data, and it is worth reading before you build, not after.