You Don't Need a Frontier Model to Run Your Business

The default assumption in most AI conversations is that serious AI means OpenAI, Anthropic, or Google. You pay per token, your data travels to someone else's servers, and your monthly API bill climbs in step with usage. For the occasional complex reasoning task, that trade-off might make sense. For the repetitive, high-volume operations that actually run an SME, it usually doesn't.

We've been watching this closely, and a clear pattern is forming. The gap between frontier models and capable open-weight models is narrowing fast. The case for owning your own inference hardware, even at SME scale, is getting harder to dismiss.

What "in-house AI" actually means now

Running AI in-house used to mean a GPU workstation under someone's desk and a lot of Python scripts held together with hope. That picture has changed.

Mistral's recently released Small 4 (https://mistral.ai/news/mistral-small-4/) is a good illustration of where things stand. It handles chat, structured reasoning, coding, and image input in a single model. It has 119B total parameters but only 6B active per token, which keeps GPU memory requirements manageable. It runs on four NVIDIA H100s and is released under Apache 2.0, meaning you can fine-tune it on your own data and run it inside your own walls with no per-call cost and no data leaving the building. Throughput is 3x higher than its predecessor.

That is not a research curiosity. That is a production-grade model you can deploy in a private data centre or a co-located rack, serve to every internal application simultaneously, and pay for once in hardware rather than continuously in API fees.

Where in-house wins clearly

For the bulk of daily SME operations, the arguments for on-premise inference are straightforward:

• Cost predictability. Hardware is a capital cost. Frontier API calls are a variable operating cost that scales with every new workflow you automate. At volume, the maths shifts decisively toward ownership.
• Data sovereignty. Invoices, HR records, customer contracts, medical documents. None of these should be in transit to a third-party API unless you have a very good reason. Mistral OCR 4 (https://mistral.ai/news/ocr-4/), for example, runs in a single container on your own servers and processes documents at $2 per 1,000 pages on the batch route. That drops to zero if you self-host. It scored the highest on OlmOCRBench and reviewers preferred its output over every rival in 72% of head-to-head comparisons.
• Latency and reliability. An on-premise model has no network round-trip and no shared-infrastructure queue. For agentic workflows making dozens of LLM calls per task, that adds up.
• Fine-tuning without exposure. You can specialise a model on your internal data without that data touching an external training pipeline.

Where frontier models still earn their keep

We're not arguing that GPT-4o or Claude 3.5 are never the right tool. They're not.

For genuinely complex, one-off reasoning tasks, multi-step research synthesis, or anything requiring the broadest possible world knowledge, frontier models still have an edge. And for teams with no ML operations capability, the managed API is often the sensible starting point.

The mistake is using frontier models as the default for everything, including the high-volume, well-defined tasks where a smaller, self-hosted model would do the job at a fraction of the cost.

The real question for an SME

The honest cost question isn't "what does GPT-4o cost per token?" It's "what does this workflow cost per month at production volume, and what does the hardware alternative cost amortised over three years?"

For a company running document extraction, internal Q&A, CRM data enrichment, and structured summaries across hundreds of records a day, the in-house arithmetic often wins by the end of year one. The open-weight model options, from Mistral's family to Meta's Llama series, cover most common tasks without touching a frontier API at all.

What to watch

Hardware prices are still falling and model efficiency is still rising. The inference hardware decision you make today will look conservative within 18 months. The more important architectural choice is to design your AI systems so that the model endpoint is swappable, whether that's an on-premise GPU cluster, a managed API, or something in between. Lock in the workflow logic. Keep the model layer replaceable.