When people start with AI, they often reach for the most powerful model available for everything. It works — but it's usually slower and more expensive than it needs to be. In our lab we treat model choice as an engineering decision, task by task.
Split the job into steps
A single workflow usually contains very different kinds of work:
- Sorting and extracting — classifying an email, pulling an invoice number from a PDF.
- Writing — drafting a reply or a summary in your tone of voice.
- Reasoning — comparing options, spotting inconsistencies, planning multi-step actions.
Sorting and extracting can often be handled by smaller, faster models. Harder reasoning may justify a more capable one. Mixing models within one workflow is normal.
Test on your own examples
Benchmarks are useful, but your documents and your customers are what matter. We build a small test set from real (anonymised) examples and compare candidates on three things:
- Quality — does it get the answer right, in the right format?
- Consistency — does it behave the same way across many runs?
- Cost and speed — what does each task cost, and how long do users wait?
Plan for change
Models improve and prices change every few months. We keep the model choice behind a simple setting, so switching later is a configuration change rather than a rebuild — and we re-run the same test set to confirm the switch is an improvement.
The goal is simple: the cheapest setup that reliably meets the quality bar — not the most impressive one.
