When an AI feature underperforms, the first instinct is usually to change the model. In our experience that is rarely where the problem is. The model call is the most reliable part of the pipeline — it is also the only part that arrives pre-built, which is precisely why the failures cluster everywhere else.
The input is the product
The quality ceiling of an AI feature is set by what you can put in front of the model, and that is an ordinary data engineering problem.
Teremu is a useful example because the constraint is so visible. The feature is: photograph a supplier invoice, and ingredient prices, dish margins, and pantry counts update automatically. The model does the extraction. But extraction quality is dominated by whether the photograph is legible, which is why the meaningful engineering went into on-device quality control at capture time — catching a blurred or cropped frame while the chef is still standing there, rather than failing silently an hour later.
The same pattern recurs. Before adding AI to an existing system, the real question is whether the data and APIs can produce clean, complete, current context at call time. Usually they cannot, not for any dramatic reason — fields are inconsistently populated, the API needs four round trips to assemble one coherent picture, nobody has ever had to read this record end to end before. That work is unglamorous and it determines the outcome.
The output needs somewhere to go
The second failure point is treating generated output as an answer rather than a proposal.
A model extracting thirty line items from an invoice will get most of them right. "Most" is not a number you can post to a ledger. The design question is what happens to the uncertain remainder, and the answer has to be a real interface, not an exception path — in Teremu's case a triage view where extracted lines sit beside the original photograph, so approving a correct extraction takes a second and correcting a wrong one takes five.
This is where human-in-the-loop stops being a principle and becomes a product decision. Review has a cost, and the cost has to be lower than the value of the automation or people will stop doing it. A review step that is slower than doing the task manually will be clicked through without reading, which is worse than having no review at all, because now the errors carry an approval.
Treat inference as a metered dependency
The third thing that breaks is economics, usually later than the technical problems and with less warning.
Inference is a variable cost tied to usage, which makes it structurally unlike the rest of a serverless stack. If the AI loop is the core of the product, model spend is a primary business metric and should be instrumented like one: measured per call, attributed per customer, and capped. Unmetered inference inside a flat-rate subscription is a margin problem that only becomes visible once the feature succeeds.
We also route model calls through a provider layer that is configuration rather than code. Not because switching is a constant activity, but because inference prices have moved substantially and repeatedly, and a product that cannot follow that curve without a refactor gives up the savings by default.
What the model choice is actually worth
None of this is an argument that models are interchangeable — capability differences are real and sometimes decisive. It is an argument about where the marginal hour goes. A better model applied to bad context produces confident nonsense. Good context with a mid-tier model produces something usable.
The practical sequencing follows from that: get the context right, design the review path, instrument the cost, and treat model selection as a tuning decision made against those constraints rather than the first decision made without them.