A great deal of money is spent applying language models to problems that a rule, a lookup table or a well-designed form would solve better, faster and for nothing. The result works, which is what makes it hard to challenge, and it carries a permanent cost the simpler version would not.
Knowing when not to use AI is more valuable than any implementation skill, and the cases are recognisable in advance.
When the rule is already known
If the logic can be written down completely, write it down. A model asked to apply a known rule will apply it correctly most of the time, which is worse than code that applies it correctly every time, costs nothing per call and can be tested exhaustively.
The test is whether an experienced person could describe the decision as a flowchart without hedging. Where they can, that flowchart is the implementation, and reaching for a model instead adds cost, latency and a failure mode for no benefit.
When being wrong is unacceptable
Some outputs have to be exactly right every time: a payment amount, a dosage, a legal deadline, a safety threshold. A system that is right ninety-nine times in a hundred is unsuitable regardless of how impressive the ninety-nine are.
- Anything where a single error causes harm that cannot be undone.
- Anything that must be reproducible, where the same input has to give the same output forever.
- Anything requiring a complete audit trail of exactly why a decision was made.
- Arithmetic, which should be done by code rather than by a model, always.
A model can still assist around these: surfacing the relevant information, drafting an explanation, flagging cases for review. What it should not do is produce the number.
A model is a good tool for problems where being approximately right is useful. A surprising number of business problems are ones where approximately right is worthless.
Ethan Caldwell, CTO & Co-Founder, Engineered With AI
When the real problem is upstream
Teams frequently reach for AI to cope with a mess rather than to fix it. A model extracting data from inconsistent forms is a reasonable stopgap and a poor permanent answer when the forms could be standardised.
Ask what would have to change for the problem to stop existing. Where the answer is a form field, a validation rule or a conversation with a supplier, that is the cheaper fix, and it usually improves several things at once rather than one.
When the volume does not justify it
Automation carries fixed costs: building it, integrating it, monitoring it, maintaining it as things change. Below a certain volume that cost never returns, however elegant the solution.
A process running twenty times a month, taking five minutes each, is two hours of work. Automating it will take longer than that to build and will need attention forever. Doing it by hand is the correct engineering decision and it is one teams find oddly hard to make.
When nobody can say what good looks like
If the people who own a process cannot agree what a correct output is, no system will resolve that, and building one just relocates the argument. Settle the definition first; frequently that conversation turns out to be the whole project.
Where it genuinely fits
The problems that suit a model are the ones with variation that resists rules: unstructured text, ambiguous language, a long tail of cases nobody enumerated, work where a good draft saves most of the effort.
Those are real and common. The point is to reach for it there rather than everywhere, because a business that applies it to the wrong problems first tends to conclude the whole category does not work. More on choosing the first project is in what to automate first.
When the maintenance will outlive the interest
Every deployed system needs somebody watching it as prompts drift, models are retired and upstream formats change. Knowing when not to use AI includes recognising when nobody will own that work after the enthusiasm fades.
A system built by one interested person who then moves on is a liability rather than an asset. If the answer to who maintains this in a year is nobody, that is a reason to choose the simpler implementation.
When you cannot explain it to the person affected
Where a decision affects someone materially and you would have to explain it to them, a system that cannot produce a reason is the wrong tool. Some sectors require this explicitly; in others it is simply what a reasonable customer expects.
Not sure this needs a model?
We will tell you honestly whether your problem needs AI or something simpler.
