LLM features need an exit

Every LLM feature has two lives. The first is the demo, where someone types a good prompt into a warm model and the room nods. The second is production, where a tired user pastes something nobody predicted and acts on whatever comes back. Products succeed or fail in the second life.

The difference between the two is not model quality. It is whether the feature has an exit: a designed path for the moment the model is wrong. We review every AI feature against four of them.

A threshold below which it stays quiet. Confidence signals are imperfect, but they are not useless. Retrieval scores, self-consistency checks, or a cheap verifier model can tell you when an answer is shaky. A feature that says nothing is often more trustworthy than one that guesses, because every confident wrong answer spends credibility you cannot buy back.

An undo that actually undoes. If the model drafted an email, deleting the draft is an undo. If the model modified records, renamed files, or sent anything anywhere, undo means a real reversal path, tested like a database rollback. If an action cannot be reversed, it needs a human in front of it, every time.

A fallback when the provider is down. Model APIs have outages, rate limits, and latency spikes, usually at the worst hour. The feature should degrade to something honest: the manual flow, a cached result clearly labeled as cached, or a plain explanation. The product must remain a product with the AI turned off.

A hand-off with context. When confidence is low or stakes are high, the right move is routing to a person, carrying everything the model gathered so the human does not start from zero. The pattern is old and proven; call centers have done it for decades. Software is only now relearning it.

None of this shows up in a demo, which is exactly why it separates shipped AI from performed AI. The teams doing this well spend most of their AI budget after the model call: on evals that catch regressions, on instrumentation that shows what users accepted or corrected, and on these exits.

The model being wrong is not a bug you fix. It is a property of the material you are building with, like wood having grain. Good products are designed with the grain in mind.