Most AI features should be cut in code review

We build AI applications for a living, which is exactly why we cut most AI features before they ship. The demo is not the test. The test is a plain question: does the model change the user’s outcome, or does it just decorate the product?

Three kinds of features fail that test constantly.

Chat bolted onto CRUD. If the user can do the thing with two clicks, a conversation about doing the thing is a slower interface wearing a smarter costume. Chat earns its place when the input is genuinely open-ended, not when a form was already the right answer.

Summaries nobody asked for. Summarizing a document the user was never going to read does not create value; it creates the feeling of value, which decays the first time the summary is wrong and someone acts on it.

Insights that restate the dashboard. “Revenue increased 12% this month” printed under a chart showing revenue increasing 12% is not intelligence. It is a caption with API costs.

What survives our review shares a shape. Retrieval that answers a real question against your own data, with citations a user can check. Extraction that removes typing a human was actually doing: invoices to fields, emails to tickets, documents to structured records. Narrow agents with a small set of verbs, clear stopping conditions, and a human hand-off when confidence drops.

And everything that survives needs an exit, because the model will be wrong on someone’s screen eventually. An exit means a confidence threshold below which the feature stays quiet, an undo that fully reverses what the model did, and a fallback path that works when the provider is down. If a feature cannot degrade gracefully, it is not a feature yet; it is a liability with good posture.

The uncomfortable arithmetic is that one AI feature with real usage beats five that impressed a boardroom. Fewer features, instrumented hard, reviewed like any other production code. That is not caution about AI. It is what taking it seriously looks like.