For the last few months we have been using generative AI to classify requests from our store colleagues. Every incoming request is read by a model, categorised (e.g. visual merchandising, pricing, purchase, stock, replenishment), and assigned to a team. It is not always right. A request about a direct-to-store delivery might land with the team that handles replenishment from the warehouse. When that happens, the person who picks it up moves it along, and nobody raises a concern. We did not expect it to be perfect. The process already had a way to fix a wrong assignment, because people had been making the same mistakes long before the model came along.

Looking back, what made this easy was not the model. It was the nature of the problem. A wrong answer could be walked back, like going back through a door you should not have opened.

Generative AI gives probabilistic answers. It predicts the most likely next word, and then the next. The answer is usually right, sometimes subtly wrong, and occasionally confidently wrong. So by default it needs a human in the loop. In my experience, the use cases that get adopted quickly are the ones where some error is already tolerated, where a wrong answer is cheap, visible and easy to correct. Assigning a service request is one of them. Reconciling the accounts at month end is not.

We have lived with probabilistic answers for a long time. The most familiar one is the weather forecast. A 70% chance of rain does not tell us what will happen; it tells us what is likely. We read it, look out of the window, and decide whether to carry an umbrella. Nobody holds the forecaster accountable for a wet afternoon. We became the human in the loop without anyone designing us into it.

In a business, the human in the loop is not always the same person. It could be an employee, like the colleague who moves a wrongly assigned request along. It could be the customer, who glances at a recommendation and decides whether it makes sense. It could be the supplier, who receives a purchase order based on a forecast and calls back when the quantities look odd. Or it could be someone in between, such as a contact centre agent explaining a delivery estimate. Knowing who that person is, and what they can see and correct, matters as much as the accuracy of the model.

Retail has run on forecasts for decades, mostly behind the scenes. A store manager gets a demand forecast for bread, milk and sunscreen every day. Everyone knows it will be somewhat wrong, but it is far better than guessing. A good manager adjusts it with what the model does not know - a football match on Saturday, a heatwave next week, road works outside the store. Customers see the same thing in other forms. “Customers who bought this also bought” is a guess at what you want next, not far from how a language model guesses the next word. “Runs small, we recommend M” is usually right and occasionally wrong, and the cost of being wrong (e.g. a return) decides how much we trust it. Even a declined card at the checkout is a fraud model making a judgement call, with the customer, and a phone call to the bank, as the human in the loop.

These are the places to start with generative AI, because the organisation already knows how to live with ambiguity. And generative AI brings something the older models could not. It can read the unstructured world - the local events listing, the weather outlook, the supplier’s email about a delayed delivery. More importantly, it can explain itself in plain words. “Orders are up 30% because of the bank holiday and the forecast heatwave” is something a manager can agree or disagree with. A forecast that shows only a number gives them nothing to argue with. Recommendations can move from “people like you bought” to “you mentioned a camping trip in the Highlands in October”. Size advice can come from thousands of reviews and returns notes, not just the size chart. The fraud analyst can get the reason a transaction was flagged, not just a score. In each case the human in the loop is not proofreading every output. They are weighing the reasoning, and adding the context they have and the model does not.

Then there are places where precision is non-negotiable. Financial reconciliation is one. An account that is probably balanced is simply not balanced. Putting a model in every reconciliation run, with a human checking every result, would be slower than the process it was meant to replace. So we change what we ask AI to do. Instead of asking it to reconcile, we ask it to write the code that reconciles. The rules are written down, the test cases are verified by people who understand the accounts, and the code is run against them until it passes. What we end up with is not an AI solution, but deterministic software, verified by humans, which happens to have been written by AI. It does not need a prompt or a model every time it runs. The human verification moves from every run to once, at the tests.

So, to me, the test for any AI use case is simple: what does it cost when it is wrong, and who notices? Jeff Bezos, in his letter to Amazon shareholders in 2016, separated decisions into two types. One-way doors are consequential and nearly irreversible, and deserve to be made slowly and carefully. Two-way doors can be walked back through, and should be made quickly by a person or a small group with good judgement. AI use cases sort themselves the same way. If the error is cheap and someone will notice and correct it - a wrongly assigned request, an odd recommendation - it is a two-way door. Let the model answer, and design the process the way we read a weather forecast. If the error is expensive or silent - a ledger, a payment, a regulatory filing - it is a one-way door. The model should not answer at all. It should build the thing that answers, and people should verify it before anyone walks through.

Where does that leave us? Start where the organisation already tolerates a forecast, because that is where adoption will be fast and the value will show early. We need to measure the AI against the process and the actions it replaces, not against perfection. Ask the model to show its reasoning, because a recommendation with a reason is one a person can correct. And where precision matters, use AI as the builder, not the operator. The more AI-written logic we turn into tested, deterministic software, the less we leave to chance.

Most of us check the weather forecast before we leave home. Very few of us would let a forecast decide our bank balance.