AI starts with the question: what to settle before buying capacity

Many AI investment proposals reach the board with the infrastructure cost worked out to the dollar and the problem they will solve described in a single vague sentence. That order is backwards.

Erik Brynjolfsson and Andrew McAfee, in "The Business of Artificial Intelligence" (Harvard Business Review, 2017), describe machine learning as a general-purpose technology, in the same family as the steam engine or electricity. They argue that the bottleneck has moved away from the technology itself and now sits in management, implementation and business imagination. And they warn that plenty of business plans mention AI on every page without any link to what the technology can do.

To avoid signing one of those plans, I suggest four questions.

Question oneWhat goes in and what should come out

Nearly all the recent business successes of AI, according to Brynjolfsson and McAfee, come from supervised learning. The system gets many examples with the right answer attached and learns to go from an input to an output: a transaction and whether it was fraud, a purchase history and the probability that the customer stays.

If the person proposing the investment cannot say in one sentence what data goes in and what result should come out, the project does not exist yet. If they can, half your evaluation is done: wherever there is behavioral data and a result worth predicting, there is a possible use.

Question twoWhich decision gets better

A prediction is only worth something if someone does something different because of it. Knowing a customer is likely to leave helps if the sales team calls that week with an offer; if nobody calls, the model is an expensive report.

So the second question points at the decision. Which person, at which moment, will act differently because the system told them something. If that person is missing from the proposal, ask for them.

The authors also insist these systems are narrow: they do one task well and their knowledge does not transfer. Assuming that a system which handles one thing well understands the thing next to it is, in their view, the biggest source of confusion about AI.

Question threeWhat data you already have

Computing power and algorithms can be rented in the cloud at falling prices, and getting started often takes less information than people assume. What tends to be missing is the realization that the data is already there.

Brynjolfsson and McAfee tell the story of Udacity, where Sebastian Thrun and his student Zayd Enam noticed that the sales chat logs were already labeled data: some conversations ended in a sale and some did not. They used them to build a system that suggested replies to the salespeople. According to the article, salesperson effectiveness rose 54% and they could handle twice as many customers at once. Nobody was replaced; the system advised.

Question fourWhich part of the process changes

AI rarely replaces a whole job, process or business model. If a process has ten steps, the authors estimate that maybe one or two get automated and the rest become more valuable for the people doing them.

And in systems that learn from rewards, the system optimizes whatever you reward, which does not always match what you care about. Reward fast ticket closures and you will get fast ticket closures, even if the customer calls again tomorrow. Defining the goal properly is the manager's job.

Warning signs

• The proposal names the technology before the problem. If the first slide is about models and the last one is about the customer, the order is inverted.
• The system is promised to "understand" the business. Today's systems do one task; people keep doing the rest.
• Perfection is demanded. Brynjolfsson and McAfee warn that results are statistical, that biases inherited from training data (past hiring decisions, for instance) can slip in, and that errors are hard to diagnose. The right benchmark is the best available alternative.