Decision models: how to test AI before automating a process

An AI test can get almost every example right and still fail precisely on the case that requires intervention. To decide whether a step can be automated, management needs to know where the errors are and what the consequence of each one is.

On October 1, 2026, Cloudflare launched Clef and Clef-flash, decision models available on Workers AI. They receive the data of a situation and questions with defined formats, such as yes or no and a choice between options. The answers include probabilities in a structure another system can use. Source: official launch log

Before applying a model of this kind, you need to scope the decision, prepare examples and define which results allow moving forward. The roadmap below proposes an evaluation for the company. It does not report our own tests of Clef or Clef-flash.

“Automating customer service” is too broad a goal to guide a test. “Indicating the team responsible for each request” lets you define the input information, the possible destinations and the expected answer. Approval should apply only to that step.

Consider a hypothetical maintenance process. A form describes the problem with a piece of equipment, and the AI suggests the service queue. The test should specify what information will be available at the time of analysis, including what may be missing.

Separate errors by consequence. Sending a common request for extra review increases the team's work. Failing to recognize a report that requires immediate attention can delay a necessary action. The overall accuracy average mixes these situations and can hide the most serious failure.

Agree on the acceptance rules before looking at the results. The area responsible for the process should say which failures block progress, which allow correction during the pilot and who has the authority to stop it.

Use records the company is authorized to process, with the applicable privacy and access precautions. Include frequent requests, incomplete descriptions and cases that usually cause doubts in the team. Choosing only clear examples makes the test easier than the operation.

For each record, document the expected answer and the criterion used. If two experienced professionals disagree on the destination, review the rule before evaluating the AI. The disagreement may reveal a poorly described category or an exception that still has no owner.

Set aside part of the examples for the final evaluation. If the instructions are adjusted repeatedly based on all the cases, performance on those same records won't serve as independent verification.

Compare the model with the current process and with a simple alternative, when there is one. A direct rule can solve a predictable step. In the other cases, record what the AI adds and which new errors appear.

The Clef API accepts yes-or-no questions, choice between options and rating on an ordered scale. The outputs include probabilities; in choice questions, there is also a selected option and a confidence value. Source: launch documentation

Compare these values with the results observed in the company's examples. A high probability doesn't prove the classification is correct, nor does it remove the need to check required fields and rules that depend on confirmed data.

The pilot can start with recommendations visible only to the team, with no automatic execution. Compare the suggestion with the human decision and review the differences. When information is missing or a situation outside the scope comes up, the flow needs to send the case for review.

Also define the permissions of the system that will receive the answer. The model classifies; the code around it can route, record or request an action, according to the rules implemented. These permissions limit what the system can do even when the classification is wrong.

Record the model version, the instructions and the criteria used in each evaluation. Keep an owner for exceptions and a path back to the previous procedure if results get worse.

Track performance by category, the cases that should have gone to review and didn't, the volume of human corrections and the total process time. The cost math includes integration, operation and review, in addition to model usage.

When an instruction, category or version changes, repeat the tests before expanding use. New cases found in the operation should go into the next evaluations, respecting the limits on data use.

To approve the automation, the team needs to be able to show which cases it tested, which failures it found and how it will contain them. If those answers are still missing, keep human review and fix the test before releasing execution.