A confidence score is not an approval rule

A customer request can be urgent, routine or outside the categories a business expected. A sales enquiry can be ready for a conversation, need more information or belong with another team.

New AI decision models are designed to make these classifications quickly. They return a bounded answer and a probability instead of an open-ended paragraph.

That makes them easier to connect to a workflow. It does not make the answer safe to act on automatically.

For a small or medium business, the useful question is not whether the model sounds confident. It is whether the business has enough evidence to decide what that confidence may control.

What changed in October 2026

On 1 October 2026, Cloudflare released Clef and Clef Flash, two decision models hosted on Workers AI and available as open source models under the Apache 2.0 licence. They accept a state, a set of typed questions and the allowed answers, then return probabilities that an application can use for classification or routing.

On 6 October 2026, OpenAI released its Decisions API in public beta. It accepts text or images and returns a probability for a condition, a choice from a fixed set or a score against a defined rubric. At publication, it supports only gpt-6-luna through a dedicated endpoint. OpenAI says general availability is expected in the coming weeks, but it remains a beta until that happens.

This is more than two product announcements. Both releases expose a similar building block: a model that turns unstructured business input into a small, typed set of possible answers with probabilities.

Cloudflare has made its hosted models available now. OpenAI's endpoint is still a beta. Neither maturity label removes the need to test the actual business use case.

Why a bounded answer can be useful

A general language model is designed to generate. A decision model is designed to choose within a boundary the application defines.

That narrower job can suit repetitive workflow steps such as:

• routing a support request to the most relevant queue;
• identifying whether an invoice needs a known exception review;
• prioritising website maintenance reports by likely impact;
• sorting an enquiry by service area without accepting or rejecting the lead;
• flagging content that needs a person to assess against a policy.

Typed outputs are easier for software to validate than free text. The application knows the permitted categories before the model runs, and it can send low confidence or out of scope cases somewhere else.

That does not mean every classification needs AI. If a clear rule can route work reliably, keep the rule. A model earns its place when the input is variable enough that ordinary logic becomes brittle, and when the value of better routing exceeds the cost of testing and maintaining it.

A probability is not the same as business certainty

A model can return billing with a confidence of 0.93. That number does not prove that 93 out of 100 similar requests will be routed correctly in this business.

The result depends on the model, the instructions, the allowed options and the inputs it sees. It may also behave differently across writing styles, languages, product lines, seasons and unusual cases.

OpenAI's Decisions API guidance tells developers to use labelled examples from their own application and to choose thresholds based on the cost of false positives and false negatives. That is the important operating principle, regardless of vendor.

A threshold is a business control informed by model evidence. It is not a value to copy from a demo.

Begin with one reversible decision

The first pilot should handle a decision where an error can be noticed and corrected.

Support triage is a better starting point than approving a refund. Drafting a queue recommendation is safer than rejecting an enquiry. Flagging an invoice for review is safer than changing its payment status.

A suitable first workflow usually has:

• a stable and repetitive decision;
• enough historical examples to test;
• categories that staff can define consistently;
• a current process with measurable time, errors or delays;
• an obvious human queue for ambiguous cases;
• a low consequence when the model is wrong;
• a clear owner who can review performance.

If the existing process is rare, highly subjective or already efficient, a pilot may not be worthwhile.

Build the test set before choosing the threshold

Start with a representative sample of past inputs and the decisions the business considers correct.

The sample should include routine cases, difficult cases and items that do not fit the expected categories. It should also reflect the channels and conditions the workflow will meet in practice. A support pilot trained around tidy emails may fail when it receives short form submissions, screenshots or messages with several issues at once.

Ask the people who do the work to label the sample. Where they disagree, do not hide the disagreement inside one answer. Clarify the category, add an uncertain path or accept that the decision needs judgement.

Keep a separate test set that was not used to refine the instructions. Otherwise the pilot can look accurate because it has been tuned around examples it has effectively seen.

The Australian Government's Guidance for AI Adoption recommends testing before deployment, monitoring after deployment and setting clear acceptance criteria tied to the intended use, context and risks.

For a small pilot, that does not require a large governance programme. It does require a written definition of success and evidence that the workflow meets it.

Measure the errors that matter

Overall accuracy can conceal a costly failure.

Suppose a model classifies support requests as routine or urgent. Missing an urgent outage is a false negative. Escalating an ordinary question is a false positive. Those errors do not have the same business cost.

The pilot should measure at least:

• correct results for each category, not only across the whole sample;
• false positives and false negatives for the important categories;
• the number of cases sent to an uncertain or general review queue;
• staff overrides and the reasons for them;
• time to a correct route, not merely model response time;
• failures caused by missing data, integration errors or timeouts;
• performance after instructions, categories or models change.

If missing an urgent request is costly, the workflow may use a lower threshold to send possible urgent cases for fast human review. That may create more false alarms. The acceptable balance depends on the team's capacity and the consequence of delay.

The model's score informs that tradeoff. The business owns it.

Keep an uncertain path

A model forced to choose between incomplete categories will still choose one.

Include an explicit option such as other, uncertain or needs review where the use case allows it. OpenAI's guide recommends a fallback option for inputs outside the supplied categories. The same design principle applies to any provider.

The workflow also needs a technical fallback. If the model times out, returns an invalid response or becomes unavailable, work should move to a known queue. It should not disappear, repeat indefinitely or silently take the default action.

People handling exceptions need the original input, the model's recommendation, the available categories and enough context to decide independently. Measuring their overrides will show where the instructions, taxonomy or model no longer fit the work.

Human judgement stays at the point of consequence

Low risk routing and high impact decisions should not share the same approval rule.

Keep a person responsible when a result can materially affect employment, eligibility, access to an important service, safety, personalised pricing, a substantial financial outcome or another person's rights or interests. Human review should be real: the reviewer needs time, evidence and authority to disagree.

Australian businesses should also check whether a workflow falls within current or upcoming privacy obligations. The existing guide to automated decision transparency explains why human involvement does not automatically remove a consequential workflow from that assessment.

A decision model can make a recommendation easier for software to consume. It cannot decide the legal, ethical or commercial significance of the action that follows.

Privacy and security change the design

Classification inputs often contain customer messages, account details, documents or images. Use only the information the decision genuinely needs.

The Office of the Australian Information Commissioner advises businesses selecting commercial AI products to examine intended use, testing, human oversight, access to personal information, server location, retention and ongoing monitoring. It also warns that cloud processing can disclose personal information outside Australia.

At publication, OpenAI says its Decisions endpoint is available in supported API regions, but regional processing is limited to the United States and Europe. Zero Data Retention and HIPAA support are available only to eligible customers. Those details should be verified against the organisation's requirements before using real data.

Security controls should cover the complete workflow:

• validate input type and size before sending it to a model;
• remove unnecessary personal or sensitive information;
• restrict who can change categories, instructions and thresholds;
• give downstream actions the least access they need;
• protect logs from becoming a second store of sensitive inputs;
• alert on repeated failures, unusual volumes and threshold changes;
• record the model and configuration used for each decision.

Untrusted input can be written to manipulate a classifier just as it can manipulate a general AI assistant. The model should not receive more authority because its output is structured.

Low token prices do not make the workflow cheap

At publication, OpenAI lists Decisions API input at $0.10 per million tokens. Cloudflare lists Clef Flash and Clef input at $0.09 and $0.24 per million tokens respectively. These are vendor list prices and can change.

For many routing tasks, model usage may be a small part of the total cost. The larger work is likely to be:

• defining categories and acceptable errors;
• preparing and maintaining labelled examples;
• integrating source and destination systems;
• building review, retry and recovery paths;
• monitoring results and investigating overrides;
• reassessing privacy, security and vendor changes.

Do not compare a token estimate with the full cost of the current process. Compare the maintained workflow with the maintained workflow.

Keep the question definitions, labels, test cases, thresholds and decision logs outside one vendor's interface where practical. A small adapter around the provider can make future testing or replacement easier. Open weights may reduce one form of dependency, but running a model introduces infrastructure and maintenance responsibilities of its own.

A practical pilot path

A sensible first implementation is:

1. Choose one stable, repetitive and reversible classification.
2. Record the current time, queue delays, error rate and review effort.
3. Define the allowed answers, including an uncertain or other path.
4. Label a representative sample and agree on the cost of each error type.
5. Connect only the information and systems required for that decision.
6. Run the model in shadow mode while people continue making the real decision.
7. Set category-specific thresholds from the results, not from a vendor example.
8. Automate only the low consequence cases that meet the acceptance criteria.
9. Keep ambiguous, sensitive and consequential cases with a person.
10. Monitor overrides, drift, failures, cost and business outcomes before expanding.

Shadow mode is valuable because it separates model testing from operational risk. The business can compare recommendations with real decisions without letting the model move work yet.

If the evidence is weak, improve the categories, input quality or process. Do not compensate by giving the model a broader role.

The next decision

Decision models make one part of AI automation more structured. They do not remove the need to understand the workflow around the score.

Start with one queue. Establish the baseline, label the difficult cases, define the cost of mistakes and preserve a human path. Then test whether the model improves the real outcome before it is allowed to act.

An AI Workflow Development engagement can help identify a suitable pilot, build the evaluation set, connect the necessary systems and design thresholds, review queues and monitoring around the business decision.