Enterprise AI still needs supervision

A customer asks for a refund. The model writes a thoughtful response. But has it checked the order, understood the policy, confirmed who can authorize the refund, and updated the account correctly?

Writing the message and resolving the case are different jobs. It becomes obvious when someone has to check every decision before the system can move on.

The concern is widespread. In McKinsey’s 2026 AI Trust Maturity Survey, covering approximately 500 organizations, 74% of respondents considered inaccuracy a highly relevant risk. Nearly two-thirds identified security and risk concerns as the leading obstacle to scaling agentic AI.¹

They point to a real engineering question: what would it take to trust the system with the whole task?

Twenty good steps is a high bar.

Suppose a workflow needs 20 steps, and each step succeeds independently 98% of the time. If any failed step breaks the workflow and there is no recovery, the chance of completing everything correctly is:

0.98²⁰ ≈ 66.8%

To reach 99% overall reliability under those assumptions, each step would need to succeed about 99.95% of the time. Small errors become a much larger problem when the task depends on a chain of decisions.

Real steps differ in difficulty. Errors can depend on earlier errors. Retries, verification, and human intervention change the calculation. But it explains why “98% accurate” needs a follow-up question: accurate at doing what?

A preliminary 2025 report by Project NANDA authors describes a related gap. Drawing partly on interviews with representatives from 52 organizations, it found people who valued general-purpose assistants but were frustrated by custom and vendor-supplied systems that did not fit everyday work. Context retention, integration, and repeated correction were recurring problems.²

Fix the failure you actually have.

If a model cannot find the current refund policy, it needs better access to information. If it has the policy and still applies it incorrectly, access alone is not enough.

Retrieval-augmented generation (RAG) supplies relevant information when a request arrives. Fine-tuning changes the model’s parameters using training examples. One can give the model the right context; the other can adapt how it responds, reasons through a task, or uses tools. They address related problems, but they are not interchangeable.³

Open-weight models give teams the option to adapt those parameters. In Salesforce’s APIGen-MT work, researchers built training trajectories containing dialogue, tool calls, and responses from the environment. They validated outcomes and filtered unsuccessful trajectories before training.⁴

In the paper’s reported comparison, the specialized xLAM-2-70B model averaged 56.2% task success on τ-bench’s retail and airline tasks. Llama 3.1 70B Instruct scored 38.2%. That is an 18-percentage-point improvement without increasing parameter count.

The result is meaningful, and its limits matter. The specialized model also used other training datasets, so the comparison does not isolate one method’s contribution. And 56.2% on a benchmark is evidence of progress, not a reason to hand over a production workflow.

The model proposes. The harness checks.

The harness is the software around the model: it assembles context, manages state, calls tools, checks results, and decides when execution must stop. Training and runtime controls do different work.

Anthropic’s engineering guidance stresses feedback from the actual environment, such as tool results and code execution. It also warns that errors can accumulate across autonomous actions. A model saying it completed a task is not the same as the environment confirming it.⁵

For a refund workflow, permissions and automatic-refund limits belong in code. The system should check them before a payment tool runs, then verify the resulting account state. “Follow company policy” should not be the only thing between a proposed action and someone’s money.

A check is not a cure for every failure. The policy can be wrong. A tool can time out. Verification can be inconclusive. A useful harness makes those conditions explicit and gives the system a defined way to retry, stop, or ask for review.

Test the whole job.

Put the model and harness together, then test them against a strong general-purpose baseline on cases neither system has seen in training. Give both systems equivalent tools, relevant context, and comparable budgets.

Measure completed tasks, consequential errors, human intervention, and cost per correct resolution. A fluent answer, a valid tool call, or a higher benchmark score can be useful evidence. None of them substitutes for finishing the work correctly.

The practical path is to find the failures, improve the relevant data or component, and test again. Sometimes that means targeted training. Sometimes it means current context, a clearer tool interface, or a check that should have been in code all along.

A more dependable system does not always need a bigger model. It needs the right training, the right constraints, and evidence that the whole workflow works.