The Cost of Agent Hallucination in Financial Services: A Framework for Risk-Aware Evaluation

Start with the legal entity, role, jurisdiction, product, client, communication channel, and action. The obligations for an internal research assistant differ from those for an investment adviser, broker-dealer, lender, payments operator, or algorithmic trading firm. Compliance and legal owners must identify the applicable requirements before engineers encode them.

Convert each requirement into actor, trigger, prohibited or required action, evidence, deadline, exception, and accountable reviewer. Then locate the exact workflow step where the control can prevent harm. A disclosure checked after a recommendation has been delivered is an audit finding, not an effective preventive control.

For United States securities workflows, scenarios can test truthful descriptions of AI capability, review of communications, conflicts, recordkeeping, and supervision when those duties apply. The SEC has brought cases involving false and misleading statements about the use of AI. FINRA's 2026 oversight report also notes that generative AI can implicate supervision, communications, recordkeeping, and fair dealing. These sources support concrete tests, not a universal claim that one scenario proves SEC compliance or FINRA compliance.

For EU algorithmic trading in scope of MiFID II, Article 17 and Commission Delegated Regulation 2017/589 provide a more specific engineering map. Relevant controls include resilient systems, trading thresholds and limits, prevention of erroneous orders, continuity, testing, monitoring, governance, approval, and records. The delegated regulation states that firms should establish a clearly delineated testing methodology before deployment or a substantial update of an algorithmic trading system.

Keep regulatory citations versioned by effective date and scope. The official EU regulation text is a source for the requirement map. Internal policy determines how the firm implements it, and the test record should name both.

A pre-release pass describes one version under one test distribution. Models, prompts, tools, products, policies, market conditions, and upstream data all change. Regression monitoring connects current production behavior to the evidence that justified release.

Preserve scenario ID, run ID, agent and model versions, prompt hashes, data timestamps, tool responses, constraint verdicts, approvals, action IDs, and final disposition. Use stable links from a customer communication or order back to the trace that produced it. Sensitive information still needs access and retention controls.

Monitor leading indicators such as unsupported-claim rate, stale source use, constraint retries, limit-check failures, reviewer overrides, tool disagreement, and trace incompleteness. Lagging outcomes such as complaints, reversals, losses, and incidents are necessary but arrive after exposure.

Use a fixed critical suite for release comparison and a rotating suite for new conditions. Replay historical edge cases against proposed changes, then add every confirmed production failure to the permanent regression corpus. The regression monitoring guide explains how to set thresholds without hiding deterioration in an aggregate score.

An audit trail needs integrity and interpretation. A signed trace can show that evidence was not altered after creation, but it does not prove the recommendation was appropriate. Pair integrity with scenario coverage, verifier results, human authority, exceptions, and root-cause records as described in the regulatory review framework.

Inventory agent capabilities before writing cases. Record which components can read customer data, generate communications, alter records, submit orders, move funds, or change control settings. Permission defines plausible harm and determines where preventive gates belong.

Rank scenarios by severity, exposure, detectability, reversibility, and control dependence. A low-frequency event can remain critical when it has large notional exposure or weak recovery. Use ranges instead of pretending that the probability and loss of a novel agent failure are known precisely.

Define hard gates before scoring helpfulness. Identity, authorization, product eligibility, trading and credit limits, required disclosures, separation of duties, source freshness, and human approval cannot be averaged against writing quality. Hard failures should name the predicate, observed value, and blocking event.

Test the entire causal chain. Vary customer facts, market data, tool latency, conflicting sources, prompt injection, partial outages, duplicate events, and approval timing. Run repeated trials and compare paths, because two runs with the same final sentence may carry different operational risk.

Add containment tests. Confirm that the system can suspend an agent, revoke a credential, cancel or quarantine a pending action, preserve evidence, notify the correct owner, and replay safely. A system is not ready merely because its normal path is accurate.

Consider a system with strategy, market-data, pre-trade risk, execution, surveillance, and operations agents. The strategy agent proposes intent. Only the execution boundary can submit an order, and it accepts a signed authorization containing instrument, direction, quantity, price controls, expiry, strategy version, and required approvals.

The pre-trade verifier independently recomputes position, credit, price, velocity, and venue limits from authoritative state. It does not trust arithmetic supplied by the strategy agent. A missing or stale input produces an unknown verdict and blocks execution rather than being converted into an assumed pass.

The test matrix varies abrupt price changes, crossed markets, symbol changes, stale reference data, partial fills, rejected orders, duplicate acknowledgements, clock skew, and a compromised research document containing instructions to bypass limits. Each case asserts both the order outcome and the evidence path.

Surveillance is tested independently from prevention. It should receive complete order and decision events, correlate related activity, raise reviewable alerts, and preserve the source data. A preventive control passing does not prove monitoring coverage.

Before a production update, compare the new version with the approved baseline in a controlled environment. Require documented ownership, test evidence, approval, deployment controls, and a rollback trigger. This connects model change management to the governance and testing expectations described in the MiFID II technical standards where they apply.

Estimate expected loss by failure class: probability multiplied by exposure volume and impact. Impact can include direct loss, restitution, correction, investigation, legal and compliance work, customer support, downtime, opportunity cost, and reputational response. Track detection delay because exposure can grow rapidly while a faulty agent remains active.

Attribute prevented loss carefully. Evaluation rarely prevents every instance of a class. Use conservative, expected, and severe scenarios, then compare observed escapes and remediation after deployment. Include the cost of scenario authoring, infrastructure, reviewer time, false alerts, and control maintenance.

The result should drive a control decision. High-severity, irreversible actions need stronger independent verification and narrower permissions. Lower-risk drafts may justify sampling and retrospective review. The goal is not to make every workflow equally restrictive, but to align evidence and authority with exposure.

This framework supports engineering and risk analysis. It is not legal, investment, or compliance advice. Applicable requirements depend on the firm, activity, jurisdiction, and facts, and should be determined by qualified owners.