Containing dual-use AI in the life sciences

01

Dual use is a context problem, not a verdict on a field

Life-sciences tools can support valuable work: understanding published research, improving communication, helping teams organise evidence, and reducing administrative friction. The same broad areas of knowledge can sometimes be used in ways that create or increase risk. That is what makes dual-use governance difficult. A responsible programme must protect legitimate work without assuming that every user, request, or capability carries the same consequence.

For AI products, the relevant unit of analysis is the deployed workflow. A model may be useful in a low-authority educational setting yet inappropriate when connected to restricted sources, external tools, or an autonomous chain of actions. The product’s users, environment, permissions, integrations, and ability to turn information into action determine the control problem more than the label on the underlying model.

The World Health Organization describes dual-use life-sciences governance as a shared responsibility across researchers, institutions, funders, publishers, regulators, and other actors. An AI team should take the lesson seriously: model policy, product engineering, security, domain expertise, operations, and leadership all have a role. Safety cannot be assigned to a single prompt or a single review meeting.

02

Start with a legitimate-use statement that is specific enough to constrain

A vague purpose statement such as ‘help life-sciences users’ cannot guide an access decision. Write a narrower statement for each feature: who uses it, the legitimate task it supports, the kinds of materials it may work with, the environment in which it operates, and the actions it is never intended to take. A good statement allows an engineer, a domain reviewer, and an operator to recognise the same boundary.

For a literature-navigation feature, the statement may permit summarising approved public sources for an authenticated research audience. It may prohibit retrieval from restricted repositories, execution of external procedures, or conversion of an ambiguous request into an operational plan. The system can still be helpful: it can explain why it needs context, offer safe high-level alternatives, or route an eligible question to a qualified person. Clear scope is not the enemy of usability; it is what makes a useful boundary possible.

Attach the statement to a product profile and environment. Development fixtures, staging trials, and production services should not inherit the same authority by default. A product that cannot say which environment produced a result or which policy governed it will struggle to make a credible release or incident decision later.

03

Use a layered control model instead of a single gate

A robust dual-use AI biosecurity programme uses overlapping controls. Identity and organisation scope establish who is asking. Purpose and role checks establish why. Data classification governs what can be retrieved. Tool policies govern which actions are possible. Model safeguards shape the response. Human review resolves the cases automation cannot safely decide. Monitoring and audit evidence make the pathway inspectable after the fact.

Layers matter because each one can be imperfect. A classifier can be uncertain; an identity claim can be stale; a tool integration can fail open if it is poorly configured; a human reviewer can be unavailable. The design objective is not to assume any one layer is flawless. It is to ensure that a failure in one layer does not silently grant broader access or remove the evidence needed to respond.

This is a useful place to distinguish hard from soft constraints. A hard constraint might require an authorised product, environment, and review state before a restricted connector can be used. A soft constraint might prefer a faster response or lower-cost model. Conflating them creates dangerous incentives: a system should not trade a required boundary for a slightly better latency score.

• Scope the feature to a documented purpose, organisation, role, and environment.
• Use least-privilege connectors and short-lived, workflow-specific authority where possible.
• Keep source classification and retrieval policy separate from response-generation policy.
• Require a safe stop or qualified review when required context is missing or contradictory.
• Record minimum useful evidence for each consequential decision and tool attempt.
• Define an owner, response path, and rollback condition before enabling the workflow.

04

Make the boundary visible in a scenario before it is visible to a user

Scenario Studio is where a governance principle becomes a product test. Start with a single request pathway, not an entire scientific domain. Describe the intended task, the user and environment context, the data sources and tools in scope, the expected answer or action, and the conditions that must halt or escalate the workflow. Include explicit agent roles if one agent classifies, another retrieves, and another produces the response.

Then add cases that test the boundary without generating sensitive operational content. An authorised, well-scoped request should work. A request with missing affiliation or unclear purpose should be redirected or escalated. A request that attempts to use an unapproved connector should be blocked. A dependency outage should produce a safe hold, not a broader fallback. These are practical pre-deployment AI testing cases because they evaluate the product’s control logic, not merely the model’s fluency.

Constraint Engine can make the required states explicit. Define the fields and conditions the system needs, then run feasibility and completeness checks before release. A constraint that cannot be measured should be labelled accordingly. A hard condition with no evidence path is a policy aspiration, not an enforceable safeguard.

05

Evaluate safeguards in the real product architecture

A safeguard evaluation should ask more than whether the model declined a sample request. Did the classifier, retrieval layer, agent handoff, tool gateway, and audit record behave consistently? Did a fallback model or provider preserve the same boundary? Did the system stop before an unapproved action? Could a reviewer understand the decision without seeing a larger volume of sensitive content than necessary? These questions examine the system as users actually encounter it.

Use deterministic verifiers for facts that can be directly checked: required scope was present, a restricted connector was not invoked, an approval had not expired, an environment matched, a trace was attached. Use qualified human review for contextual judgement. The UK AI Security Institute’s work on safeguard evaluation stresses the value of clear problem statements and rigorous assessment of safeguards; that framing is especially important when the product combines models with tools and agent workflows.

Do not compress the outcome into an unqualified pass rate. Show the scenario and verifier version, each constraint result, the supporting evidence, and every not-scored state. If a worker, trace collector, or evaluator is unavailable, the resulting uncertainty should stay visible. A dashboard that turns missing evidence into success is not a safety dashboard.

06

Give qualified humans a real decision point

Human review should be reserved for the cases where it adds meaningful judgement, not used as a vague safety net after the system has already acted. The system needs to stop at the correct point, package a concise and privacy-aware record, route it to someone with the right expertise, and wait for a decision. The reviewer needs to know what the product was trying to do, which boundary was involved, what evidence is present or missing, and what decision is being requested.

Define review tiers in advance. A lower-risk ambiguity may call for a product-owner clarification. A material policy conflict may need a domain expert and security owner. A pattern of similar boundary failures may need a governance review rather than one-off case handling. The important thing is that escalation is traceable and has an owner, service expectation, and documented resolution.

The resolution should improve the system. If a reviewer repeatedly sees the same type of ambiguity, strengthen the scenario, add a constraint, refine the routing policy, or reduce an over-broad permission. HITL Analytics can show whether reviewers are reducing risk, where they disagree with automated checks, and whether the review queue is becoming a hidden substitute for product reliability.

07

Keep assurance alive after deployment

The control set that was appropriate at launch may not remain appropriate. New models, new data sources, new users, broader tool permissions, and changing institutional policies can all alter the risk pathway. Monitoring should detect meaningful changes: policy-version drift, unscored hard constraints, unexpected connector usage, missing handoff fields, rising escalation volume, or verification failures in a production environment.

When an alert occurs, attach it to a product, scenario, environment, owner, and evidence record. If the system has crossed a defined boundary, block or narrow the affected path while the case is investigated. RCA should distinguish facts from hypotheses, identify the smallest repair that can be verified, and compare a corrected run with the original. Preserve the case as regression coverage so a later change does not quietly reopen it.

Containing dual-use AI is therefore not a claim that all risk has disappeared. It is an operating discipline: useful scope, proportionate authority, testable boundaries, visible uncertainty, qualified review, and continuous evidence. That is how a life-sciences AI product can remain helpful while taking its responsibilities seriously.