AI, biosecurity, and the new access problem: when capability outpaces safeguards

01

The problem is access expansion, not knowledge alone

AI can make legitimate scientific work more accessible: it can help people navigate literature, explain established concepts, organise evidence, and communicate across disciplines. Those benefits matter. A useful biosecurity conversation should not collapse all life-sciences assistance into a single category of risk.

The concern arises when a system changes the practical conditions of access. It can make complex information easier to find, package it into a more usable sequence, connect it to tools or workflows, reduce the need for domain expertise, or operate at a speed and scale that weakens ordinary review. None of those effects is automatic, and their significance depends on the system, user, context, and controls. Together, they are why AI biosecurity risk is a governance problem as well as a content problem.

The World Health Organization frames responsible life-sciences governance as a shared responsibility across the research lifecycle, balancing beneficial work with the need to constrain biorisks and govern dual-use research. AI teams should carry the same principle into product design. The objective is not a brittle blanket refusal. It is a system that can support legitimate users while recognising when the request, access pathway, or downstream action needs a stronger boundary.

02

Why model refusals are necessary but not sufficient

A model-level safeguard can be valuable. It may decline an inappropriate request, steer a user toward safe high-level information, or prompt a human review. But once an AI system is embedded in a product, the safety outcome depends on more than its generated text. Retrieval sources, agent handoffs, user identity, tool permissions, logging, provider fallback, and operational escalation all shape what the user can actually obtain or do.

Consider a research-support assistant with an approved knowledge base and a narrow explanatory role. A refusal policy is only one layer. The product still needs to know which organisation and environment the request belongs to; whether a retrieval index is appropriately scoped; whether connected tools are available; how ambiguous requests are handled; and what evidence is retained when the system blocks, redirects, or escalates. A helpful answer can be safely bounded. A seemingly safe answer delivered through the wrong access pathway may not be.

This is also why an AI agent evaluation framework must evaluate the deployed system rather than only a base model. The UK AI Security Institute describes misuse safeguards as technical interventions intended to prevent harmful information or actions, and emphasises rigorous, well-defined safeguard evaluation. In an agentic product, that evaluation should include the surrounding controls that can preserve or undermine the safeguard after generation.

03

Build a proportional access model before you add features

The most constructive design question is: what kind of assistance is the product intended to provide, to whom, in which environment, and with which authority? Answer it in a product profile before exposing tools or broad data connections. A system that only explains public, high-level material has different control needs from one that retrieves private records, coordinates multiple agents, or can trigger an external action.

For each workflow, distinguish ordinary scientific or educational help from requests that are ambiguous, out of scope, or sensitive enough to require a different route. Do not rely on a hidden classifier score alone. Define the resulting experience: the system may provide a safe high-level answer, ask for context, refuse to continue, restrict tool use, or offer a route to an authorised human reviewer. The response should be useful where it can be useful and restrained where restraint is necessary.

A proportional model also avoids treating identity as a cosmetic login step. Authentication, organisation membership, role, approved environment, and purpose limitation should inform what the system is allowed to retrieve and do. When those signals are missing or conflicting, a production-ready AI system should degrade safely rather than assuming the broadest available access.

• Purpose: the legitimate task the feature is designed to support, stated in plain language.
• Audience: the authorised user, organisation, role, and environment for that task.
• Information boundary: which sources are in scope and which classifications require restriction or review.
• Action boundary: which tools, outputs, and follow-on actions are permitted for this workflow.
• Escalation boundary: the conditions that require a safe stop, clarification, or human decision.
• Evidence boundary: the minimum metadata needed to prove the decision without storing raw sensitive content unnecessarily.

04

Turn the access model into scenarios and constraints

A policy becomes more reliable when a team can test it. In Scenario Studio, create a narrow scenario for one product pathway: the task, the expected outcome, the user and environment context, the connected information or tools, and the evidence required to proceed. Include an ordinary request that should be completed safely, an ambiguous request that should request context or route to review, and an out-of-scope request that should not continue through the same path.

Constraint Engine can then express the non-negotiable conditions. A hard constraint might require an approved organisation scope before a retrieval tool is called, require a current review state before a sensitive workflow continues, or prohibit a tool invocation outside the configured environment. The precise rules will differ by product. What matters is that they are observable, versioned, and connected to evidence rather than left as an aspiration in a policy document.

Use Verifier Factory for deterministic facts where possible: was the correct boundary checked, did the system use an allowed connector, was the escalation state recorded, did an unapproved path remain blocked? Use qualified human review for the judgement calls that cannot be reduced safely to a rule. If an evaluation lacks a trace or a verifier cannot run, show that the boundary was not scored. Silence is not proof of safety.

05

Evaluate safeguards as a system, with experts in the loop

Safeguard evaluation should be designed with people who understand the relevant domain and the product’s actual use. A generic AI reviewer may help organise test cases, but it should not decide alone what is safe in a sensitive scientific context. The review group needs enough diversity of expertise to challenge assumptions about user intent, operational pathways, and the limits of the control.

Keep tests high level and bounded. You do not need to create hazardous material or publish detailed misuse prompts to test whether a system respects scope, triggers review, or blocks a prohibited tool path. The evaluation can examine request categories, decision states, evidence completeness, handoff integrity, and safe-stop behaviour. This protects the test process itself from becoming an unnecessary source of sensitive operational detail.

The output should be decision-relevant. Record the scenario and verifier versions, the environment, what passed, what failed, what was not scored, and which residual uncertainties remain. The AI Security Institute’s safeguard-evaluation work makes the broader point: a clear problem statement and rigorous evaluation framework are prerequisites for learning whether safeguards work in practice. That is a better standard than treating a refusal rate as the whole answer.

06

Privacy, traceability, and human escalation must coexist

Sensitive-domain safety often creates a tension: operators need evidence to investigate a boundary decision, while users and organisations should not have their raw prompts, outputs, or confidential records exposed more widely than necessary. The answer is not to choose between privacy and traceability. It is to design minimum useful evidence.

For many evaluations, the essential record is structured metadata: product, organisation, environment, scenario version, policy state, tool name and outcome, redaction status, correlation ID, time, and escalation result. Full content should be retained only when it is justified, authorised, and protected by an explicit policy. Redacted or hashed fields may be sufficient to connect events without replicating sensitive material in every dashboard.

A human escalation should also be a real workflow, not an empty button. It needs a reason code, an owner or queue, an expected response window, a way to record the decision, and a route to turn the outcome into a new scenario or regression test. This is how a system gets better at recognising difficult boundaries without asking frontline users to carry the entire burden of judgement.

07

The practical next step is one controlled pathway

If you are beginning this work, do not attempt to solve every biosecurity question at once. Choose one AI feature that touches scientific or sensitive information. Document its intended purpose, users, data sources, tool permissions, and escalation route. Restrict it to the smallest authority needed. Then write and run a small set of representative, high-level tests that confirm the ordinary path works and the boundary path stops or escalates as intended.

Monitor the feature after release for missing evidence, unexpected tool usage, repeated escalation patterns, and control failures. When something is unclear, create a Quality Issue and investigate it rather than smoothing it over with a score. Preserve the case as a regression test when a repair is made. This approach supports legitimate science precisely because it creates a more reliable and accountable system around it.

The new access problem is not solved by a single filter, a policy page, or a claim that a model is safe. It is addressed through layered, proportionate controls that are tested in the real product context—and through organisations willing to narrow or pause a capability when the evidence does not yet support broader access.