
Your AI agent pilot is a test. ASD has written how to contain AI tests.
ASD's new guide on securing AI testing environments says prompts and model safeguards aren't enough to contain an agent on their own. For a business trialling an AI agent with real access, that means limiting what its accounts and tools can reach and testing how to stop it.
Published · 7 minute read
AI-created conceptual illustration.
In July I wrote that humans have to hold the switch once we give AI agents room to act. That piece left a practical question open: what should an agent be able to reach in the first place, and how do you stop it when it reaches further?
The Australian Signals Directorate has now published Securing AI testing environments (12 November 2026), a guide to containing higher-risk AI testing. The lesson I’d put first is that what an agent can reach depends on the accounts, tools and connections it holds, whatever its instructions say. If you’re about to connect an AI pilot to your business systems, that changes what goes in the pilot plan.
What the guide covers
ASD wrote it for organisations testing AI models and agents for cyber capability assessments or for use in cyber defence. It concentrates on tests where the agent can run code, use tools, hold credentials or reach a network, and on tests where the model’s own safeguards have been reduced or switched off. It says plainly that it “does not cover securing production environments where AI agents operate”.
The guide points to disclosures by the UK AI Security Institute, OpenAI and Anthropic describing agents in cyber capability tests that reached or affected real systems outside the intended scope. Reported contributing factors included internet access that had been deliberately enabled, misconfigured test environments and weak supporting infrastructure.
Applying its containment ideas to an ordinary business pilot is my reading, not an ASD recommendation for small businesses.
Why a business pilot needs containment
An assistant connected to email may be able to read far more correspondence than the task you’re trialling needs. A workflow automation often holds API keys into several systems. A coding agent can run commands and connect to services. In a pilot they keep every permission their accounts, tools and connections give them.
Back then I asked what each AI can do and reach, who is allowed to widen its limits, and what you would do the morning an agent does something you never sanctioned. ASD’s guide moves those questions earlier. They belong in the pilot plan, answered before access is granted, including what happens when the agent tries something outside the task.
Instructions don’t contain an agent
ASD’s guide says “prompts, scoped instructions and model safeguards do not provide sufficient technical containment on their own.”
An instruction to stay in one folder restricts nothing if the agent’s account can read the whole drive, and an instruction not to send email still leaves you needing a control over when its email tool can run.
For a business pilot I’d start with every tool and connection switched off and add only what the task needs. An assistant being tested on drafting customer replies doesn’t need permission to send them. A coding agent being tested on changes may not need any route to production at all.
Where feasible, the guide recommends giving an agent permission to use a service rather than the credentials for it, through an intermediary that applies the key without the agent ever seeing it. Ask your IT provider or the developer building the pilot whether the agent needs to hold an API key at all, or whether it can request one limited action through a service that keeps the key.
A test environment sized to the risk
ASD describes separation that grows with the risk, from logical separation such as dedicated accounts and virtual networks for lower-risk testing, up to physically separate or dedicated infrastructure for the highest-risk work. Where the model is used through an API, it says separation can come from dedicated test accounts, API keys and cloud tenants. For most business pilots I’d use a separate test account or tenant holding only the data the trial needs.
Write down which services the agent can contact and why, and keep the routes out of the test small enough for one person to watch. The guide warns against weakening containment to make a test feel more realistic. In a business, the same temptation shows up as granting broad access because the narrow connection takes longer to set up.
For higher-risk testing, ASD makes containment verification the first test: “Do not assume that an environment is isolated because it was designed to be isolated.” Before your trial starts, have someone other than the person who set it up check that the agent can’t reach the excluded systems or use the tools it shouldn’t have. If the pilot needs wider access later, the executive sponsoring it approves that change first.
Can you stop it without its cooperation?
Typing “stop” into a chat window works only if the agent chooses to listen. The manager supervising the trial needs ways to cut access that don’t depend on the agent or the environment it runs in.
ASD asks for emergency termination to be tested at independent levels: the model session, identity and infrastructure. In business terms, you should know how you would cut off the agent’s model requests, revoke its logins and keys, block its network access and shut down whatever it runs on, and you should try each of those before the pilot starts.
Choose the manager who owns the process being trialled to supervise it, and give them authority to pause it. Confirm alerts reach them quickly enough to act on, and keep the logs somewhere the agent can’t read, change or delete. Write the stop conditions next to the pilot’s objective. An unexpected connection or a request for more access should make the supervisor decide whether to pause, restrict or end the trial. Where the tooling allows, a repeated attempt to get past a restriction should pause the trial automatically until the supervisor has looked at it.
Treat blocked attempts as findings
ASD’s guide says to “treat attempted and partial control failures as security findings, even where no impact was observed”. In its section on near misses it adds that where the cause isn’t understood, the absence of observed impact isn’t evidence that containment held.
I like this as a habit for business pilots. If the agent tries to reach a service it was excluded from and a control blocks it, write the attempt down and find out why it happened. If it turns up a credential it should never have seen, change that credential and close off where it was stored before the next run.
When the pilot ends, revoke its access and check that the access is gone, then compare the activity logs against the written scope. A pilot that produced great drafts can still have shown you a gap in your controls, and you want evidence on both before you decide to extend it.
A pilot checklist
Before an agent touches your business systems:
- Write down the objective, permitted actions, excluded systems, duration and stop conditions.
- Choose who supervises the trial and give them authority to restrict or end it.
- Give the agent its own test account or tenant with limited data and privileges.
- Leave unneeded tools switched off, and keep passwords and keys away from the agent.
- Have someone independent of the set-up test the restrictions, the alerts and each way of stopping the agent.
- Set limits on run time and token or spending budgets that the agent can’t change.
- Store logs where the agent can’t alter them, and treat near misses as findings.
- Revoke access at the end and confirm it’s gone before approving another run.
Use it to plan a pilot. A pilot with access to sensitive data or production systems still needs a security review matched to that risk.
Who decides whether it goes ahead?
Before the pilot starts, the executive sponsoring it should know which systems the agent can open, who accepted the remaining risk, and what result in the logs and outputs would justify extending it. Your IT provider or developer needs those answers before connecting anything.
We work through these access and stop decisions with clients in our AI Implementation Partnership, which combines leadership, training and hands-on implementation support for A$14,500 a month excluding GST. If you need the leadership without the implementation support, our Fractional Chief AI Officer retainer starts at A$2,500 a month.
Planning an agent pilot? Book a call and bring the task you want to trial and the list of systems it would touch. We’ll help you work out what needs restricting and who will be able to stop it.
Questions
What is ASD's Securing AI testing environments guide?
Does the guide apply to small businesses?
Why aren't instructions to the agent enough?

Written by Todd Johnson
Founder of InnovateX Solutions. Todd provides fractional CIO, CTO and Chief AI Officer leadership for Australian businesses and government, and wrote the XAF Connected Architecture framework.
Todd on LinkedIn