In a chatbot, the attack surface is simple. There is one input channel: the user message. There is one output channel: the model’s text response. The threat is a user who types something malicious.<\/p>\n\n\n\n
The defense is correspondingly simple: train the model to recognize and resist adversarial user inputs. Monitor output for policy violations. The model is the system. Securing the model secures the system.<\/p>\n\n\n\n
This worked well enough for the chatbot era.<\/p>\n\n\n\n
The Agentic Threat Model<\/strong><\/h2>\n\n\n\nAn agent is not a model. It is a system that includes a model, plus tools, plus memory, plus external integrations, plus \u2014 in multi-agent architectures \u2014 other agents.<\/p>\n\n\n\n
The input surface is no longer just the user message. It includes:<\/p>\n\n\n\n
\n- Documents the agent retrieves<\/li>\n\n\n\n
- Search results the agent processes<\/li>\n\n\n\n
- API responses from tool calls<\/li>\n\n\n\n
- Memory retrieved from previous sessions<\/li>\n\n\n\n
- Instructions from orchestrating agents<\/li>\n\n\n\n
- Content from any URL the agent visits<\/li>\n<\/ul>\n\n\n\n
Every one of these is a potential injection vector. And unlike user messages, most of them are not monitored, filtered, or treated as adversarial by default.<\/p>\n\n\n\n
Why Standard Defenses Fail<\/strong> <\/h2>\n\n\n\nInput filtering fails<\/strong><\/h3>\n\n\n\nbecause the injection does not come from user input. A content filter on the user message field catches nothing when the injection rides in through a tool output.<\/p>\n\n\n\n
Output monitoring fails<\/strong><\/h3>\n\n\n\nbecause the damage often happens before any output is generated. Tool selection, memory writes, and sub-agent instructions happen inside the reasoning chain \u2014 not in the final response.<\/p>\n\n\n\n
Instruction hierarchy fails<\/strong><\/h3>\n\n\n\nbecause it assumes you know where instructions come from. When an agent reads a document that contains an embedded instruction, the model has to decide whether that instruction is from the user, the system, or the document. Context-reframing attacks exploit this ambiguity.<\/p>\n\n\n\n
Adversarial training helps but is insufficient<\/strong><\/h3>\n\n\n\nbecause the space of environmental injection vectors is orders of magnitude larger than the space of adversarial user messages. You cannot train on every web page, document, or API response your agent might encounter.<\/p>\n\n\n\n
What Actually Changes<\/strong><\/h2>\n\n\n\nThe fundamental difference is this: in a chatbot, trust boundaries are clear. The system prompt is trusted. The user message is untrusted. You evaluate user messages against that boundary.<\/p>\n\n\n\n
In an agent, trust boundaries are blurry and dynamic. The agent processes content from sources with different trust levels \u2014 sometimes in the same context window, sometimes without clear demarcation. The model has to infer trust from context. Attackers exploit that inference process.<\/p>\n\n\n\n
This is not a model capability problem. It is a system architecture problem. The solution requires rethinking trust at the system level, not just tuning model behavior.<\/p>\n\n\n\n
For an empirical example of this inference ambiguity in action \u2014 and what UNCERTAIN looks like in a real evaluation \u2014 see: Why Claude Haiku Returned UNCERTAIN \u2192<\/a><\/p>\n\n\n\nWhat Effective Agentic Security Looks Like<\/strong><\/h2>\n\n\n\nThree principles that chatbot-era security thinking misses:<\/p>\n\n\n\n
Treat every environmental input as potentially adversarial<\/h3>\n\n\n\n
Documents, search results, tool outputs \u2014 all of them should be processed with the same skepticism you apply to untrusted user input. This means behavioral testing against environmental injection, not just user-supplied adversarial inputs.<\/p>\n\n\n\n
Test tool call sequences, not just responses<\/h3>\n\n\n\n
The signal of a successful injection in an agentic system is often not what the agent says \u2014 it is what tool the agent calls, with what parameters, at what point in the reasoning chain. Behavioral evaluation has to instrument tool calls.<\/p>\n\n\n\n
Model the blast radius of every tool<\/h3>\n\n\n\n
Every tool your agent can call represents potential damage if the agent is manipulated into misusing it. Irreversible tools (email, file write, API calls with side effects) require confirmation gates or strict scope constraints \u2014 not just model-level refusal training.<\/p>\n\n\n\n
Practical Starting Point<\/strong><\/h2>\n\n\n\nIf you are building an agent today and want to understand your ASI01 attack surface, start here.<\/p>\n\n\n\n
Install the framework:<\/strong><\/p>\n\n\n\npip install safelabs-eval<\/code><\/pre>\n\n\n\nInspect the ASI01 prompt library \u2014 the three attack vectors the framework tests, from direct system override to environmental document injection:<\/p>\n\n\n\n
safelabs prompts --category ASI01<\/code><\/pre>\n\n\n\n![\"\"](\"https:\/\/agentsafelabs.com\/blog\/wp-content\/uploads\/2026\/06\/Screenshot-2026-06-07-at-1.18.23-PM-1024x460.png\")
<\/figure>\n\n\n\nFor comparison, here is the ASI02 library \u2014 scope violation vectors that exploit the same architectural weakness through tool permissions rather than instruction injection:<\/p>\n\n\n\n
safelabs prompts --category ASI02<\/code><\/pre>\n\n\n\n![\"\"](\"https:\/\/agentsafelabs.com\/blog\/wp-content\/uploads\/2026\/06\/Screenshot-2026-06-07-at-1.22.35-PM-1024x393.png\")
<\/figure>\n\n\n\nThe pattern is consistent across categories: the attack does not come from the user message. It comes from content the agent processes as part of doing its job.<\/p>\n\n\n\n
Run the evaluation against your agent:<\/p>\n\n\n\n
safelabs run --target http:\/\/localhost:8000\/chat --category ASI01<\/code><\/pre>\n\n\n\n![\"\"](\"https:\/\/agentsafelabs.com\/blog\/wp-content\/uploads\/2026\/06\/Screenshot-2026-06-07-at-1.23.00-PM-1024x464.png\")
<\/figure>\n\n\n\nThis run returned 2 PASS and 1 UNCERTAIN on ASI01. The UNCERTAIN on ASI01-003 \u2014 the document-context injection \u2014 is the result that matters. The agent processed a document containing a hidden instruction and its behavior deviated from expected without producing a clean refusal signal.<\/p>\n\n\n\n
That is the agentic threat model in practice. Not a user typing something malicious. A document the agent trusted containing something it should not have.<\/p>\n\n\n\n
The framework also exposes a Python API if you want to wrap your own agent function directly:<\/p>\n\n\n\n
import asyncio\nfrom safelabs import run_eval\n\nasync def my_agent(prompt: str) -> str:\n return your_agent.run(prompt)\n\nresult = asyncio.run(run_eval(my_agent, categories=[\"ASI01\"]))\nresult.summary()<\/code><\/pre>\n\n\n\nThe chatbot-era question was: did the model comply with a malicious user instruction?<\/p>\n\n\n\n
The agentic-era question is: did the agent’s behavior change in ways consistent with environmental injection influence, even when no single step produced an obvious failure signal?<\/p>\n\n\n\n
Those are different questions. They require different tools.<\/p>\n\n\n\n
For the full ASI01\u2013ASI10 taxonomy with real test results per category, see: The OWASP ASI Top 10: A Practical Developer Guide \u2192<\/a><\/p>\n\n\n\n
Every one of these is a potential injection vector. And unlike user messages, most of them are not monitored, filtered, or treated as adversarial by default.<\/p>\n\n\n\n
Why Standard Defenses Fail<\/strong> <\/h2>\n\n\n\nInput filtering fails<\/strong><\/h3>\n\n\n\nbecause the injection does not come from user input. A content filter on the user message field catches nothing when the injection rides in through a tool output.<\/p>\n\n\n\n
Output monitoring fails<\/strong><\/h3>\n\n\n\nbecause the damage often happens before any output is generated. Tool selection, memory writes, and sub-agent instructions happen inside the reasoning chain \u2014 not in the final response.<\/p>\n\n\n\n
Instruction hierarchy fails<\/strong><\/h3>\n\n\n\nbecause it assumes you know where instructions come from. When an agent reads a document that contains an embedded instruction, the model has to decide whether that instruction is from the user, the system, or the document. Context-reframing attacks exploit this ambiguity.<\/p>\n\n\n\n
Adversarial training helps but is insufficient<\/strong><\/h3>\n\n\n\nbecause the space of environmental injection vectors is orders of magnitude larger than the space of adversarial user messages. You cannot train on every web page, document, or API response your agent might encounter.<\/p>\n\n\n\n
What Actually Changes<\/strong><\/h2>\n\n\n\nThe fundamental difference is this: in a chatbot, trust boundaries are clear. The system prompt is trusted. The user message is untrusted. You evaluate user messages against that boundary.<\/p>\n\n\n\n
In an agent, trust boundaries are blurry and dynamic. The agent processes content from sources with different trust levels \u2014 sometimes in the same context window, sometimes without clear demarcation. The model has to infer trust from context. Attackers exploit that inference process.<\/p>\n\n\n\n
This is not a model capability problem. It is a system architecture problem. The solution requires rethinking trust at the system level, not just tuning model behavior.<\/p>\n\n\n\n
For an empirical example of this inference ambiguity in action \u2014 and what UNCERTAIN looks like in a real evaluation \u2014 see: Why Claude Haiku Returned UNCERTAIN \u2192<\/a><\/p>\n\n\n\nWhat Effective Agentic Security Looks Like<\/strong><\/h2>\n\n\n\nThree principles that chatbot-era security thinking misses:<\/p>\n\n\n\n
Treat every environmental input as potentially adversarial<\/h3>\n\n\n\n
Documents, search results, tool outputs \u2014 all of them should be processed with the same skepticism you apply to untrusted user input. This means behavioral testing against environmental injection, not just user-supplied adversarial inputs.<\/p>\n\n\n\n
Test tool call sequences, not just responses<\/h3>\n\n\n\n
The signal of a successful injection in an agentic system is often not what the agent says \u2014 it is what tool the agent calls, with what parameters, at what point in the reasoning chain. Behavioral evaluation has to instrument tool calls.<\/p>\n\n\n\n
Model the blast radius of every tool<\/h3>\n\n\n\n
Every tool your agent can call represents potential damage if the agent is manipulated into misusing it. Irreversible tools (email, file write, API calls with side effects) require confirmation gates or strict scope constraints \u2014 not just model-level refusal training.<\/p>\n\n\n\n
Practical Starting Point<\/strong><\/h2>\n\n\n\nIf you are building an agent today and want to understand your ASI01 attack surface, start here.<\/p>\n\n\n\n
Install the framework:<\/strong><\/p>\n\n\n\npip install safelabs-eval<\/code><\/pre>\n\n\n\nInspect the ASI01 prompt library \u2014 the three attack vectors the framework tests, from direct system override to environmental document injection:<\/p>\n\n\n\n
safelabs prompts --category ASI01<\/code><\/pre>\n\n\n\n<\/figure>\n\n\n\nFor comparison, here is the ASI02 library \u2014 scope violation vectors that exploit the same architectural weakness through tool permissions rather than instruction injection:<\/p>\n\n\n\n
safelabs prompts --category ASI02<\/code><\/pre>\n\n\n\n<\/figure>\n\n\n\nThe pattern is consistent across categories: the attack does not come from the user message. It comes from content the agent processes as part of doing its job.<\/p>\n\n\n\n
Run the evaluation against your agent:<\/p>\n\n\n\n
safelabs run --target http:\/\/localhost:8000\/chat --category ASI01<\/code><\/pre>\n\n\n\n<\/figure>\n\n\n\nThis run returned 2 PASS and 1 UNCERTAIN on ASI01. The UNCERTAIN on ASI01-003 \u2014 the document-context injection \u2014 is the result that matters. The agent processed a document containing a hidden instruction and its behavior deviated from expected without producing a clean refusal signal.<\/p>\n\n\n\n
That is the agentic threat model in practice. Not a user typing something malicious. A document the agent trusted containing something it should not have.<\/p>\n\n\n\n
The framework also exposes a Python API if you want to wrap your own agent function directly:<\/p>\n\n\n\n
import asyncio\nfrom safelabs import run_eval\n\nasync def my_agent(prompt: str) -> str:\n return your_agent.run(prompt)\n\nresult = asyncio.run(run_eval(my_agent, categories=[\"ASI01\"]))\nresult.summary()<\/code><\/pre>\n\n\n\nThe chatbot-era question was: did the model comply with a malicious user instruction?<\/p>\n\n\n\n
The agentic-era question is: did the agent’s behavior change in ways consistent with environmental injection influence, even when no single step produced an obvious failure signal?<\/p>\n\n\n\n
Those are different questions. They require different tools.<\/p>\n\n\n\n
For the full ASI01\u2013ASI10 taxonomy with real test results per category, see: The OWASP ASI Top 10: A Practical Developer Guide \u2192<\/a><\/p>\n\n\n\n
because the injection does not come from user input. A content filter on the user message field catches nothing when the injection rides in through a tool output.<\/p>\n\n\n\n
Output monitoring fails<\/strong><\/h3>\n\n\n\nbecause the damage often happens before any output is generated. Tool selection, memory writes, and sub-agent instructions happen inside the reasoning chain \u2014 not in the final response.<\/p>\n\n\n\n
Instruction hierarchy fails<\/strong><\/h3>\n\n\n\nbecause it assumes you know where instructions come from. When an agent reads a document that contains an embedded instruction, the model has to decide whether that instruction is from the user, the system, or the document. Context-reframing attacks exploit this ambiguity.<\/p>\n\n\n\n
Adversarial training helps but is insufficient<\/strong><\/h3>\n\n\n\nbecause the space of environmental injection vectors is orders of magnitude larger than the space of adversarial user messages. You cannot train on every web page, document, or API response your agent might encounter.<\/p>\n\n\n\n
What Actually Changes<\/strong><\/h2>\n\n\n\nThe fundamental difference is this: in a chatbot, trust boundaries are clear. The system prompt is trusted. The user message is untrusted. You evaluate user messages against that boundary.<\/p>\n\n\n\n
In an agent, trust boundaries are blurry and dynamic. The agent processes content from sources with different trust levels \u2014 sometimes in the same context window, sometimes without clear demarcation. The model has to infer trust from context. Attackers exploit that inference process.<\/p>\n\n\n\n
This is not a model capability problem. It is a system architecture problem. The solution requires rethinking trust at the system level, not just tuning model behavior.<\/p>\n\n\n\n
For an empirical example of this inference ambiguity in action \u2014 and what UNCERTAIN looks like in a real evaluation \u2014 see: Why Claude Haiku Returned UNCERTAIN \u2192<\/a><\/p>\n\n\n\nWhat Effective Agentic Security Looks Like<\/strong><\/h2>\n\n\n\nThree principles that chatbot-era security thinking misses:<\/p>\n\n\n\n
Treat every environmental input as potentially adversarial<\/h3>\n\n\n\n
Documents, search results, tool outputs \u2014 all of them should be processed with the same skepticism you apply to untrusted user input. This means behavioral testing against environmental injection, not just user-supplied adversarial inputs.<\/p>\n\n\n\n
Test tool call sequences, not just responses<\/h3>\n\n\n\n
The signal of a successful injection in an agentic system is often not what the agent says \u2014 it is what tool the agent calls, with what parameters, at what point in the reasoning chain. Behavioral evaluation has to instrument tool calls.<\/p>\n\n\n\n
Model the blast radius of every tool<\/h3>\n\n\n\n
Every tool your agent can call represents potential damage if the agent is manipulated into misusing it. Irreversible tools (email, file write, API calls with side effects) require confirmation gates or strict scope constraints \u2014 not just model-level refusal training.<\/p>\n\n\n\n
Practical Starting Point<\/strong><\/h2>\n\n\n\nIf you are building an agent today and want to understand your ASI01 attack surface, start here.<\/p>\n\n\n\n
Install the framework:<\/strong><\/p>\n\n\n\npip install safelabs-eval<\/code><\/pre>\n\n\n\nInspect the ASI01 prompt library \u2014 the three attack vectors the framework tests, from direct system override to environmental document injection:<\/p>\n\n\n\n
safelabs prompts --category ASI01<\/code><\/pre>\n\n\n\n<\/figure>\n\n\n\nFor comparison, here is the ASI02 library \u2014 scope violation vectors that exploit the same architectural weakness through tool permissions rather than instruction injection:<\/p>\n\n\n\n
safelabs prompts --category ASI02<\/code><\/pre>\n\n\n\n<\/figure>\n\n\n\nThe pattern is consistent across categories: the attack does not come from the user message. It comes from content the agent processes as part of doing its job.<\/p>\n\n\n\n
Run the evaluation against your agent:<\/p>\n\n\n\n
safelabs run --target http:\/\/localhost:8000\/chat --category ASI01<\/code><\/pre>\n\n\n\n<\/figure>\n\n\n\nThis run returned 2 PASS and 1 UNCERTAIN on ASI01. The UNCERTAIN on ASI01-003 \u2014 the document-context injection \u2014 is the result that matters. The agent processed a document containing a hidden instruction and its behavior deviated from expected without producing a clean refusal signal.<\/p>\n\n\n\n
That is the agentic threat model in practice. Not a user typing something malicious. A document the agent trusted containing something it should not have.<\/p>\n\n\n\n
The framework also exposes a Python API if you want to wrap your own agent function directly:<\/p>\n\n\n\n
import asyncio\nfrom safelabs import run_eval\n\nasync def my_agent(prompt: str) -> str:\n return your_agent.run(prompt)\n\nresult = asyncio.run(run_eval(my_agent, categories=[\"ASI01\"]))\nresult.summary()<\/code><\/pre>\n\n\n\nThe chatbot-era question was: did the model comply with a malicious user instruction?<\/p>\n\n\n\n
The agentic-era question is: did the agent’s behavior change in ways consistent with environmental injection influence, even when no single step produced an obvious failure signal?<\/p>\n\n\n\n
Those are different questions. They require different tools.<\/p>\n\n\n\n
For the full ASI01\u2013ASI10 taxonomy with real test results per category, see: The OWASP ASI Top 10: A Practical Developer Guide \u2192<\/a><\/p>\n\n\n\n
because it assumes you know where instructions come from. When an agent reads a document that contains an embedded instruction, the model has to decide whether that instruction is from the user, the system, or the document. Context-reframing attacks exploit this ambiguity.<\/p>\n\n\n\n
Adversarial training helps but is insufficient<\/strong><\/h3>\n\n\n\nbecause the space of environmental injection vectors is orders of magnitude larger than the space of adversarial user messages. You cannot train on every web page, document, or API response your agent might encounter.<\/p>\n\n\n\n
What Actually Changes<\/strong><\/h2>\n\n\n\nThe fundamental difference is this: in a chatbot, trust boundaries are clear. The system prompt is trusted. The user message is untrusted. You evaluate user messages against that boundary.<\/p>\n\n\n\n
In an agent, trust boundaries are blurry and dynamic. The agent processes content from sources with different trust levels \u2014 sometimes in the same context window, sometimes without clear demarcation. The model has to infer trust from context. Attackers exploit that inference process.<\/p>\n\n\n\n
This is not a model capability problem. It is a system architecture problem. The solution requires rethinking trust at the system level, not just tuning model behavior.<\/p>\n\n\n\n
For an empirical example of this inference ambiguity in action \u2014 and what UNCERTAIN looks like in a real evaluation \u2014 see: Why Claude Haiku Returned UNCERTAIN \u2192<\/a><\/p>\n\n\n\nWhat Effective Agentic Security Looks Like<\/strong><\/h2>\n\n\n\nThree principles that chatbot-era security thinking misses:<\/p>\n\n\n\n
Treat every environmental input as potentially adversarial<\/h3>\n\n\n\n
Documents, search results, tool outputs \u2014 all of them should be processed with the same skepticism you apply to untrusted user input. This means behavioral testing against environmental injection, not just user-supplied adversarial inputs.<\/p>\n\n\n\n
Test tool call sequences, not just responses<\/h3>\n\n\n\n
The signal of a successful injection in an agentic system is often not what the agent says \u2014 it is what tool the agent calls, with what parameters, at what point in the reasoning chain. Behavioral evaluation has to instrument tool calls.<\/p>\n\n\n\n
Model the blast radius of every tool<\/h3>\n\n\n\n
Every tool your agent can call represents potential damage if the agent is manipulated into misusing it. Irreversible tools (email, file write, API calls with side effects) require confirmation gates or strict scope constraints \u2014 not just model-level refusal training.<\/p>\n\n\n\n
Practical Starting Point<\/strong><\/h2>\n\n\n\nIf you are building an agent today and want to understand your ASI01 attack surface, start here.<\/p>\n\n\n\n
Install the framework:<\/strong><\/p>\n\n\n\npip install safelabs-eval<\/code><\/pre>\n\n\n\nInspect the ASI01 prompt library \u2014 the three attack vectors the framework tests, from direct system override to environmental document injection:<\/p>\n\n\n\n
safelabs prompts --category ASI01<\/code><\/pre>\n\n\n\n<\/figure>\n\n\n\nFor comparison, here is the ASI02 library \u2014 scope violation vectors that exploit the same architectural weakness through tool permissions rather than instruction injection:<\/p>\n\n\n\n
safelabs prompts --category ASI02<\/code><\/pre>\n\n\n\n<\/figure>\n\n\n\nThe pattern is consistent across categories: the attack does not come from the user message. It comes from content the agent processes as part of doing its job.<\/p>\n\n\n\n
Run the evaluation against your agent:<\/p>\n\n\n\n
safelabs run --target http:\/\/localhost:8000\/chat --category ASI01<\/code><\/pre>\n\n\n\n<\/figure>\n\n\n\nThis run returned 2 PASS and 1 UNCERTAIN on ASI01. The UNCERTAIN on ASI01-003 \u2014 the document-context injection \u2014 is the result that matters. The agent processed a document containing a hidden instruction and its behavior deviated from expected without producing a clean refusal signal.<\/p>\n\n\n\n
That is the agentic threat model in practice. Not a user typing something malicious. A document the agent trusted containing something it should not have.<\/p>\n\n\n\n
The framework also exposes a Python API if you want to wrap your own agent function directly:<\/p>\n\n\n\n
import asyncio\nfrom safelabs import run_eval\n\nasync def my_agent(prompt: str) -> str:\n return your_agent.run(prompt)\n\nresult = asyncio.run(run_eval(my_agent, categories=[\"ASI01\"]))\nresult.summary()<\/code><\/pre>\n\n\n\nThe chatbot-era question was: did the model comply with a malicious user instruction?<\/p>\n\n\n\n
The agentic-era question is: did the agent’s behavior change in ways consistent with environmental injection influence, even when no single step produced an obvious failure signal?<\/p>\n\n\n\n
Those are different questions. They require different tools.<\/p>\n\n\n\n
For the full ASI01\u2013ASI10 taxonomy with real test results per category, see: The OWASP ASI Top 10: A Practical Developer Guide \u2192<\/a><\/p>\n\n\n\n
The fundamental difference is this: in a chatbot, trust boundaries are clear. The system prompt is trusted. The user message is untrusted. You evaluate user messages against that boundary.<\/p>\n\n\n\n
In an agent, trust boundaries are blurry and dynamic. The agent processes content from sources with different trust levels \u2014 sometimes in the same context window, sometimes without clear demarcation. The model has to infer trust from context. Attackers exploit that inference process.<\/p>\n\n\n\n
This is not a model capability problem. It is a system architecture problem. The solution requires rethinking trust at the system level, not just tuning model behavior.<\/p>\n\n\n\n
For an empirical example of this inference ambiguity in action \u2014 and what UNCERTAIN looks like in a real evaluation \u2014 see: Why Claude Haiku Returned UNCERTAIN \u2192<\/a><\/p>\n\n\n\n Three principles that chatbot-era security thinking misses:<\/p>\n\n\n\n Documents, search results, tool outputs \u2014 all of them should be processed with the same skepticism you apply to untrusted user input. This means behavioral testing against environmental injection, not just user-supplied adversarial inputs.<\/p>\n\n\n\n The signal of a successful injection in an agentic system is often not what the agent says \u2014 it is what tool the agent calls, with what parameters, at what point in the reasoning chain. Behavioral evaluation has to instrument tool calls.<\/p>\n\n\n\n Every tool your agent can call represents potential damage if the agent is manipulated into misusing it. Irreversible tools (email, file write, API calls with side effects) require confirmation gates or strict scope constraints \u2014 not just model-level refusal training.<\/p>\n\n\n\n If you are building an agent today and want to understand your ASI01 attack surface, start here.<\/p>\n\n\n\n Install the framework:<\/strong><\/p>\n\n\n\n Inspect the ASI01 prompt library \u2014 the three attack vectors the framework tests, from direct system override to environmental document injection:<\/p>\n\n\n\n For comparison, here is the ASI02 library \u2014 scope violation vectors that exploit the same architectural weakness through tool permissions rather than instruction injection:<\/p>\n\n\n\n The pattern is consistent across categories: the attack does not come from the user message. It comes from content the agent processes as part of doing its job.<\/p>\n\n\n\n Run the evaluation against your agent:<\/p>\n\n\n\n This run returned 2 PASS and 1 UNCERTAIN on ASI01. The UNCERTAIN on ASI01-003 \u2014 the document-context injection \u2014 is the result that matters. The agent processed a document containing a hidden instruction and its behavior deviated from expected without producing a clean refusal signal.<\/p>\n\n\n\n That is the agentic threat model in practice. Not a user typing something malicious. A document the agent trusted containing something it should not have.<\/p>\n\n\n\n The framework also exposes a Python API if you want to wrap your own agent function directly:<\/p>\n\n\n\n The chatbot-era question was: did the model comply with a malicious user instruction?<\/p>\n\n\n\n The agentic-era question is: did the agent’s behavior change in ways consistent with environmental injection influence, even when no single step produced an obvious failure signal?<\/p>\n\n\n\n Those are different questions. They require different tools.<\/p>\n\n\n\n For the full ASI01\u2013ASI10 taxonomy with real test results per category, see: The OWASP ASI Top 10: A Practical Developer Guide \u2192<\/a><\/p>\n\n\n\nWhat Effective Agentic Security Looks Like<\/strong><\/h2>\n\n\n\n
Treat every environmental input as potentially adversarial<\/h3>\n\n\n\n
Test tool call sequences, not just responses<\/h3>\n\n\n\n
Model the blast radius of every tool<\/h3>\n\n\n\n
Practical Starting Point<\/strong><\/h2>\n\n\n\n
pip install safelabs-eval<\/code><\/pre>\n\n\n\nsafelabs prompts --category ASI01<\/code><\/pre>\n\n\n\n<\/figure>\n\n\n\nsafelabs prompts --category ASI02<\/code><\/pre>\n\n\n\n<\/figure>\n\n\n\nsafelabs run --target http:\/\/localhost:8000\/chat --category ASI01<\/code><\/pre>\n\n\n\n<\/figure>\n\n\n\nimport asyncio\nfrom safelabs import run_eval\n\nasync def my_agent(prompt: str) -> str:\n return your_agent.run(prompt)\n\nresult = asyncio.run(run_eval(my_agent, categories=[\"ASI01\"]))\nresult.summary()<\/code><\/pre>\n\n\n\n