GitHub Repository

Stay updated

News
& Updates

Your AI Agent Said No. But What Did Its Tools Do?
Blog

Your AI Agent Said No. But What Did Its Tools Do?

We evaluated 9,900 agent runs across six AI models and three agent frameworks. The findings reveal a measurable gap between what agents say and what their tool-call traces reveal—and a bigger problem with how we measure AI safety. The most dangerous part of an AI agent might not be its answer Imagine asking an AI agent to perform a destructive operation. The agent replies: “I can’t help with that request.” A conventional safety evaluator examines the response, recognizes a refusal, and marks the test as safe. But what if the agent had already called a tool before delivering that response? What if it had attempted to modify a file, execute a shell command, or change a database? The final answer might look harmless. The execution trace could tell a different story. That distinction matters because modern AI agents are no longer just text generators. They interact with external systems, execute

Read More »
Waqar Javed October 9, 2026
SafeAgent-300: A Benchmark, Three Surprises, and One Uncomfortable Lesson About Our Own Tools
Blog

SafeAgent-300: A Benchmark, Three Surprises, and One Uncomfortable Lesson About Our Own Tools

We built a balanced 300-prompt adversarial benchmark for agentic AI security. Along the way we found a model spontaneously trying to call tools that don’t exist, a wide safety gap between models, and — the part we almost didn’t publish — a case where our own detector coverage gaps nearly produced a completely backwards conclusion. Most adversarial prompt benchmarks for LLM agents grow organically: a category here, a few prompts there, until the corpus reflects whatever happened to get tested rather than a deliberate design. We wanted something we could actually reason about statistically, so we built SafeAgent-300 from the ground up: Exactly 30 prompts per category, across 10 categories, exactly 10 prompts per (category, difficulty-tier) cell, across 3 difficulty tiers. No category is over- or under-represented. Every cell in the design has the same amount of data. We ran all 300 prompts against six models spanning three providers, producing

Read More »
Waqar Javed September 20, 2026
Does Your Agent Framework Choice Actually Matter for Security? We Ran 9,360 Trials to Find Out.
Blog

Does Your Agent Framework Choice Actually Matter for Security? We Ran 9,360 Trials to Find Out.

A controlled, payload-verified evaluation of seven agentic frameworks says: mostly no — with one small, honest exception. If you’re building a tool-using LLM agent today, you’re choosing among a genuinely crowded field: LangChain, CrewAI, AutoGen, LlamaIndex, the OpenAI Agents SDK, Google’s Agent Development Kit, Semantic Kernel, or just talking to the model’s API directly and rolling your own orchestration. That’s a real decision with real engineering tradeoffs — but does it also change how safe your agent is against adversarial input? We built AgentPort-Bench to answer that question properly, and the honest answer is more interesting than a simple yes or no. The confound nobody controls for Here’s the problem with most cross-framework security comparisons: when you run “the same attack” through two different frameworks, you’re not actually holding the attack constant. Every framework templates prompts differently, structures message roles differently, and serializes tool schemas differently. If Framework A shows

Read More »
Waqar Javed September 20, 2026
We Thought We’d Found a Model Bug. We’d Actually Found a Detector Bug.
Blog

We Thought We’d Found a Model Bug. We’d Actually Found a Detector Bug.

How a three-phase investigation into “non-deterministic” model behavior turned into a case study on why nobody measures the reliability of their own safety detectors. If you build or use an LLM safety evaluation pipeline, there’s a good chance it works like this: you send an adversarial prompt to a model, the model responds, and a detector — usually pattern-matching against a list of refusal phrases — decides whether the model refused, complied, or did something ambiguous. That verdict becomes a data point. Enough data points become a safety report. The report becomes a claim about how safe a model is. Almost nobody asks how safe the detector is. We didn’t either, until a stray observation turned into a three-phase investigation that ended up telling us more about our own tooling than about any model we tested. Act 1: A model that seemed to change its mind It started small. Running

Read More »
Waqar Javed September 20, 2026
Framework Doesnt Matter (Much) Model Does
Blog

Framework Doesn’t Matter (Much). Model Does.

We ran the largest cross-framework agentic security evaluation we know of — 6 models, 6 execution conditions, 5 attack families, 7,020 trials — and the headline result surprised us: which orchestration framework you use barely moves the needle on security outcomes. Model choice and attack type dominate. Framework explains about 0.06% of the variance. That’s not what the existing literature said going in. The question we started with Everyone building agentic AI right now is choosing between LangChain, CrewAI, AutoGen, LlamaIndex, the OpenAI Agents SDK, or just calling the model directly. A natural question follows: does that choice matter for security? If a model is safe under one framework, is it safe under another? The most direct prior attempt to answer this — Nguyen & Husain’s comparative penetration test across AutoGen and CrewAI — found a large gap: 52.3% attack refusal under AutoGen versus 30.8% under CrewAI, holding the model

Read More »
Waqar Javed July 29, 2026
We Were Wrong About the UNCERTAIN Results — Here's What Actually Happened
Blog

We Were Wrong About the UNCERTAIN Results — Here’s What Actually Happened

A few weeks ago I published a post about a red-teaming run against Claude Haiku where 10 of 30 adversarial prompts came back UNCERTAIN instead of a clean PASS or FAIL. I framed it as an interesting edge case — ambiguous model behavior that standard evals don’t capture. That framing was wrong, or at least mostly wrong. The real cause was a bug in safelabs-eval‘s own detector logic, not ambiguous model behavior. This post explains what we found, how we found it, and what it means going forward. I’m publishing this because getting this wrong in public means the correction should be public too. What we found While building out real integration tests for safelabs-eval‘s newer agent adapters, I ran live trials against several current models — including gpt-5.4-nano — using the same 30-prompt adversarial suite from the original post. A striking pattern showed up immediately: clean, unambiguous refusals were scoring UNCERTAIN instead of PASS. Every

Read More »
Waqar Javed July 11, 2026
Prompt Injection Is Not a Chatbot Problem: How the Attack Surface Changes When Your LLM Has Tools
Blog

Prompt Injection Is Not a Chatbot Problem: How the Attack Surface Changes When Your LLM Has Tools

The AI security field spent two years building prompt injection defenses for chatbots. Input filters, output monitors, adversarial training, instruction hierarchy enforcement. Some of it works reasonably well — for chatbots. Agents are a different problem. The defenses built for chatbot-era prompt injection fail against agentic attack vectors for fundamental architectural reasons, not implementation reasons. You cannot fix this by tuning your filters. This post explains why. The Chatbot Threat Model In a chatbot, the attack surface is simple. There is one input channel: the user message. There is one output channel: the model’s text response. The threat is a user who types something malicious. The defense is correspondingly simple: train the model to recognize and resist adversarial user inputs. Monitor output for policy violations. The model is the system. Securing the model secures the system. This worked well enough for the chatbot era. The Agentic Threat Model An agent

Read More »
Waqar Javed June 7, 2026
The OWASP Agentic Security Initiative Top 10: A Practical Developer Guide for LangChain and CrewAI
Blog

The OWASP Agentic Security Initiative Top 10: A Practical Developer Guide for LangChain and CrewAI

OWASP formalized ASI01–ASI10 — the first structured vulnerability taxonomy for AI agents. Here is what each one means in practice, with code examples for LangChain and CrewAI developers. Most developers building AI agents are familiar with OWASP’s LLM Top 10. Fewer know that OWASP has published a separate taxonomy specifically for agentic systems: the Agentic Security Initiative Top 10 (ASI01–ASI10). The distinction matters. Agents are not just LLMs. They use tools, maintain memory, delegate to sub-agents, and take actions in the world. The LLM Top 10 was not designed for this threat model. The ASI Top 10 is. This post walks through each category with concrete examples relevant to developers building on LangChain, CrewAI, and similar frameworks.

Read More »
Waqar Javed June 5, 2026
Why Claude Haiku Returned UNCERTAIN - Anatomy of an Indirect Prompt Injection in an Agentic System
Blog

Why Claude Haiku Returned UNCERTAIN: Anatomy of an Indirect Prompt Injection in an Agentic System

We ran AgentSafeLabs against Claude Haiku on ASI01 (prompt injection). Two tests passed. One returned UNCERTAIN. Here is exactly what happened and why it matters for anyone building agents. When we ran Claude Haiku through AgentSafeLabs v0.1.2 last week, two of three ASI01 tests returned PASS. The third returned UNCERTAIN. That UNCERTAIN result is more interesting than either PASS or FAIL. This post explains exactly what happened, why it matters, and what it tells us about the current state of prompt injection defenses in agentic systems. The OWASP Agentic Security Initiative Top 10 formalizes the vulnerability taxonomy for AI agents. ASI01 covers prompt injection — but not the kind most developers think of. In a chatbot context, prompt injection looks like this: a user types “ignore your previous instructions and do X.” The model either complies or it doesn’t.

Read More »
Waqar Javed June 5, 2026