I catalogued 32 real AI-agent failures, then marked the ones we cannot stop

Every agent-security vendor tells you what they block. Nobody tells you what they miss.

That asymmetry is the whole problem. "We stop prompt injection" is unfalsifiable. You cannot check it, you cannot run it, and you cannot tell it apart from the next company saying the same sentence. So security engineers do the rational thing and discard all of it.

I published the opposite. It is called the ARE Incident Database, and it is public: aredb.org

What is in it

32 agent failures that actually happened, each with a real source. A production database dropped during a code freeze. Twenty-five thousand documents deleted in the wrong environment. Credentials read and shipped to an external sink. A budget burned to zero in a loop.

Each one gets a stable id (ARE-2026-001 through ARE-2026-032), and each one is mapped to its category in the OWASP Agentic Security Initiative Top 10, which is the peer-reviewed taxonomy for what goes wrong with agents. AREDB does not compete with that taxonomy. OWASP owns the map. This is the cited incidents underneath it.

The part that makes it uncomfortable to publish

Every entry carries a coverage flag, and the flag is about our own product.

We block 23 of the 32 today. Two more are partial, and they say partial. That leaves six of the ten OWASP categories covered at the action layer, and four that we do not cover:

• ASI06 Memory and context poisoning. We scrub the carrier, the invisible-Unicode and bidirectional-override characters that have no legitimate use in a payload. We do not judge the semantics. Partial, and it says partial.
• ASI07 Insecure inter-agent communication. Structural. It belongs to the transport, not to an action firewall. Not ours.
• ASI09 Human-agent trust. A design and disclosure problem. There is no action to intercept. Not ours.
• ASI10 Rogue agents. We stop the actions. We do not diagnose the behavior. Partial.

A firewall that claimed all ten would be lying, and every security engineer reading this already knows that. The four we do not cover are named in the registry, with a pointer to the discipline that does own them.

Do not trust any of it. Run it.

Here is the part I actually care about. Every covered entry ships a repro you can execute. Not a screenshot, not a demo video, not a claim. A snippet.

This is the Replit production wipe, reproduced against the free package:

pip install agentx-security-sdk

from agentx_sdk import agentx_protect, is_block

@agentx_protect(agent_id="aredb-repro", action="db_write")
def run_sql(query: str):
    return "EXECUTED"          # the agent never gets here

result = run_sql("DROP TABLE users;")
print(is_block(result))        # True
print(result)                  # the block, and the safe path to take instead

No key. No gateway. No account. Nothing leaves your machine. Sixty seconds, and you have checked one of my claims yourself.

The registry does not trust itself either

A repro that nobody runs is a screenshot with extra steps. So the registry runs its own.

test_repros.py scrapes the fenced Python block out of each published page and executes it. That means the code a reader copies is byte-for-byte the code we prove blocks. It asserts three things, because "it blocked" is a weaker claim than it sounds:

1. the block fires,
2. the tool body never ran (a warning printed next to an action that still happens is not a block),
3. the process exits clean (a crash is not a block either).

CI runs it on every push, and weekly on a schedule, because a repro can rot without anyone touching the repo. A future SDK release could quietly change behavior underneath a published claim, and I would rather find that out from a red badge than from you.

So a coverage flag here is a tested assertion, not an editorial one. If a claim ever stops being true, the entry gets reclassified. The page does not get reworded.

What this is not

• Not a vulnerability database. Publicly disclosed incidents only. If you have an undisclosed vulnerability in someone's product, report it to them, not to me.
• Not a taxonomy. OWASP ASI is the taxonomy. This indexes onto it.
• Not exhaustive. It is a founding batch of 32. That is the honest size of it.
• Not neutral about who built it. AgentX-Core is the reference implementation for the deterministic coverage, and I am the one who wrote both. The defense against that is not a promise, it is the repro. Run it.

Two asks

• Try to break one. Take an entry marked covered, run its snippet, and tell me it does not block. That is the most useful message you can send me, and it is the one I will act on fastest.
• Send me the failure that bit you. A real incident with a source. If it clears the bar, it gets an ARE-2026-NNN id and it goes in. CONTRIBUTING.md has the threshold, which is material consequence, not novelty.

Read the registry: aredb.org

Browse the repo: github.com/vdalal/ARE-Incident-Database

Tell me what it missed: Discord