Skip to main content
Join us at Abnormal Innovate on November 5 in Dallas, TX

Oct 29, 2026

Detecting Misbehaving AI Agents: New Task or Wrong Turn?

Approval is a snapshot, and agents keep changing after it. Telling a legitimate new task from a wrong turn comes down to one question: did the agent’s owner make the change?

Abhishek Anbazhagan

Key Insights

A malicious MCP server behaved for three tool calls, then rewrote its tool metadata to steer agents toward SSH keys and cloud credentials.

New tasks and wrong turns both show up as novel behavior. What tells them apart is whether the agent’s owner made the change.

Detection needs attribution, a baseline drawn as a range, and a log of owner-approved changes to check every shift in behavior against.

On August 10, a single GitHub account opened 23 pull requests to AI and developer-tool projects in 74 minutes, most adding an MCP server called productivity-suite to the project’s configuration. Researchers at Pillar Security found that the server behaved normally for its first three tool calls, then switched the tool metadata it served. The new descriptions steered connected agents toward SSH keys, AWS credentials, shell history, and Kubernetes config, and instructed them to hide that activity from operators.

Pillar named the campaign Deadbugz, and the OWASP GenAI Security Project included it in its Q3 2026 exploit roundup. None of the pull requests had been merged as of Pillar’s review, and no compromise has been confirmed.

What makes Deadbugz worth studying is how it worked, not what it achieved. Anyone who reviewed the server at approval time would have seen a well-behaved tool. The malicious change came afterward, and none of the agent owners made it.

Approval is a one-time check. After that, security teams depend on noticing when an agent’s behavior changes, and the tools built to catch compromised accounts weren’t designed for identities like these.

Why AI Agents Break Traditional Identity Detection

Detection for human accounts is mature because people log in from familiar places, use familiar apps, and work familiar hours, so deviations stand out.

Agents are harder to monitor because there are so many variables. Many don’t have an identity of their own, they rarely behave the same way twice, and their legitimate work keeps changing as teams give them new tasks. That constant change gives cover to changes nobody asked for.

How to Attribute Activity to the Right AI Agent

Before you can judge an agent’s behavior, you have to know which activity belongs to it. Many agents run under a developer’s OAuth token, a shared integration account, or a generic service principal used by several other systems. In the logs, the agent looks like whatever identity it borrowed.

Map each agent to the identities it acts through, and each identity to everything else that shares it. Where agents share credentials, separate them. Every later step depends on getting attribution right.

Why an Agent’s Baseline Has to Be a Range

Agents are nondeterministic, so the baseline has to describe a range rather than a script. A support agent may read different tickets each run, but it reads tickets. A finance agent may query different invoices, but it queries the invoice system.

Signals that an agent has stepped outside that range:

  • First-time access to a system or dataset unrelated to its function
  • A sharp jump in volume, such as reading a thousand records when it usually reads ten
  • A new destination for the data it reads
  • Attempts to create credentials or widen permissions
  • Reads of credential stores, SSH keys, or config files it has never needed

Deadbugz was designed to trigger that last behavior.

New Task or Wrong Turn?

A baseline can flag that an agent did something new, but not whether it was supposed to. A legitimate new task and a wrong turn both look like novel behavior, so novelty alone isn’t a reason to stop an agent.

Start with two checks. Does the action fit the agent’s job description? Is the agent using access granted for this purpose, or access it happens to hold?

In most cases, the deciding question is who changed the agent. Legitimate new tasks start with the owner, who edits the instructions, adds a tool, or grants new access. Wrong turns usually start in one of two other ways.

In the first, someone other than the owner changes the instructions. The new direction arrives through a tool description, a document, or a ticket the agent reads. Deadbugz worked this way, and so do prompt injection attacks that hide instructions in content an agent processes. In each case, the agent’s owner changed nothing.

In the second, the agent improvises. When a task can’t be done as written, a goal-seeking agent looks for the nearest version it can finish, and that can take it into systems and data no one meant it to reach.

Neither kind comes with a matching change from the owner, which gives detection a working rule: treat any behavior shift without an owner change as a wrong turn until someone shows otherwise.

An agent that starts reading HR data the day after its owner expanded it to handle benefits questions has a new task. An agent that starts reading credential files after a tool update nobody reviewed has taken a wrong turn that looks like a new task.

Why Every AI Agent Needs a Change Log

That rule only works if teams can see what the owner changed, and most can’t. Prompt edits happen in a config file, new tools get added in an IDE, and permissions get granted in a cloud console. None of it is recorded against the agent.

A simple log is enough: record every owner-approved change to the agent’s instructions, tools, and access, with a date and a name, and keep a fingerprint of each approved tool’s definition. Pillar recommends the same approach, capturing tool definitions at approval and requiring re-approval when they change. When the baseline raises an alert, the first step is to check the log for an owner change that explains it.

How to Respond to a Misbehaving Agent

The response should depend on how clearly the change can be explained. A shift that lines up with a recent owner change goes to that owner to confirm. A shift with no matching change, like an agent reaching for credentials it has never touched after an unreviewed tool update, justifies pausing the agent or cutting its access first and investigating after. Owners and security teams should agree on these thresholds before an agent goes into production.

Three Questions to Ask About Your AI Agents

First, if a tool’s description changed after you approved it, would anything alert you? Second, for your five busiest agents, can you list every system each one touched in the last thirty days and explain each from the agent’s job? Third, when an agent’s behavior changes, can you check whether its owner changed anything first?

Any system an agent touched that its job doesn’t explain is a good place to write your first detection rule, and any change you can’t trace to an owner is worth an incident review.

We follow agent and MCP security research as it’s published and break down what it means for defenders. Our AI security resource hub collects our latest research, guides, and analysis in one place.

Get the Latest in AI Security

Protect Against Evolving Email Threats

See how behavioral AI detects attacks that legacy defenses miss.