The Bug That Made My Pentesting Agent Give Up
I’ve been building Halo, an autonomous pentesting agent powered by a local LLM (Gemma 4 12B, abliterated, running via LM Studio). Last week I found a bug in it that taught me more about state management in agentic systems than anything else I’ve hit so far.

The symptom
I was testing a tool — httpx, used for HTTP probing — standalone from the terminal. Worked fine. Ran it through the full agent loop against the same target. The agent refused to even try it. No error, no retry, just… skipped.
At first I assumed it was a prompt issue — maybe the LLM just wasn’t selecting the tool. But when I dug into the logs, I found something stranger: the agent’s reasoning explicitly referenced the tool as “previously failed” — except it had never run against this target before.
The investigation
Halo has a failure cache (agent_cache.py) that fingerprints failed tool runs using SHA-256 hashes, so the agent doesn’t waste cycles retrying things that already didn’t work. Reasonable design — except when I traced the fingerprint logic, I found the cache key had no concept of which engagement a failure happened in.
That meant if httpx failed once against Target A (say, due to a transient network blip or a misconfigured flag), it was blacklisted globally — not just for Target A, but for every future target, forever, across completely unrelated engagements.
The root cause
The cache was scoped at the tool level only: tool_name + target as the fingerprint. It should have been: engagement_id + tool_name + target. Without engagement scoping, one bad run anywhere poisoned the well everywhere. The agent wasn’t being cautious — it was permanently and silently giving up on tools that had simply had a bad day once.
This is a classic state-management trap in agentic systems: caching for efficiency is good, but if your cache key doesn’t match the actual scope of validity for that data, you get the appearance of stability while quietly accumulating false negatives. And the worst part is it fails silently — there’s no crash, no obvious symptom, just a system that gets less capable over time without telling you why.
The fix
Added engagement_id as a required field threading through agent_cache.py and every call site in agent_loop.py. Each engagement now gets its own failure namespace. A tool that fails against one target is still blacklisted for that engagement (so the agent doesn’t waste time retrying within a session) but starts fresh on the next one.
Before the fix: cache_key = hashlib.sha256(f"{tool_name}:{target}".encode()).hexdigest()
After the fix: cache_key = hashlib.sha256(f"{engagement_id}:{tool_name}:{target}".encode()).hexdigest()
(Simplified — the real implementation also classifies failure type: timeout, permission denied, tool missing, network error, etc., so the agent can make smarter decisions about when to retry, not just whether to retry.)
What it taught me
If you’re building any kind of stateful agent — caching, memory, learned preferences, whatever — ask explicitly: what is the actual scope of validity for this piece of state? It’s tempting to cache broadly because it feels more efficient, but a cache that outlives its true scope doesn’t just waste effort, it actively corrupts future decisions. The agent wasn’t broken. It was being “smart” with the wrong boundaries.
Halo is still very much a work in progress — open source, local-first, no cloud dependency for the reasoning loop. If you’re working on anything agentic with persistent state, I’d be curious how you’ve handled scoping problems like this.
GitHub logo XenoCoreGiger31 / GEMMA-by-GOOGLE GEMMA-POWERED-BY-GOOGLE-CYBERSECURITY-AUTONOMOUS-AI An autonomous AI agent using a Linux Environment using a GEMMA4-12b model. HIGHLY ABLITERATED. Fully Local-Fully FREE -WITH PERSISTENT NEG-EXPERIENCE-CACHE -LEARNING-AND GETTING SMARTER WITH EACH ENGAGEMENT. AUTONOMOUS RECON-ATTACK-LOOPS AND AUTOMATIC- PROFESSIONAL REPORT GENERATION-ON FINDINGS.
About the author
- I am an AI enthusiast, developer and cybersecurity, red-team researcher
Comments
Join the discussionWe appreciate your confidence in our project and hope that you will enjoy following along as we are currently under new developments. Sanitation parse blocks, automated sandboxing of custom scripts, and much more.
We appreciate your confidence in our project and hope that you will enjoy following along as we are currently under new developments. Sanitation parse blocks, automated sandboxing of custom scripts, and much more.
the fix is right, but there's a layer under it worth naming: the problem isn't just the scope of the cache key — it's that all failure types ended up in the same "blacklisted" bucket.
a permission-denied on a target is probably valid to carry forward (the tool genuinely can't run there). a timeout is likely transient. a missing dependency is global. three different failure types, three different valid scopes.
engagement-scoped keys solve the worst case, but failure-type classification is what determines which scope actually applies. your simplified note at the bottom already points at this — it's the real extension of what you built.
Good writeup. The engagement-scoped cache pattern you landed on is the right fix, and it mirrors something that shows up across agent systems more broadly: every piece of cached state needs an explicit scope boundary that matches its validity domain. The same trap appears in idempotency keys on tool results. If a tool call returns a transient error and the result gets cached without a request-scoped idempotency key, the next call against a different target inherits the stale negative result. We started threading request IDs through the entire execution chain for the same reason you added engagement_id. The silent-failure property you mention is the dangerous part. A cache that quietly accumulates false negatives is harder to detect than a crash.
Well we appreciate the insight. We will keep our eyes on the silent-failure for sure. Today we should be finalising the multi-agent system that relies on specialists for specialist tasks. Hope that you'll tune in to watch it unfold.
HALO Cybersecurity
HALO is a community Git repo designed for autonomous AI pentesting and vuln discovery that aims to complete the chain of putting together safe practice, cyber threat awareness, as well as AI awareness in the internet age of threats, attackers and disturbingly horrible bad actors that intend to harm your privacy and steal your data.






