How to Adapt to AI Orchestrators for Penetration Testing: The Modern Security Model

With the advent of autonomous AI-driven penetration testing tools, the cybersecurity landscape is dramatically changing. Organizations must rethink their defensive strategies and adopt a new, comprehensive approach to protecting digital assets.

How to Adapt to AI Orchestrators for Penetration Testing: The Modern Security Model

In recent months, there has been a significant increase in the number of large language model (LLM)-powered penetration testing and orchestration frameworks. Just a few weeks ago, one well-known open-source agentic penetration testing system that automates offensive security workflows was available. Today, however, we are seeing dozens of families of these frameworks attempting to scan systems worldwide. The number of IP addresses from which these frameworks operate has grown into the thousands and continues to rise daily. The proliferation of these tools has significantly altered the state of cyber defense: defenders are struggling to keep pace with the volumetric nature of these autonomously driven attacks.

While the original project was withdrawn by its creator on October 8th, the ecosystem has grown with new, fragmented versions in the month since that event. Currently, there are over 12,000 repositories and hundreds of unique forks building on the original project. Additionally, hundreds more similar agentic orchestrators have emerged since October 8th. The daily number of requests for AI penetration testing and exploitation tools has increased thirtyfold since October 1st. These tools have also begun chaining stolen credentials within more complex attacks, such as lateral movement across an entire environment.

We anticipate that these frameworks will continue to proliferate, and every organization should be prepared to deal with this threat. Against LLM-based penetration testing frameworks, we recommend all organizations adopt what is known as the Modern Security Model, which rests on six key pillars.

Open-source penetration testing frameworks are capable of orchestrating AI agents to execute phases of the traditional cyber kill chain at scale. Some of these tools can rapidly integrate existing contextual model protocols and innovatively link traditional standalone penetration testing tools. Some even allow an operator to specify the type of task to be completed. These tools can map and enumerate environments, both internally and externally. These and similar tools map assets and environments onto a graph, enabling multiple independent, parallel agents to perform enumeration, validation, prototyping, and attacks at inference speed. In this way, compromises become a problem that can be effectively “climbed,” an area in which agents are particularly adept.

We are tracking dozens of AI LLM agents. Some of the most popular are fully autonomous. The fully autonomous nature of these agents represents a doctrinal shift in offensive cyber capabilities:

As of a certain date, the original platform and its forks constituted more than 60% of known AI penetration testing orchestrators. Since the withdrawal of the original repository in early October, traffic has increased more than tenfold, and even more than thirtyfold since the beginning of October. As previously stated, the withdrawal of the original project in October and the related news likely caused a “hydra effect,” where the removal of the original repository led to a massive surge in the number of forks and modifications, evidenced by a steep acceleration in the growth of unique instances around that date.

In recent weeks, incident response teams have been called to several investigations involving these and similar LLM-powered exploitation frameworks. We have witnessed firsthand how exposed credentials are being chained with LLM-based exploit frameworks to give attackers an advantage over defenders. We have recorded several incidents where an attacker gained initial access to a hyperscaler environment and subsequently moved directly into a customer environment. Root cause analysis revealed a common failure in credential hygiene. Attackers repeatedly compromised static, non-expiring tokens running in CI/CD pipelines that provided broad access across various environments within an organization's hyperscaler infrastructure. LLM frameworks significantly increase the likelihood that a client's API key will be misused to take control of other tenant environments.

During one incident investigation, our team observed a specific framework gaining access to a compromised account-level API token previously used for CI/CD without IP address restrictions or rate limits. Available evidence suggested that the compromise most likely originated from a third-party environment. We noted that a large number of new scripts were created with distinguishable names, referencing the framework in question. The names of individual scripts correlated with the distinct behaviors and actions they performed, often supplemented by an alphanumeric identifier. Activity exceeding 69 requests per minute was observed across various parts of the infrastructure, and 400 database export requests per hour. We also observed attempts at persistence mechanisms and efforts to validate session tokens for other third-party applications. While we believe much of this activity was lightly guided by a human operator, this is just one example of the speed and scale that these and similar frameworks can achieve once a compromise occurs.

Large language models enhance an attacker's ability to rapidly traverse the cyber kill chain against a target. This occurs because the attacker gains the advantage of speed and validation. Each step of the cyber kill chain can be validated and passed to the next phase of the attack. This level of orchestration is unprecedented and effectively devalues older cybersecurity models. To counter these attacks, a modern model must be employed. To better explain this model, let's use the metaphor of protecting a house on a property. Each subsequent section describes how to defend against the threat at each step.

Secure the Main Gate: Traffic Control and WAF with Advanced Rules
Before anyone enters your property, they must reach the perimeter fence. On an unprotected property, an adversary can send an endless fleet of decoy vehicles directly to your driveway, completely blocking entry and breaching barriers (volumetric Layer 3/4 floods and direct IP address probing). With a security system at the perimeter gate, all incoming traffic is routed through massive, distributed outer gates, far from the main house. Unauthorized deliveries are filtered on the highway, fraudulent service tunnels attempting to eavesdrop on your property are interrupted, and only legitimate foot traffic reaches your address. Network traffic management plays a crucial role in stopping volumetric attacks.

Lobby Security Checkpoint: Anomaly Detection
Once visitors arrive at the main entrance, they encounter a strict security checkpoint in the lobby. Automated intruders don't just kick down doors; they disguise themselves. They try odd ways to get in—stuffing paper into the lock, trying variations like //main-door//back-room/ to trick the mechanism into opening. The security guard at the door enforces clear rules: You cannot knock 500 times in five seconds. You cannot disguise your path with deception. You cannot attempt to push a massive cart through the front hall to remove internal house plans (memory diagnostic dumps). Suspicious behavior is immediately rejected. Implement security keys and FIDO-based hardware keys. A Web Application Firewall (WAF) serves as a key defensive mechanism in stopping application exploits through rate limiting and anomaly detection.

Identify Fraudulent Visitors: Implement Bot Defense
A sophisticated intruder rarely looks like a masked thief; they disguise themselves as ordinary visitors. But automated agents, headless scrapers, and AI reconnaissance tools do not behave like real humans. They do not blink, their posture is disturbingly rigid, their cadence is mechanical, and their “identification” is often a sloppy imitation of a genuine visitor pass. Bot Defense and challenges act as behavioral specialists, observing the crowd through one-way glass. They flag unnatural cadences, detect synthetic micro-behaviors, and quietly escort machines off the premises before they can touch a single doorknob.

Dominik Medal

Planning a new website or app?

Tell me what you have in mind — I will take a look and advise on the best way forward.

Remove Backdoors: Implement Identity-Driven Access and Microsegmentation
Many houses were historically built with side doors or basement hatches (internal admin panels, diagnostic endpoints, and remote gateways). Even if locked, an intruder could stand on the sidewalk, shine a flashlight on the basement door, and begin picking the lock. Zero Trust application access does something radical: it walls off the entrance from the outside and restricts which rooms can be entered. For anyone wandering the perimeter, the basement door doesn't exist. There's no lock to pick, no handle to rattle, and no light shining inside. The only way in is with an escort who checks your biometric ID, confirms your device is healthy, and takes you directly to the specific room you need—the rest of the house remains out of sight. For those who do enter, it is essential to ensure they cannot penetrate rooms adjacent to the one they entered. Microsegmentation is a highly effective strategy for stopping lateral movement. It limits the penetration of compromised credentials into production, testing, and development systems.

Control Deliveries: Enforce Validated Schemas and Explicit Security Checks
Deliveries—groceries, mail, spare parts—arrive at your property daily. In a digital sense, this is your incoming API traffic, which must be inspected and verified. Attackers often attempt to bypass checks by sending unvalidated payloads or falsified delivery manifests intended for restricted internal systems. A “porch protocol” mandates thorough inspection and schema validation for all traffic: Every incoming request must pass OpenAPI schema validation to ensure it strictly conforms to expected definitions and structures. Drivers and couriers must present validated mutual TLS (mTLS) certificates to establish trusted, authenticated connections. Any request that fails inspection or lacks valid credentials is immediately rejected at the perimeter. A Default-Deny Posture: Every positive security implementation concludes with an explicit rule to deny all at the end of the rule set.

Neighborhood Watch: Implement Threat Intelligence
Even with bulletproof walls, an isolated property can be vulnerable if a syndicate devises a brand-new infiltration technique on the other side of town. Threat Intelligence acts as your warning network with neighboring properties. When a rogue actor sets up a new secure command-and-control house nearby or begins deploying dynamic tunneling deliveries in your district, you receive timely information. Your property updates its watchlists, recalibrates its sensors, and slams the gate shut before the attacker even reaches your street.

These and similar large language model-based frameworks pose a significant challenge to older cyber defense models. Even the historical “defense-in-depth” model—which emphasizes a classic network DMZ—falls short in today's complex architectures. Tomorrow's attacks will only evolve as orchestration tools become the new standard. Securing legacy equipment and on-premises devices requires an identity-driven approach that logically prevents these devices from being reached from the internet. Beyond identity, the goal of microsegmentation is to slow down or increase the noise of lateral movement. This architecture emphasizes a “containment-first” approach to security, which is essential when users are constantly losing their keys. When combined, the elements of the Modern Security Model are the answer to this pressing problem.

Dominik Medal

Let's talk about your project

Tell me what you need and together we will work out the best way forward.

Stop scrolling, call me

+420 735 505 585