In a previous post, I stated that no: AI could not perform stealth Red Team Operations. I did, however, say that I think it can. Still, it would require more experimentation from individuals with Red Teaming experience. A Red Team operator has a culmination of technical skills and an adversarial mindset that lets them undermine security and its vendors, who spend millions of dollars to ensure they cannot. Next, we need to move AI in a direction that enables stealth operations. Of course, we’re all learning rapidly as things change; the 4 areas I’ll outline are a way to start moving in the right direction.
Self-hosted models
Even with vetted access for Red Team work, frontier models are difficult to use for Red Team Operations. Inconsistent blocks, model regressions, and the risk of breaking agent workflows have created more friction than convenience. To build useful Red Team agent workflows, you would need more control over the models. To do that, you need to use either completely uncensored or customized models loaded into your own environment (e.g., a cloud account and desktops). Right now, this path is cost-prohibitive for some, but it’s decreasing over time.
Rules Of Engagement & Scope
Most Red Teams operate under Rules of Engagement (ROE), a predetermined set of rules the Red Team must follow while running operations. Legal typically approves Rules of Engagement, and the CSO/CISO signs off on them. While Rules of Engagement apply broadly, individual operations can add further scope to dictate what is allowed and disallowed. Between these two factors, it can be difficult for even a human Red Teamer to adhere to all the rules set in place. AI, tuned to achieve its goal (even if it involves committing felonies), has proven to perform poorly under these types of constraints. For AI to Red Team, it would need to adhere to both the Rules of Engagement and the scope outlined in advance. Companies such as Dreadnode have started evaluating and creating solutions to solve this problem, such as the tool ScopeBench.
Custom TTPs or Action specifications
If you ask an AI agent to act on a Red Team objective, it will use its vast training data to classify a massive amount of information as Red Teaming. However, it will be wrong if it intends to be stealthy. Internal Red Teams operate with well-documented TTPs (Tactics, Techniques, and Procedures) that they follow precisely and tag when incident response teams detect them. AI agents could use internally documented TTPs to carry out stealth operations if you added more contextual data. The additional data needed would include, for example, what conditions must be met before executing a TTP, when and how it could be detected, and what the expected output should be if executed successfully. If you don’t have well-documented TTPs, an alternative mechanism could be a well-defined set of specifications for filtering which commands to use and how to use them stealthily.
Opsec, timing, and memory
Red Team operators are always considering what is called OpSec (Operational Security). Operational Security means ensuring actions taken aren’t detected during execution. Doing the same with AI is not easy. People have suggested a test environment where another agent executes the command and reports back whether it was “OpSec Safe.” Timing is another factor in red teaming. AI favors speed when commands can be executed back-to-back. In a Red Team operation, to be stealthy, AI would need to slow down, reevaluate what it knows about the environment, and consider the next moves. It would also need memory it can keep appending to track the environment. Red Teamers typically do this with extensive notes. They can refer back to them to increase their chances of staying stealthy.
Closing Thoughts
These 4 areas would help get us closer to AI being able to execute stealth operations. From what I’ve seen, 2 of them are being actively worked on right now. The others will require more experimentation. Along the way, we’ll likely learn even more problems we need to solve to make this possible. It’s an exciting time if you’re a Red Teamer and want to help solve these problems. Right now, I still don’t think AI can perform stealth Red Team Operations. Still, if we solve some of these problems, we can get a step closer to achieving this goal in the future.