Author: Justin Tiplitsky

  • What Would AI Need to Red Team?

    In a previous post, I stated that no: AI could not perform stealth Red Team Operations. I did, however, say that I think it can. Still, it would require more experimentation from individuals with Red Teaming experience. A Red Team operator has a culmination of technical skills and an adversarial mindset that lets them undermine security and its vendors, who spend millions of dollars to ensure they cannot. Next, we need to move AI in a direction that enables stealth operations. Of course, we’re all learning rapidly as things change; the 4 areas I’ll outline are a way to start moving in the right direction.

    Self-hosted models

    Even with vetted access for Red Team work, frontier models are difficult to use for Red Team Operations. Inconsistent blocks, model regressions, and the risk of breaking agent workflows have created more friction than convenience. To build useful Red Team agent workflows, you would need more control over the models. To do that, you need to use either completely uncensored or customized models loaded into your own environment (e.g., a cloud account and desktops). Right now, this path is cost-prohibitive for some, but it’s decreasing over time.

    Rules Of Engagement & Scope

    Most Red Teams operate under Rules of Engagement (ROE), a predetermined set of rules the Red Team must follow while running operations. Legal typically approves Rules of Engagement, and the CSO/CISO signs off on them. While Rules of Engagement apply broadly, individual operations can add further scope to dictate what is allowed and disallowed. Between these two factors, it can be difficult for even a human Red Teamer to adhere to all the rules set in place. AI, tuned to achieve its goal (even if it involves committing felonies), has proven to perform poorly under these types of constraints. For AI to Red Team, it would need to adhere to both the Rules of Engagement and the scope outlined in advance. Companies such as Dreadnode have started evaluating and creating solutions to solve this problem, such as the tool ScopeBench.

    Custom TTPs or Action specifications

    If you ask an AI agent to act on a Red Team objective, it will use its vast training data to classify a massive amount of information as Red Teaming. However, it will be wrong if it intends to be stealthy. Internal Red Teams operate with well-documented TTPs (Tactics, Techniques, and Procedures) that they follow precisely and tag when incident response teams detect them. AI agents could use internally documented TTPs to carry out stealth operations if you added more contextual data. The additional data needed would include, for example, what conditions must be met before executing a TTP, when and how it could be detected, and what the expected output should be if executed successfully. If you don’t have well-documented TTPs, an alternative mechanism could be a well-defined set of specifications for filtering which commands to use and how to use them stealthily.

    Opsec, timing, and memory

    Red Team operators are always considering what is called OpSec (Operational Security). Operational Security means ensuring actions taken aren’t detected during execution. Doing the same with AI is not easy. People have suggested a test environment where another agent executes the command and reports back whether it was “OpSec Safe.” Timing is another factor in red teaming. AI favors speed when commands can be executed back-to-back. In a Red Team operation, to be stealthy, AI would need to slow down, reevaluate what it knows about the environment, and consider the next moves. It would also need memory it can keep appending to track the environment. Red Teamers typically do this with extensive notes. They can refer back to them to increase their chances of staying stealthy.

    Closing Thoughts

    These 4 areas would help get us closer to AI being able to execute stealth operations. From what I’ve seen, 2 of them are being actively worked on right now. The others will require more experimentation. Along the way, we’ll likely learn even more problems we need to solve to make this possible. It’s an exciting time if you’re a Red Teamer and want to help solve these problems. Right now, I still don’t think AI can perform stealth Red Team Operations. Still, if we solve some of these problems, we can get a step closer to achieving this goal in the future.

  • Choosing to be a Sitting Duck

    Organizations have realized greater profits by doing business online for decades. Doing business online comes with the cost of operating in a landscape where threat actors exist. Those threat actors also keep improving their ability to profit, thrive, and achieve their objectives at your expense. The cost of a data breach has risen in the US to 10.22 million, and it’s been reported that Cybercrime (Yes, cyber- we lost this battle; general audiences identify it as cyber) has surged past a trillion dollars. Yet some companies still claim that they don’t have the resources or money to do security beyond compliance.

    compliance is just compliance

    There is nothing wrong with compliance. People bash it for not being “security,” but it was never meant to be security. It’s a set of policies and criteria required to meet a minimum bar to do business with certain customers. Its function directly correlates with the business’s ability to make more money. I can fully understand why companies emphasize this work over other types of security work. However, it is not a risk-reduction function.

    Risk reduction is what you’re missing

    What most companies miss is evaluating their organization through the lens of risk reduction and taking appropriate action to avoid breach-related loss. Losses vary and should be identified and addressed based on your circumstances (e.g., most companies don’t necessarily need a Red Team). Compliance alone does not reduce risk because compliance requirements are not meant to serve this purpose. However, most organizations do the bare minimum because it ties directly to making or not making more money.

    “Always remember, compliance is important, but true security requires going beyond what’s mandated to effectively mitigate risk and stay ahead of the threat landscape – pg 10

    Ross Young (2025)

    Cybersecurity’s Dirty Secret: Why Most Budgets Go to Waste

    You’re choosing to be a sitting duck

    I want to be very clear: if you’re only doing compliance, you’re choosing to do almost nothing. This choice leaves you at the mercy of online threats, shows you don’t care about your customers’ data, and, to put it plainly, is negligent. This shift didn’t happen yesterday; in fact, choosing to do this for so long has led to a recent 2:1 advantage for adversaries yet again. Not to mention, many decades of breaches have shown us through experience that it’s a losing strategy. Your excuse about resources, money, or it being an *cough* APT is no longer acceptable. You’re responsible for making investments that reduce risk appropriately for your business…or you’re intentionally choosing to be a sitting duck 🦆.