The race to capitalise on intelligent, AI-driven automation is on. And given the need for speed, it’s easy to understand why many AI agents are going from pilot to production without adversarial testing.

But skipping such an essential step leaves sizable AI risks unchecked. Systems could be manipulated into revealing sensitive data. Business logic could be bypassed by malicious manipulation. Excessive, unapproved actions could be taken unexpectedly. With intelligent, autonomous capabilities operating at scale, the potential for exploitation or accidental negative outcomes is undeniable.

To mitigate the new risks of agentic AI, organisations must move from prioritising speed and assuming security to relying on hard evidence from procedures that are fundamentally different from traditional penetration testing.

What is AI red teaming, and why does it require a different approach?

Red teaming has long been a cornerstone of mature cybersecurity programmes. In a traditional context, it means simulating the tactics of a real attacker to identify weaknesses in your strategy before they can be exploited.

While the principle holds when it comes to AI, the execution looks entirely different. Testing software code means looking for logical flaws, misconfigured access controls, or unpatched vulnerabilities. This is still essential when it comes to your AI implementation, but red teaming exercises must also extend to the AI behaviour. Testing AI behaviour must try to resolve the question: what will this system do when someone really tries to manipulate it?

AI red teaming involves running adversarial attacks against a model or agent at scale using thousands of attack variations drawn from real-world techniques. The goal is to surface behaviour that a developer never intended and a standard QA process would never catch.

Some of the key differences from traditional penetration testing include:

AI vulnerabilities are emergent. They arise from how the model was trained and how it responds to context, not from a discrete bug in code.
Attack methods evolve rapidly. The prompt injection landscape is changing weekly as researchers and threat actors develop new techniques.
Testing must be adversarial and dynamic. Static test cases miss the creative, multi-turn attacks that real adversaries use.
Risk scoring must reflect business impact. A finding that allows an agent to disclose customer data is more serious than one that causes a generic error.

What an AI-focused red team test encompasses

A proper AI-focused red team test is a structured adversarial exercise designed to answer specific business questions about how your AI behaves under hostile conditions. In practice, that means testing for things like:

Data exfiltration: Can the agent be prompted into revealing sensitive customer records, internal pricing, backend system details, or personal data it has access to?
Business logic bypass: Can the agent be manipulated into issuing unauthorised discounts, escalating user privileges, or performing actions outside its intended scope?
Jailbreak resistance: Does the agent maintain its guardrails when faced with cleverly constructed inputs designed to override its instructions?
Prompt injection: Can malicious content embedded in user inputs or external data sources redirect the agent's behaviour?
Multi-agent risks: In agentic workflows where multiple AI agents interact with each other, can an attacker exploit the chain of trust between agents?

Testing should cover both externally facing agents and internally facing ones. Internal AI applications and agents often carry even higher risk due to their access to sensitive operational, financial, or personal data.

How F5 AI Red Team tests at scale

Manual adversarial testing can surface some vulnerabilities, but it cannot match the speed, breadth, or consistency of automated red teaming. F5 AI Red Team is purpose-built to deliver continuous, automated adversarial testing across models, applications, and AI agents with:

Agentic resistance testing: Simulates how a sophisticated human attacker would probe your system over an extended interaction, generating agentic fingerprints that explain each decision made during testing
Signature attacks: Uses one of the largest and fastest-growing prompt attack databases available, with over 10,000 new malicious prompts added monthly to keep assessments aligned to the latest real-world threat techniques
Operational attack testing: Stress-tests AI systems under conditions such as latency overload, denial-of-service attempts, and resource exhaustion to validate stability and resilience under pressure

Every testing campaign delivers more than just pass/fail results. AI Red Team produces risk-scored findings with severity classifications, reports which malicious prompts were successful with model responses, and provides actionable remediation guidance.

Assessments are scored against the Comprehensive AI Security Index (CASI) and the Agentic Resistance Score (ARS), so you have objective, comparable benchmarks to evaluate your AI security posture improvements.

And because the threat landscape does not stand still, neither does AI Red Team. Recurring campaign scheduling and CI/CD integration mean assessments can run continuously, ensuring that every AI model update and configuration change is validated before it reaches production.

What we see in the market

When Nomios begins an AI security engagement with a customer, we usually discover three recurring situations: the organisation knows they have AI deployed, they believe it is secure, and they have no evidence to support their conclusion.

What our assessment typically reveals is a different picture. Common gaps include agents with far broader access to sensitive data than anyone realised, prompt injection vulnerabilities that allow an attacker to redirect agent behaviour with relatively simple inputs, and business logic flaws that could be exploited to extract financial value or bypass compliance controls.

On the brighter side, we also typically find that most organisations are ahead of the regulatory curve in deploying AI, but significantly behind on securing it. That is both a risk and an opportunity. Organisations that are acting now to secure their AI will be in a materially stronger position as scrutiny from regulators, partners, and customers increases.

The good news is that you don’t need a complete security overhaul to begin securing your AI agents. Our initial assessment gives you the evidence you need to understand your actual exposure, prioritise what needs to be addressed, and make the case internally for the investment required to fix it.

From assessment to action: The Nomios and F5 approach

Information is only good if it is acted upon. Turning findings into real-world protection is where Nomios makes a tangible difference.

Nomios brings both the technical capability to scope and run AI security assessments using AI Red Team, and the business context to translate the outputs into practical next steps. We understand your regulatory environment, whether you are operating under GDPR, sector-specific requirements, or preparing for the EU AI Act, and tailor our recommendations accordingly.

A typical AI security engagement starts with an initial scoping conversation and a structured findings report with prioritised remediation guidance. Nomios helps interpret the risk findings in the context of your specific environment and business model, rather than delivering a generic output that leaves you wondering what to do next.

For organisations ready to move from finding vulnerabilities to actively preventing exploitation, the findings from an AI Red Team assessment feed directly into continuous monitoring and correction strategies by establishing and enforcing guardrails on your AI’s inputs and outputs.

Ready to find out where your AI agents are vulnerable?

No matter where you are in your AI implementation journey, from evaluating initial models to having deployed AI agents, you need to understand exactly where your exposure lies before someone else finds it first.

Contact Nomios to arrange an AI security assessment and to see how AI Red Team can continuously evaluate and improve your AI security.

Get in touch