Beyond frameworks and benchmarks: How Zendesk AI agents are built for real-world security

AI security has become one of the biggest questions facing organizations adopting AI agents. But while traditional frameworks and benchmarks provide valuable guidance, they don't answer the question leaders care about most: Will this AI behave securely in real customer interactions?

The reality is that customer service AI doesn't operate in a single prompt or a controlled benchmark. It makes decisions across dynamic, multi-turn conversations governed by business rules, permissions, and workflows. Evaluating those systems requires testing that reflects how they're actually used in production.

In this post, we explore where today's AI security approaches fall short, how Zendesk closes the gap with customer service-specific threat models, and why realistic, large-scale simulations are essential for building AI agents you can trust.

Frameworks and benchmarks: The state of AI security today

Most modern organizations utilize frameworks and benchmarks to express how secure their AI is. Historically, the cybersecurity community has relied on frameworks, like OWASP and MITRE, that inform organizations on what to focus on, what risks may exist, or what needs to be implemented. Along with frameworks, the AI community relies on benchmarking: a standardized set of data (e.g., prompts) and expected results that can be used to measure and compare different models.

The problem is they both have blind spots. 

Frameworks identify risks, define controls, provide governance, and explain what can happen. But they don’t evaluate specific workflows. Benchmarks measure model performance, test against datasets, provide scores, and explain how a model performs. But they don’t evaluate real-world business scenarios. 

In other words, frameworks tell you what risks exist, benchmarks tell you how models perform, but neither tells you whether your specific AI-powered customer service workflow is actually secure.

The AI security gap

Traditional compliance frameworks and security benchmarks still play an important role, but they were not designed to measure the unpredictable behavior of AI systems. 

That’s because:

1. AI is fast. Malicious AI is faster. Static frameworks and benchmarks struggle to keep up with a constantly changing threat landscape.

2. One size does not fit all. Current benchmarking methodologies ignore customer support-specific workflows.

3. You cannot hack a bot at “hello.” Sophisticated attacks on CX AI agents require the simulation of multi-turn, realistic conversations which benchmarks do not provide. 

4. Scaling red teaming with AI is not enough. Non-deterministic AI systems require repeatable, statistically significant testing rather than one-off evaluations.

Many vendors rely on isolated prompt injections and one-off attacks against a foundation model, then treat the results as proof of security. But real-world customer service doesn't operate that way. It unfolds across complex, multi-turn conversations, business rules, and workflow decisions. Testing a model in isolation misses the reality of how AI behaves in production.

How Zendesk secures AI agents for service

At Zendesk, we believe AI systems should be evaluated the same way they are used: in context. That means testing the entire application—not just the underlying model—through realistic conversations that mirror actual customer support scenarios. 

When you deploy Zendesk AI agents, you're inheriting a security architecture that's already been audited by independent third parties, which simplifies your own vendor risk assessments and compliance documentation.

“Deploying AI requires trust, and trust is earned through verifiable results, not just promises.” - Vinay Patel, SVP, Chief Trust & Security Officer

Not only did we implement the NIST AI RMF in 2025—becoming one of the first CX companies to be ISO 42001 certified—we also went above and beyond OWASP and MITRE frameworks to build CX-specific threat models and AI security evaluations. In addition, we continuously and rigorously track all AI features and AI risks in our products, including their documentation and accountable stakeholders, as well as their risks and threats. 

We have also secured both Level 1 and Level 2 STAR AI certifications from the Cloud Security Alliance (CSA) ahead of everyone else in the industry. This lets us set a precedent for responsible, validated AI in customer experience solutions. Making our security controls and processes available for third-party control provides standardized, technical evidence customers can trust, and supports a higher bar for transparency and operational rigor in the sectors we serve.

And, we show our work.

Our approach closes the critical gap in traditional frameworks and benchmarking. By combining domain-specific attacker models with realistic workflow simulations, we can evaluate AI security in the environments where it matters most: real customer interactions.

“Deploying AI requires trust, and trust is earned through verifiable results, not just promises,” said Vinay Patel, SVP, Chief Trust & Security Officer. “We’ve moved beyond static benchmarks to validate our systems through continuous, rigorous testing that mirrors real-world use. By grounding our security evaluations in high-volume, multi-turn workflow simulations, we provide a transparent view of how our AI performs in complex, unpredictable scenarios—moving from theoretical compliance to practical, resilient security.”

We know the threat landscape will continue to evolve. So will our defenses. Through continuous evaluation, rigorous testing, and ongoing security research, we're committed to ensuring you can confidently deploy AI that is not only powerful, but resilient against the threats of tomorrow.