GenAI: The new attack vector for trust & safety
Malicious actors are rapidly moving from testing GenAI to deploying it for large-scale abuse across digital platforms. This report uncovers how these groups manipulate AI models to bypass traditional safeguards and automate harmful activities.
- Understand the shift from manual to AI-accelerated threats in fraud and disinformation.
- Identify the specific techniques predators and extremists use to subvert safety filters.
- Learn proactive strategies to fortify your moderation workflows against synthetic harms.
‍
Overview
The democratization of Generative AI has provided bad actors with a powerful new toolkit to amplify their operations at an unprecedented scale. While much of the industry focuses on internal model safety, our research focuses on the "wild"—the hidden communities where threat actors share tutorials on jailbreaking models and generating prohibited content. From creating hyper-realistic synthetic media for disinformation to automating the grooming of minors, the nature of online harm is undergoing a fundamental shift.
In the report, "Generative AI: The New Attack Vector for Trust and Safety," Alice draws on exclusive threat intelligence to show how these groups are bypassing existing safety guardrails. We examine real-world case studies, including a 172% increase in AI-generated harmful imagery and the rise of deepfake audio used for political instability. By understanding these adversary TTPs (Tactics, Techniques, and Procedures), Trust and Safety teams can move from a reactive posture to a proactive defense. This research provides the context necessary to anticipate how GenAI will be used as a weapon, helping you build more resilient systems that protect your users and your brand’s integrity.
‍
Download the Full Report
What’s New from Alice
Curiouser Soundbites: The AI Risk Debt Your Enterprise Is Already Carrying
Chances are your enterprise AI is moving a lot faster than your visibility into it and Alison Cossette has a lot to say about that. She joined Mo on Curiouser & Curiouser to get into the risk debt that's quietly building inside agentic systems, why observability and traceability aren't optional anymore, and what leaders actually need to do about it.
The Problem With AI Observability Nobody Wants To Admit
Most enterprises have guardrails. Far fewer have visibility into what their AI is actually doing. Alison Cossette, Founder and CEO of ClariTrace, joins Mo to talk about the risk debt quietly building inside agentic systems, why observability and traceability aren't optional anymore, and what leaders need to put in place before something forces their hand.
Distilling LLMs into Efficient Transformers for Real-World AI
This technical webinar explores how we distilled the world knowledge of a large language model into a compact, high-performing transformer—balancing safety, latency, and scale. Learn how we combine LLM-based annotations and weight distillation to power real-world AI safety.
Beneath the Surface: The Growing Ecosystem of AI Nudification
Alice analyzed 100 AI nudification websites to uncover how synthetic NCII ecosystems scale through frictionless onboarding, affiliate monetization, and cross-platform distribution.
