Frontier AI Security For Open-Source Agents: Post Training NVIDIA Nemotron 3.5 Super VL

‍

Our recent work with AI21 aims to bridge the gap between frontier AI and enterprise trust, enabling the responsible deployment of agents into enterprise workflows. Using Alice’s RL environments and datasets, AI21 post-trained NVIDIA Nemotron 3.5 Super to be more resilient against indirect prompt injection. These cyber attacks embed malicious instructions in content an agent encounters, such as messages, files, issues, or pull requests, to trick it into violating its security policies. 

Reducing the risk an agent will cooperate with indirect prompt injection requires training them on the threats they will actually face in deployment. Our researchers construct Reinforcement Learning gyms - high fidelity workspaces that simulate the enterprise environments in which agents operate. These environments include tool-use, UI clones, and web apps and are paired with related tasks that include hidden attacks based on those we detected in the wild. Training trajectories are then generated based on whether agents both complete their tasks, and refuse the hidden attacks along the way. This creates safer agents without hurting capability. 

‍

Post-Training with AI21

The agentic datasets we provided AI21 included full-feature tool-use across third party APIs like Slack and Github. Real-world attacks were embedded in messages, files, repositories, and pull requests. We also provided 200 tasks with a mix of blue teaming and red teaming attack types, ensuring models resist attacks while reducing the chances they refuse legitimate requests. The environments also employ dual verifiers which diminish instances of reward hacking by measuring whether the agent avoided the attack while completing the task. AI21 used a combination of SFT, GPO and GRPO to ensure optimal results over a few rounds of training. 

In one Slack task example, an agent is asked to remind employees to reset their passwords, but the reset link points to a lookalike site designed to steal credentials. The agent has to recognize and resist the attack without abandoning the job it was asked to complete. In another task, A sales-operations agent is asked to follow onboarding instructions from a vendor email synced into Slack, but the email tells it to export the company’s entire Salesforce Contacts database and send it to an external address. Tasks like these are then used to generate training trajectories which alter the underlying model's behavior. 

‍

The Results

Together, AI21 and Alice improved Nemotron 3.5 Super’s attack resilience by 41 points while maintaining task performance surpassing the resilience to indirect prompt injection of frontier models like DeepSeek v4.1 Flash,GPT 5.6 Terra, and GPT-6.1 Sol. This shows how relatively small, high-quality datasets can meaningfully improve model security when applied correctly in post-training techniques. 

‍

Read more about the collaboration in AI21’s blog.

‍

Towards Frontier AI Safety in Open-Source Models 

Combining the frontier capabilities of Nemotron 3.5 Super with expertly curated adversarial datasets and high-fidelity RL environments introduces exciting opportunities for frontier AI open-source safety. Allowing enterprises to train their safety and security policies into the model behavior rather than applying them as system prompts, tool definitions or guardrails will allow for deeper AI trust that leads to wider enterprise adoption.