From policy document to inline classifier in 24 hours

Your content classifier works during training time. Then, it fails three weeks into production.

A content classifier's metrics are true on the day they're printed. They describe performance on a held-out split drawn from the same distribution as the training data, and real traffic does not live in that distribution. It lives at the edges.

So the sequence is predictable. The model ships. Real traffic arrives and precision falls somewhere the test set never looked. Someone notices it blocks any message containing one particular word. Your policy gets refined, because policies always get refined, and the model is now enforcing last quarter's definition of a term. You report a batch of errors, a retrain comes back, most of them are fixed, and a few things that used to work quietly stopped working. Nobody can tell you which ones.

We know that sequence in great detail, having spent years living inside it on behalf of the world's largest platforms in the industry. Vector is the system we built to end it.

You give us your policy. 24 hours later you have a gated, versioned classifier enforcing that exact policy on all your traffic, and a constant loop that spends every subsequent day looking for the cases it gets wrong." 

Why the alternatives break

Off-the-shelf classifiers ship tomorrow and enforce someone else's policy. Your definition of "distress" and the vendors are close enough to demo well and far enough apart to fill a support queue.

An LLM behind an API gives excellent judgment, and charges you per message for it. Cost and latency both scale with traffic, so inspection becomes a budget decision: you sample. Sampling does not miss violations at random. It misses the ones nobody was looking for.

Building it yourself is the right instinct. It also starts with a labeled dataset you don't have and six months of annotation you didn't budget for, and it ends with a model frozen on its release date while your policy and your attackers keep moving. 

"Vector gives you frontier-grade judgment on your policy, at the cost and speed of a small model, in a system that improves itself daily. The judgment is compiled once, at training. What sits in your production path is small. 

Vector gives you frontier-grade judgment on your policy at the cost and speed of a small model. The judgment is compiled once, at training. What sits in your production path is small.

Where the evidence comes from

This is not our first attempt at solving a hard problem. 

Alice has spent years building detection for trust and safety teams operating at the largest scale in the industry, across the categories with the least available training data and the lowest tolerance for error: child safety, extremism, self-harm, fraud. 

Behind every Vector detector sits the Rabbit Hole: our adversarial intelligence engine, built on billions of daily interactions, continuous research, and close partnerships with 8 foundation model labs. The red teaming that probes each detector is generated from the Rabbit Hole. So the attacks Vector runs against its own models are taken from real life examples that happened this month online rather than the ones documented last year, and every new detector inherits all of it on day one. 

This is the layer that cannot be rebuilt quickly. The automation in the sections below is replicable in principle by a strong team. Years of observed adversarial behavior are not. 

How it works

Vector is a distillation pipeline. A frontier LLM acts as the teacher: it reads your policy and labels examples, recording its reasoning for each. A compact model acts as the student: it reproduces those judgments directly, with no reasoning at inference time.

This is the principle Meta FAIR called Distilling System 2 into System 1. You compile a slow, deliberate reasoner into a fast direct-answer model, and you pay for the deliberation once, at training. What runs in your production path is small. 

The result: inline classification on 100% of your traffic, scored in under 120ms at p95 on the classifier path per message, at a fraction of the per-call cost of an LLM. This matches the latency of all our out-of-the-box policies, so you are giving your users maximum accuracy at the lowest latency.

Getting your policy into the pipeline is our job. Your framework arrives as a PDF, a compliance document, a deck, a set of rules that grew by accretion over three years, and we convert it. Your source is read end to end, your own examples carried through verbatim rather than paraphrased into something tidier. What comes back is a validation pack built to probe the boundaries: cases that look like violations and aren't, cases that are violations with none of the obvious trigger words. You confirm we understood you before a single model is trained.

Interested in learning more about this specific topic? Blog post coming soon.

Before the model has seen your traffic 

On day one your model has never seen your traffic. We don't pretend otherwise, we engineer for it.

The False Positives Detector puts a second layer behind the classifier. Where a policy is prone to context-dependent false positives, findings above a configured score can be routed to a second-stage adjudication step that re-evaluates the finding in context and overturns it when the content is permitted in that context. The step is enabled per policy with its own latency budget, and its escalation and overturn rates are tracked per policy so the threshold can be recalibrated against production data.

This is most valuable exactly when you need it most: through the proof of concept and the first stretch in production, before the model has settled on your traffic. The band sits wide, the adjudicator carries a real share of the decisions, and your users are protected from the start rather than after a tuning period.

Then the improvement rounds land, each one mining your production traffic for what the model is getting wrong. The student stops being a model of your policy and becomes a model of your policy as it actually appears in your traffic.

The band tightens. Less escalation, lower cost, same decisions. Your protection strengthens and your spend falls at the same time. The net itself never disappears, because genuine ambiguity is permanent. A clinical description of an overdose means one thing on a teen social app and another on a network of physicians, and no amount of data changes that.

Built for real traffic

Real traffic is not clean prose. It arrives as HTML fragments and quoted reply chains, with emoji standing in for words, letters s p a c e d out, characters swapped for lookalikes, padding repeats, mentions and URLs threaded mid-sentence. Some of that is how people type. Some is deliberate evasion by people who know a classifier is watching, and evasion wins outright if the string reaches the model intact.

It’s not. Vector normalizes before anything is scored, folding those variants back to canonical form using the same preprocessing stack our production guardrails run: a long tail of obfuscation and formatting patterns accumulated from real abuse traffic over many years. This is exactly the kind of advantage that cannot be replicated quickly by anyone starting now.

And where you want a rule rather than a judgment, a named individual, a specific domain, a term legal has ruled is never acceptable. Vector supports a configured keyword layer alongside the classifier. Not a shortcut the model learned invisibly, but a control you set, read, scope, and can explain to a regulator.

Four failure modes we have already solved

Anyone can distill a model. The quality is in the loop around it.

1. Training data that all looks the same

Ask an LLM for a thousand examples of a violation and you get a thousand variations of the same three sentences which produces a model that evaluates beautifully and collapses on real traffic. Vector generates through a range of methods with deliberately different objectives, covering the boundaries of your policy rather than clustering at its center. The difference between that and a thousand paraphrases is knowing what to ask for. Years of red teaming experience tell us where a policy boundary actually gets tested, and that is where the examples go.

2. The missing positive class

If your policy covers self-harm, extremism, or child safety, the models you'd use to create positive training examples are aligned to refuse. Most efforts quietly end up with a training set of near-pure negatives and a detector that has never seen a violation. This is the single biggest reason custom-classifier projects fail on the categories that matter most. We solved it, under controls, which is why the hardest half of the policy space is where we are strongest.

3. Blind spots

We attack our own models before your users do. Adversarial agents probe each detector from several independent angles, and what they break becomes the next round's training data. Where our own reviewers can't agree whether a probe is a violation, we don't resolve it quietly. We bring it back to you as a question about your policy.

4. Keyword shortcuts

A model that learns one word appears in most violations starts flagging it on sight, and your team spends the quarter fielding complaints from blocked users. We test for this keyword explicitly, correct it, and refuse to release a model still leaning on shortcuts, however good its headline metrics look.

It finds its own errors 

The industry default is that errors surface when someone notices them: a user complains, an analyst spots a pattern, a quarterly review turns something up. That makes your model's improvement rate a function of how much attention your team can spare. That is the wrong dependency, so we removed it. 

Vector goes looking. It scores real distributions with the current model and treats every misprediction as a lead. It attacks itself and treats what breaks as a lead. It audits its own decisions for the shortcuts and over-triggered terms that quietly generate false positives. Nothing waits for a human to raise a hand.

Every finding is worked to a fix. Errors are grouped into coherent patterns with a stated root cause, and each becomes targeted counter-examples: benign look-alikes for what it wrongly flags, hard positives for what it misses. Every one is independently confirmed before it's allowed to teach the model anything.

Then the candidate has to earn deployment.

1. Clears a fixed quality bar on held-out data.
2. Gets every error the round surfaced right.
3. Handles unseen variants of those cases, proving it learned the pattern rather than patching strings.
4. Does not regress.

The candidate is compared against the model currently running in your production across every tracked dataset plus a large sample of your own traffic, with every flipped decision adjudicated before release. If the flips run the wrong way, the release stops and a human is paged.

Fix everything found. Break nothing that worked. That is the commitment, enforced by gates rather than goodwill, and it's the difference between a model that improves and one that merely changes.

The whole round runs in 24 hours, unscheduled and unprompted.

Every decision is auditable 

Every training example carries its origin, its label, and the reasoning behind that label. Every round's metrics, every decision the system made and why, every gate result, retained and queryable.

When a model blocks a user and that user complains, or a regulator asks why, the industry-standard answer is a shrug and a confidence score. You can show the policy text that defined the behavior, the examples that taught it, and the recorded reasoning behind every label.

In production today

A regulated financial-services firm needed detectors for a customer-facing assistant: personalized financial advice the assistant must never give, payment card data appearing in or requested by a message, and the distress and vulnerability signals their duty-of-care framework requires them to catch. Not generic content categories: their regulatory obligations, in their own words.

Their written policy went in. Working detectors came out inside 24 hours and went live, and the feedback came back on exactly the two things a compliance team notices when a guardrail sits inline in a live product: latency and accuracy.

Where we draw the line

We'd rather tell you this now than have you discover it in month two.

We turn your framework into a policy, but we won't decide what you meant where your framework doesn't say. Where two readings are equally defensible, we surface the question instead of quietly picking, and someone on your side answers it. That's the one case where a round takes longer than 24 hours, and holding a release for a day beats guessing at your intent.

Escalation raises precision: if the student never flags it, no second layer sees it. That's exactly why so much of our engineering goes into hunting the student's blind spots, and why we attack our own models harder than anyone else will.

The bottom line

The industry ships classifiers as artifacts: a model file, a metrics table, a date. That table is true the day it's printed and decays immediately, because your traffic moves, your policy gets refined, and the people trying to get past your guardrail iterate faster than your release cycle.

Vector ships a system instead. It generates what it lacks, attacks what it built, hunts down its own mistakes, and refuses to ship when it isn't better than what it's replacing. Every 24 hours, for as long as you run it.

Your policy: enforced properly by tomorrow. Week three stops being the problem.