LLM safety

LLM safety probes that read the model, not just its output

LLM probes, also called activation probes, read harmful-intent signals from a model's hidden states during generation, with no separate guard model to host.

83–99% F1
WildJailbreak 99.1%, AEGIS 2.0 83.5%, BeaverTails 82.7% (Llama 3.1 8B)
~550× smaller
A 12.6M-parameter probe vs. 7B guard models
<1ms
Probe runtime vs. 50–500ms generation time

No separate guard model

The signal comes from activations the model already computes

Can stop mid-generation

An inline probe can end a request before the next token is produced

Competitive with guard models

Ahead of WildGuard and LlamaGuard3 on AEGIS 2.0; behind BeaverDam, MD-Judge, GPT-4 and WildGuard on BeaverTails

For models you host

Probes need access to activations, so they won't work on a closed API

Harmful-intent detection

Trained on prompt and response benchmarks that cover intent and response harmfulness, from the hidden states of the model being served.

Start from the probe library

Our unsafe-content probe is published with its benchmarks and known limits.

Or train on your policy

We train custom probes on your model and your own risk categories, and report how they perform.

Run it with Undercurrent

Our open-source runtime attaches probes to LLMs on Hugging Face Transformers or vLLM (experimental).

Benchmark results above are on Llama 3.1 8B, using final-layer activations and binary labels. The WildJailbreak figure uses its vanilla subset, which tests intent rather than adversarial jailbreak techniques. Robustness to an attacker who knows about the probe is untested, and results on other models are measured when probes are trained for them.

See it on your own model

Tell us which LLM you host and what you need to detect.

Talk to us