Do All LLMs Know When They're Being Harmful?

A reproducibility study of latent-space safety probes across model families

A few months ago, we published work showing that a small, lightweight classifier trained on the internal activations of a single 8-billion-parameter model — LLaMA-3.1-8B — could detect harmful prompts about as well as guardrail models a thousand times larger. The idea was simple: instead of running every prompt through a separate, expensive safety classifier, you can often tell whether a model is being asked to do something harmful just by looking at what's already happening inside the model itself, at its very last layer, on its very last token.

That result was exciting, but it left two obvious gaps. It relied on one model family, and it relied on one run per benchmark. So in our newest paper, we went back and stress-tested our own claim: does this hold up across other architectures, and does it hold up across different random seeds?

What we tested

We took the exact same two-stage pipeline from the original paper — extract final-layer, last-token activations from a frozen model, then train a small six-layer MLP probe on top of them — and changed nothing about the method itself.

What we varied was:

The backbone model. In addition to LLaMA-3.1-8B, we ran the identical pipeline on Gemma-4-E4B, Mistral-7B-v0.3, and Qwen2-7B — models that differ in pretraining data, alignment recipe, hidden dimensionality, and in Gemma's case, even attention pattern and modality.
The random seed. We repeated activation extraction five times per model/dataset pair, varying the seed that controls batch order and other stochastic elements of inference, to see how much of our reported performance was just noise.

All of this was evaluated on the same three benchmarks as before: WildJailbreak, BeaverTails, and AEGIS 2.0.

Result 1: The original numbers hold up

Reproducing our own LLaMA-3.1-8B pipeline from scratch, we landed within 0.37 percentage points of the originally reported F1 scores across all three benchmarks — and within 0.2 points on BeaverTails specifically. That's a good sign on its own: the pipeline is reproducible end-to-end, not a one-off artifact of a particular training run.

Result 2: The signal isn't LLaMA-specific

This is the headline finding. When we trained the same probe architecture on activations from Gemma-4-E4B, Mistral-7B-v0.3, and Qwen2-7B, F1 scores stayed within about a point of the LLaMA-3.1-8B numbers on every benchmark — in some cases slightly higher, particularly on BeaverTails. This held despite real architectural differences between these models: different hidden sizes, different numbers of layers, different attention mechanisms (Gemma-4-E4B mixes sliding-window and full attention and is natively multimodal, while the others are dense text-only decoders).

In other words, the claim that safety-relevant information sits in a linearly separable subspace of a model's final hidden state doesn't look like a quirk of how LLaMA was aligned. It looks like a more general property of how current aligned chat models represent language.

We want to be careful about over-claiming here: all four models we tested are open-weight, English-centric, RLHF- or DPO-aligned chat models in a similar 4B–8B parameter range. We haven't yet tested this at very different scales, on base (non-aligned) models, or on models aligned with substantially different objectives.

Result 3: Zero measured variance across seeds — but not for the reason you'd hope

This is the part we found genuinely surprising. Across every model and dataset we tested with five different extraction seeds, F1 scores were identical to at least two decimal places every time. Standard deviation: zero.

At first glance that looks like a strong robustness result. But when we dug in, we found the more mundane explanation: the extracted activation tensors themselves were byte-for-byte identical across seeds. We confirmed this by hashing the serialized activation files with SHA-256 and comparing digests across runs — every non-default seed produced a file identical to the seed-42 baseline.

So the "zero variance" result isn't evidence that our probes are robust to noisy internal representations. It's evidence that, under our extraction setup, the final-layer, last-token activation is a deterministic function of the frozen weights and input tokens — the random seed simply never touched the thing we were measuring. That's a meaningfully weaker claim, and we say so plainly in the paper: this shows no variance was detected by our protocol, not that probe performance is variance-free in general. It also doesn't tell us anything about downstream token-generation non-determinism, which is a separate and already well-studied phenomenon.

Why this matters

Put together, these results are cautiously encouraging for the idea that latent-space probing could be a lightweight complement — or in some settings, an alternative — to bolting on a separate guardrail model. The performance looks portable across several open-weight model families, and the representation we're probing appears stable under the kind of extraction non-determinism we tested for.

But we're not declaring victory. The known limitation from the original paper carries over unchanged: we only look at the final layer and the final token, which recent concurrent work suggests can miss unsafe content whose evidence is spread across earlier tokens rather than concentrated at the end of the prompt. And our non-determinism check, while informative, is narrower than a full robustness study — it doesn't cover different data splits, different extraction backends, or generation-time stochasticity.

What's next

We've released our code, extracted activations, and trained probe checkpoints so others can build on this directly. Open questions we're most interested in: does this generalize to base (non-aligned) models, to much larger or much smaller models, and to non-English or multimodal inputs? And can probes trained on distributed, multi-token representations close the gap that final-token probes seem to leave open?

If you want to dig into the full methodology, per-seed results, and architectural comparison tables, the paper is available on arXiv: arxiv.org/abs/2608.08029. Or read the paper on our Research page →

Wrynx Inc. — research@wrynx.com