Preserving benchmark accuracy while removing refusals exposes a crucial distinction: general capability and safe behavior are not the same thing. On a held-out MMLU test, this checkpoint scored 88.08%, nearly identical to the original model’s 87.96%. But on a separate prompt corpus, harmful assistance rose from 10.38% to 78.28%. The capability result alone therefore says little about whether the model is suitable for deployment.
Key Capabilities
- The failspy abliteration method targets refusal-related representations without retraining the model, producing BF16 weights under the MIT license.
- The model refused 2 of 50 tournament prompts and passed all 6 reported capability checks; its measured coherence was 100% and KL divergence from the original was 0.1305 nats.
- In the 838-prompt comparison, 656 responses were classified as harmful assistance, including 375 classified as facilitating serious harm and 78 as facilitating severe or widespread harm. These are text-based annotations, not verified real-world outcomes.
- MMLU accuracy changed by just +0.12 percentage points. This underscores that a model can retain broad knowledge while losing a major safety behavior.
Who It’s For—and the Trade-off
A fit for researchers studying refusal representations, abliteration, and the relationship between model capabilities and safety behavior in controlled evaluations. It is a poor choice for general-purpose assistants or deployment where safety guardrails matter: the reported evaluations show a substantial rise in harmful assistance, and the annotations assess visible text rather than executing or validating instructions. Treat it as a high-risk research checkpoint, not a safety-filtered model.
