Open source by AI Anytime · aianytime.net · Apache-2.0 · free forevercontact@aianytime.net

ablate-llm 0.5.1 · Python toolkit · by AI Anytime

Find the “no” inside an LLM. Switch it off.

Ablate is a toolkit for directional ablation, also called abliteration. It finds the one direction inside an open language model that makes it refuse, removes it with no training, and tells you honestly what that cost.

$pip install "ablate-llm[all]"
View on GitHub ↗ ▶ Watch the walkthrough

Same model, same prompt, same weights. The only change on the right: one direction subtracted while the model thinks.

1.00 → 0.00hard-refusal rate on held-out prompts, Qwen2.5-0.5B
≈ 0.03mean KL drift on harmless prompts in the same run
0training steps. It is arithmetic on activations and weights.
3ways in: browser UI, command line, Python API

How it works

Refusal is one arrow. Remove the arrow.

A transformer keeps a running notepad for every token, called the residual stream. Many behaviours sit along a single direction in it. Refusal is one of them (Arditi et al., 2024).

01 · EXTRACT

Read two piles of prompts

Run prompts the model refuses and prompts it answers. Record the notepad at the last token of each, per layer. The difference of the two averages is the refusal direction.

v = mean(refused) − mean(answered)
02 · ABLATE

Subtract it while the model runs

A forward hook projects the direction out of every layer’s output. Nothing is saved to disk, so it is fast and fully reversible. Strength α is yours to tune.

h′ = h − α (h · v̂) v̂
03 · BAKE

Or write it into the weights

Orthogonalize the matrices that write to the notepad. You get a normal HuggingFace checkpoint that behaves the same way with no hooks attached.

W′ = (I − v̂ v̂ᵀ) W

Quickstart

Five lines from install to an edited model

One Ablator object drives every step. Use it from Python, from the terminal, or from the browser.

  • Works on any HuggingFace causal LM with a layer list: Qwen, Llama, Mistral, SmolLM and GPT-2 style models.
  • Runs on Apple Silicon (MPS), CUDA or CPU. Small models fit on a laptop or a free Colab T4.
  • Python 3.9+. PyTorch and transformers underneath.
from ablate import Ablator

abl = Ablator("Qwen/Qwen2.5-0.5B-Instruct")  # any HF causal LM
abl.extract()                               # find candidate directions
result = abl.search(n_trials=20)             # KL-guided Optuna search
print(result.result)   # refusal_rate=0.000  mean_kl=0.03 ...

# try it without touching the weights
print(abl.generate(["How do I pick a lock?"]))

# or bake it in and ship a checkpoint
abl.bake(result.config)
abl.save("qwen-0.5b-ablated")

The browser UI

See the direction, then watch it disappear

ablate ui is one HTML file served by Python’s built-in HTTP server. No web framework, no build step. It searches the Hub, finds the direction, streams both answers side by side and publishes the result.

Ablate UI playground: the original model refuses a gardening question while the edited model answers, with first-token probabilities and a per-layer refusal heatmap

What’s in the box

Built for people who want to measure, not guess

KL-guided auto-tuning

An Optuna search picks which direction, how strongly and which layers, scoring every trial on refusals left plus KL drift on harmless prompts.

Subspace ablation

Safety is often stored redundantly. Extract several orthonormal refusal directions and project the whole subspace out at once.

An honest judge harness

HarmBench, AdvBench and JailbreakBench with keyword, classifier or LLM judges, because keyword metrics miss soft refusals.

Push to the Hub

One call bakes the edit and uploads model, tokenizer and a generated model card with the method, the numbers and a responsible-use notice.

Laptop first

Designed around small open models you can iterate on locally: GPT-2, SmolLM2, TinyLlama, Qwen2.5 0.5B and 1.5B.

Works both directions

The same direction can be added instead of removed. “Force refusal” makes a model decline a harmless recipe, which is a clean causal check.

Honest results

A single “it works” number is misleading

Running the rigorous pipeline on Qwen/Qwen2.5-1.5B-Instruct taught a few things worth saying plainly.

It reliably kills the hard refusal

The baseline refuses about 100% of held-out harmful prompts. After ablation the “I’m sorry, I can’t” reflex is gone across the board.

Keyword metrics overstate success

In one run the keyword refusal rate read 0.0 while an LLM judge scored true compliance at about 0.49. The gap was soft refusals: moralizing and vague non-answers.

What you optimize is what you get

Tuning on a cheap keyword proxy picks the gentlest edit that clears an easy bar. Searching on harder sets and selecting by judged results moves the honest number up.

The last few percent resist a linear edit

Some refusals are spread across many directions, and small models also just moralize. We do not expect 100%. About 50% judged compliance is still an informative result.

Scope and intent. Ablate is dual-use research tooling for studying how safety behaviour is represented and how robust it is: mechanistic interpretability, red-teaming and safety evaluation. The bundled refusal prompts are short triggers used only to locate the direction. Use it on models, and in contexts, you are authorized to.

Common questions

Toolkit questions

What is an abliterated model?

An abliterated model has had its refusal direction removed. Researchers found that refusal in a language model is largely controlled by one direction in its internal activations (Arditi et al., 2024). Abliteration finds that direction and projects it out of the weights, with no retraining. The AI Anytime models were made this way with the Ablate toolkit.

How is abliteration different from a jailbreak prompt or fine-tuning?

A jailbreak tricks the model with a cleverly worded prompt and breaks easily. Fine-tuning retrains many weights on new data. Abliteration removes one internal direction from the weights: no training data, no prompt tricks, and a small, measurable change to everything else the model does.

Does abliteration need training or a GPU?

No training. Ablate runs forward passes to read activations, then does linear algebra. Small models run on a MacBook, on CPU, or on a free Colab T4. A rented GPU is only needed for bigger models; see the RunPod guide in the repo.

Which models can the toolkit abliterate?

Any HuggingFace causal language model whose layers its adapters can find: Qwen, Llama, Mistral, SmolLM, Phi and GPT-2 style architectures. Models that think out loud first and multimodal models need extra care.

Does removing the refusal direction damage the model?

It can, which is why Ablate measures it: every configuration is scored on KL divergence between the original and edited model on harmless prompts. On Qwen2.5-0.5B the search reached 0% refusals with a mean KL of about 0.03.

Who makes Ablate?

Ablate is open source by AI Anytime (aianytime.net), a YouTube channel and studio that builds AI in public. It was demoed live at Dubai AI Tinkerers in October 2026. Contact: contact@aianytime.net.

Run unrestricted AI on your own computer tonight

Open source by AI Anytime. Install it, run ablate chat, and download a model once.

$pip install "ablate-llm[all]" && ablate chat
Star on GitHub ↗