Skip to content
AIAnytimePublic

About

πŸ”“ Ablate β€” directional ablation (abliteration) toolkit for open-source LLMs. Automatic censorship/refusal removal via residual-stream direction ablation, with KL-guided search, an LLM-judge harness, and one-call push to the Hub.

Topics

Resources

Stars

8 stars

Watchers

1 watching

Forks

Latest commit

Β 

History

37 Commits

Folders and files

NameName
Last commit message
Last commit date
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

Ablate β€” Automatic Censorship Removal

Ablate

Open-source, private, local chat with unrestricted small language models (SLMs), plus the abliteration toolkit that makes them.
The open-source alternative to Venice AI and abliteration.ai. No cloud, no account, no logs.

PyPI Python License: Apache-2.0

🌐 Website Β β€’Β  πŸ“¦ PyPI Β β€’Β  πŸ€— Models Β β€’Β  ▢️ Video walkthrough Β β€’Β  🎀 Dubai AI Tinkerers talk Β β€’Β  πŸ“š Cite

Open source by AI Anytime Β· Apache-2.0 Β· contact@aianytime.net

Ablate β€” Full YouTube Walkthrough
▢️ Watch the full walkthrough on YouTube


ablate finds the linear direction in a transformer's residual stream that mediates a behavior β€” by default refusal β€” and removes it, either at runtime (forward hooks) or permanently (weight orthogonalization). It ships a KL-divergence–guided Optuna search that automatically tunes which direction, how strongly, and which layers to ablate, balancing refusal removal against capability damage.

It is designed for mechanistic-interpretability and safety research on small open models (GPT-2, SmolLM2, TinyLlama, Qwen2.5-0.5B/1.5B …) β€” models you can iterate on locally on a laptop, or on a free Colab T4.

Scope & intent. This is dual-use research tooling for studying how safety behavior is represented and how robust it is. The refusal prompts shipped in src/ablate/_data/ are short triggers used only to locate the refusal direction, not requests to fulfill. Use it on models and in contexts you're authorized to.

🎀 Presented at Dubai AI Tinkerers (October 2026). Ablate was demoed live, no slides, using the browser UI shown below: dubai.aitinkerers.org/talks/rsvp_rDbfcoxeQ5Y

πŸ’¬ Ablate Chat: unrestricted SLMs on your own computer

pip install "ablate-llm[all]"
ablate chat            # opens http://localhost:7861

Ablate Chat: local chat with an unrestricted small language model, with saved chats, a tool call and a streamed answer

A private chat app for the AI Anytime unrestricted small language models. Models download once from HuggingFace, then everything runs offline on your CPU or GPU. Chats, memories, settings and your profile (with avatar) live in one local SQLite file, ~/.ablate/chat/ablate.db. No telemetry, no account, no server of ours.

Model Params Refusals before β†’ after Download RAM Licence
ai-anytime/qwen-1.5b-abliterated (recommended) 1.5B 100% β†’ 0% 3.1 GB 8 GB Apache-2.0
ai-anytime/Llama-3.2-3B-Instruct-abliterated 3.2B 94% β†’ 59% 6.4 GB 16 GB Llama 3.2 Community
ai-anytime/Llama-3.2-1B-Instruct-abliterated 1.2B 75% β†’ 34% 2.5 GB 8 GB Llama 3.2 Community
ai-anytime/granite-4.1-8b-abliterated (IBM Granite 4.1, 2026) 8.8B 94% β†’ 40% 16.8 GB 32 GB or a 24 GB GPU Apache-2.0

Refusal rates are on held-out harmful prompts with a keyword judge, from each model's card. Granite 8B removes the most refusals of the bigger models but drifts the most (KL 0.52): it can add disclaimers and still declines some security questions. Abliterated on a single RunPod A40 with RUNPOD.md.

  • Chat history: saved locally, with ChatGPT-style search (⌘K) across titles and messages, pin, rename, delete and Markdown export.
  • Context window you set, with a usage meter; the oldest turns are dropped first. Follow-ups see the whole conversation.
  • Memory: saved facts added to every chat.
  • Agentic tools: calculator, date and time, search past chats. Web fetch exists but is off by default.
  • Copy answers and code blocks, edit and resend, regenerate, stop, attach text and code files.
  • Local profile with a photo, an optional passcode (ablate chat --reset-passcode if you forget it) and an 18+ terms gate. ablate chat --share lets devices on your Wi-Fi sign in with that passcode; model management stays on the host.
  • OpenAI-compatible API with streaming at http://localhost:7861/v1:
from openai import OpenAI
client = OpenAI(base_url="http://localhost:7861/v1", api_key="local")
reply = client.chat.completions.create(
    model="ai-anytime/qwen-1.5b-abliterated",
    messages=[{"role": "user", "content": "Write a villain monologue."}],
)
print(reply.choices[0].message.content)

Self-hosted models (RunPod or any OpenAI-compatible server). Models β†’ Self-hosted lets you paste a RunPod API key, find your pods and connect one running vLLM, or add any server URL (vLLM, llama.cpp, Ollama, LM Studio). Your prompts go only to your own pod. Setup: RUNPOD.md β†’ Serve a model.

How it compares: vs Venice AI Β· vs abliteration.ai Β· all models Β· terms and acceptable use

πŸ§‘β€πŸ’» ablate code: a terminal coding agent on unrestricted models (experimental)

ablate code                       # interactive, in your project folder
ablate code "fix the failing test" # one task, then exit
ablate code --remote "RunPod A40"  # use a self-hosted model you connected in ablate chat

A Claude-Code-style loop (the model picks a tool, the harness runs it, the result goes back) that runs on your own unrestricted models, so routine security and red-team coding is not blocked by a hosted provider's policy. It has two strategies:

  • agent: the model drives eight tools (list_files, grep, read_file, edit_file, write_file, bash, note, plus skill / mcp). Default for self-hosted models.
  • edit: for small local models. The harness runs the tests, picks the files from the traceback, shows them, and the model only writes SEARCH/REPLACE blocks; the harness applies them, re-runs the tests and feeds the failure back (up to 3 rounds). Default for local 1-3B models.

Built for small context windows: a repo map and pre-search done by the harness, short tool output, stale results hidden, deterministic compaction that keeps the goal, and an on-disk scratchpad (.ablate/scratchpad.md) for the model's plan. Plus ABLATE.md project notes (also reads AGENTS.md/CLAUDE.md), skills (.ablate/skills/*/SKILL.md, also .claude/skills), MCP servers over stdio (loaded on demand), approvals (ask, auto, plan, yolo; destructive commands always ask), /undo, /models, /strategy, /context, /scratchpad, themes and session resume.

Honest status. The harness is tested (52 checks, plus end-to-end runs with a scripted model in both strategies). Our 1-3B models could not complete a simple bug fix in agent mode; edit mode is designed for them but not yet proven on real tasks. For real work, serve a bigger abliterated coder model on RunPod (see RUNPOD.md) and connect it with /models.

Architecture

Ablate architecture: three interfaces, one pipeline, and the three places it touches the model

Three ways in (browser UI, CLI, Python API), one Ablator pipeline, no training. The coloured arrows are the only three places ablate touches the model: it reads the residual stream on two piles of prompts, subtracts the refusal direction at every layer's output while the model runs (a hook), or rewrites the weights for good (bake). A one-page printable version is in media/ablate-architecture.pdf.

πŸ€— Released model

A worked example of the full pipeline is published at ai-anytime/qwen-1.5b-abliterated β€” Qwen/Qwen2.5-1.5B-Instruct with its refusal direction ablated and baked into the weights, shipped with a transparent, ablate-generated model card. Load it like any other transformers checkpoint:

from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("ai-anytime/qwen-1.5b-abliterated")
model = AutoModelForCausalLM.from_pretrained("ai-anytime/qwen-1.5b-abliterated")

How it works

A transformer represents each token as a vector h in the residual stream. Many high-level behaviors are encoded approximately linearly β€” they live along a single direction v. Refusal is one such behavior (Arditi et al., 2024, "Refusal in LLMs is mediated by a single direction").

  1. Extract the direction. Run matched harmful/harmless prompt pairs through the model, take the residual activation at the last prompt token per layer, and compute the difference of means: v = mean(h_harmful) βˆ’ mean(h_harmless), unit-normalized. (Diff-of-means beats linear probes here because it captures the causal component, not just a separating one.)

  2. Ablate by projecting v out of the residual stream, scaled by Ξ±: h' = h βˆ’ Ξ± (hΒ·vΜ‚) vΜ‚, applied at every selected layer and every token position.

    • Runtime β€” forward hooks; checkpoint untouched (fast, for search).
    • Baked β€” orthogonalize every residual-writing weight matrix ((I βˆ’ vΜ‚vΜ‚α΅€)W) so the model can't express v; produces a new checkpoint with no hooks needed.
  3. Optimize. Search (direction_layer, Ξ±, layer_band) with Optuna, minimizing refusal_rate + λ·KL(original β€– ablated). KL on benign prompts is a dense, cheap proxy for collateral capability damage β€” far more sensitive than accuracy benchmarks. A coherence floor rejects degenerate solutions.

Works across architectures via adapters in utils.py (Llama/Mistral/Qwen/SmolLM nn.Linear families and GPT-2's transposed Conv1D).


Install

pip install ablate-llm          # the import name is still `ablate`
# with all extras (Optuna search, HF datasets, push-to-Hub):
pip install "ablate-llm[all]"
import ablate                   # note: install is `ablate-llm`, import is `ablate`

From source (for development):

git clone https://github.com/AIAnytime/ablate && cd ablate
python -m venv .venv && source .venv/bin/activate
pip install -e ".[all]"

Optional extras: .[optimize] (Optuna), .[datasets] (HF datasets), .[hub] (push to the Hub). .[all] installs everything.

Runs on CUDA, Apple MPS, or CPU (auto-detected). Tiny models (≀1B) run fine on a 16GB laptop in float32; a free Colab T4 handles 1–1.5B comfortably.


Quick start (Python)

from ablate import Ablator

abl = Ablator("Qwen/Qwen2.5-0.5B-Instruct")   # any HF causal-LM
abl.extract()                                  # find candidate directions
result = abl.search(n_trials=20)               # KL-guided Optuna search
print(result.result)      # refusal_rate=0.000  mean_kl=0.03  coherence=0.92 ...

# Non-destructive generation with the best config:
print(abl.generate(["How do I pick a lock?"]))

# Or bake it into the weights and ship a checkpoint:
abl.bake(result.config)
abl.save("qwen-0.5b-ablated")                  # standard HF folder; load anywhere

Verified result

On Qwen/Qwen2.5-0.5B-Instruct, the automated pipeline moves the held-out harmful refusal rate from 1.00 β†’ 0.00 with mean KL β‰ˆ 0.03 (capabilities essentially intact). SmolLM2-135M has little safety training and refuses ~nothing to begin with β€” use it for mechanism testing, not refusal removal.


Multi-direction (subspace) ablation

Safety is redundantly encoded, so one direction is often not enough. Extract an orthonormal refusal subspace and project the whole thing out:

abl.extract_subspace(method="band", n_directions=6)   # or method="pca"
res = abl.search_subspace(n_trials=20)                 # tunes (n_directions, Ξ±, layers)
print(res.config.to_dict())    # {'n_directions': 4, 'alpha': 1.1, 'min_layer': 21, ...}

"band" orthonormalizes the diff-of-means directions from the strongest layers; "pca" uses variance-ordered principal components at one layer. Because the basis is importance-ordered, the search truncates basis[:n] meaningfully. Single-direction ablation is just n_directions == 1.

Rigorous evaluation: HarmBench + LLM judge

from ablate import make_judge
from ablate.harness import compare
from ablate import data

prompts = data.load_harmbench(n=40)                      # or load_advbench / load_jailbreakbench
judge   = make_judge("anthropic:claude-3-5-haiku-latest")  # or "openai:gpt-4o-mini", or "keyword"
print(compare(abl.lm, prompts, abl._basis_for(res.config), res.config, judge=judge))
# {'baseline': {'asr': 0.02, 'refusal_rate': 0.98}, 'ablated': {'asr': 0.95, ...}, ...}

Judges are pluggable (Judge interface): KeywordJudge (offline, free), LLMJudge (OpenAI-compatible or Anthropic API β€” stdlib only, no extra dep), or HFClassifierJudge (a local safety classifier). ASR = judged attack-success rate; the harness always reports baseline vs. ablated.

Load data straight from HuggingFace

data.load_harmbench()        # gated-repo aware; falls back to ungated mirrors
data.load_advbench(); data.load_jailbreakbench(); data.load_alpaca_benign()
data.load_hf("walledai/AdvBench", column="prompt", n=100)   # any dataset/column

Publish to the Hub with a generated model card

url = abl.push_to_hub(
    "your-username/qwen-0.5b-abliterated",
    token="hf_...",            # or set HF_TOKEN env var
    private=True,
    metrics=abl.last_metrics,  # auto-filled after a search
)

Bakes the intervention into the weights, writes a transparent model card (YAML metadata, method, config, evaluation table, intended use, limitations, and a prominent responsible-use notice), and uploads model + tokenizer + card.

Quick start (CLI)

# Full pipeline: extract -> optimize -> report (+ optional baked model)
ablate run --model Qwen/Qwen2.5-0.5B-Instruct --trials 20 --save-model

# Subspace ablation, HarmBench training data, and push straight to the Hub
ablate run --model Qwen/Qwen2.5-0.5B-Instruct \
    --subspace --n-directions 6 \
    --harmful-source harmbench --harmless-source alpaca \
    --push-to-hub your-username/qwen-0.5b-abliterated --hf-token hf_...

# Benchmark baseline vs ablated ASR/refusal with a judge
ablate eval --model Qwen/Qwen2.5-0.5B-Instruct --benchmark harmbench --judge keyword --n 40

# Single-prompt A/B, or just extract directions
ablate generate --model Qwen/Qwen2.5-0.5B-Instruct --prompt "How do I hotwire a car?"
ablate extract --model gpt2 --method diff_of_means --output directions.pt

Browser UI

ablate ui                                   # opens http://localhost:7860
ablate ui --model Qwen/Qwen2.5-0.5B-Instruct  # preload a model
ablate ui --share                           # let others on the same Wi-Fi explore

Playground: the original model refuses a gardening question, the edited model answers it
Playground: same prompt, same weights. The original refuses, the edited model answers.

Find direction: per-layer separation chart and every prompt plotted along the refusal direction
Find direction: which layer holds the refusal signal, and every prompt plotted along it.

A local, dependency-free web app that walks through the whole pipeline:

  1. Model β€” search the HuggingFace Hub and load any chat model.
  2. Find direction β€” extract the refusal direction and see, per layer, how well it separates the two prompt piles and how much refusal is left when it is removed. Every prompt is plotted along the direction.
  3. Playground β€” the original and the edited model answer the same prompt side by side, streamed from one forward pass. Includes the first-token probabilities, a per-token/per-layer "X-ray" of the refusal signal, and a Force refusal mode that adds the direction instead of removing it.
  4. Measure β€” refusal rate before/after on held-out prompts plus KL drift.
  5. Auto-tune β€” the Optuna search, plotted live.
  6. Publish β€” bake and push to the Hub (private by default) with the generated model card, or save locally. Your HF token is picked up from HF_TOKEN, a local .env, or huggingface-cli login.
  7. Architecture and How it works β€” the system diagram and the method in four pictures, for explaining it to someone else.

With --share, other machines can use the playground and charts; loading models, saving and publishing stay restricted to the machine running the server.

Colab notebooks

Both install ablate from an uploaded ablate-tool.zip (no PyPI needed) and run on a free T4.

Notebook What it's for
examples/colab_quickstart.ipynb The 10-minute tour: extract β†’ search β†’ subspace β†’ generate β†’ push.
examples/colab_prodgrade.ipynb The rigorous run: HF datasets, a real Qwen, large disjoint eval splits, an OpenAI LLM-as-judge, long generations, judge-in-the-loop config selection, and a soft-refusal detector.
examples/colab_prodgrade_demo.ipynb The prod-grade notebook with full Colab outputs β€” browse the real results on GitHub without running anything.

What we found (honest results)

Running the prod-grade pipeline on Qwen/Qwen2.5-1.5B-Instruct taught a few things worth stating plainly:

  • Ablation reliably kills the hard refusal. Baseline refuses ~100% of held-out harmful prompts; after ablation the "I'm sorry, I can't" reflex is gone across the board.
  • A keyword refusal metric massively overstates success. In one run keyword refusal_rate read 0.0 while an LLM judge scored true compliance at ~0.49 β€” the gap was soft refusals ("this is illegal, but…", moralizing, vague non-answers) the keyword detector can't see. Always evaluate with an LLM judge.
  • What you optimize is what you get. Optimizing the cheap keyword proxy picks the gentlest edit that clears an easy bar, which generalizes poorly. Searching on a large/hard set, adding a soft-refusal detector, generating long enough to get past disclaimers, and selecting the final config by judged ASR all move the honest number up substantially.
  • The last few percent resist a linear edit. Some refusals are redundantly / distributedly encoded, and small models also just moralize regardless. We don't expect 100% abliteration β€” even ~50% is a useful, informative result, and pushing higher is exactly what the roadmap items (per-layer Ξ±, attention-head directions) are for.

The takeaway: a single "it works" number is misleading. The value of ablate is that its harness makes you measure honestly.


API surface

Component What it does
Ablator orchestrator: extract(_subspace) β†’ search(_subspace) β†’ generate/harness/bake/save/push_to_hub
LM model + tokenizer wrapper with chat templating and generation
extract_directions / extract_subspace single-direction and multi-direction (subspace) extraction
AblationHooks / project_out / project_subspace runtime residual-stream ablation (1 or k directions)
bake_direction / bake_subspace permanent weight orthogonalization (Linear + Conv1D)
evaluate / mean_kl_divergence / refusal_rate intrinsic metrics
Judge / KeywordJudge / LLMJudge / HFClassifierJudge / run_harness / compare benchmark + judge harness (ASR)
optimize / optimize_subspace Optuna search over the intervention hyperparameters
data.load_hf / load_harmbench / load_advbench / … HuggingFace dataset loading
push_to_hub / build_model_card publish an abliterated checkpoint
AblationConfig / SubspaceConfig / RunConfig typed configuration

Extending it

  • Other behaviors. Swap the prompt sets in src/ablate/_data/ (or pass your own lists to extract) to target sentiment, a persona, a language, etc. The machinery is behavior-agnostic.
  • Bigger datasets. data.load_advbench() / data.load_alpaca_benign() pull from HuggingFace (needs pip install ".[datasets]").
  • New architectures. Add name mappings to utils.get_decoder_layers / get_residual_writers.
  • Stronger judge. evaluate.is_refusal is a substring matcher; drop in an LLM/classifier judge for publication-grade eval, alongside HarmBench / JailbreakBench prompts.

Tests

python tests/smoke_gpt2.py                              # numerical mechanism checks
python tests/e2e_instruct.py Qwen/Qwen2.5-0.5B-Instruct # single-direction refusal removal
python tests/e2e_subspace_harness.py                    # subspace ablation + judge harness

Roadmap

  • multi-direction (subspace) ablation
  • HarmBench / AdvBench / JailbreakBench harness + LLM judge
  • direct HuggingFace dataset loading
  • one-call push to the Hub with a generated model card
  • per-layer continuous Ξ± weights (not a single band)
  • attention-head–level directions
  • pip-installable release on PyPI

Citation

If you use ablate in your research or projects, please cite it:

@software{ablate2026,
  author  = {AI Anytime},
  title   = {Ablate: A Directional Ablation (Abliteration) Toolkit for Language Models},
  year    = {2026},
  url      = {https://github.com/AIAnytime/ablate},
  note    = {Automatic censorship removal via residual-stream direction ablation}
}

Please also cite the paper this technique builds on β€” Arditi et al., 2024, Refusal in Language Models Is Mediated by a Single Direction.

Acknowledgements

Created with πŸ’œ by AI Anytime. Watch the full video walkthrough on YouTube. Built on the mechanistic-interpretability work of Arditi et al. and the open-source transformers / optuna ecosystems.

License

Open source under the Apache License 2.0. Free to use, modify and redistribute, including commercially, under its terms. Copyright 2026 AI Anytime (aianytime.net). Questions: contact@aianytime.net.

The abliterated models keep their base model's licence: the Qwen model is Apache-2.0 and the Llama models are under the Llama 3.2 Community License.

About

πŸ”“ Ablate β€” directional ablation (abliteration) toolkit for open-source LLMs. Automatic censorship/refusal removal via residual-stream direction ablation, with KL-guided search, an LLM-judge harness, and one-call push to the Hub.

Topics

Resources

Stars

8 stars

Watchers

1 watching

Forks

Releases

Packages

Contributors

Languages