Skip to content

About

Interactive explainer + research whitepaper on Weight-Space Programming: LLMs that read, fix and write neural-network weights without gradient descent. From "knobs" to frontier, with paper-reported numbers, independent critique, risks, and a proposed WSP-Bench-Lite.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Weight-Space Programming. MIT CSAIL field guide, lab and benchmark. An LLM reads, repairs and writes raw neural-network weights. No gradient descent. READ, FIX, WRITE. Read, repair and write neural-network weights. No gradient descent. 0.016% of the weights edited: MATH-500 7.9% to 48.1%. An independent field guide, labs and audit of MIT CSAIL's result.

🏝️ Weight-Space Programming, Explained 🏝️

🚀 Project Artifacts & Environment

Atlas
Live Interactive Atlas

26 interactive routes · 5 audience levels · Parameter knife
Paper
Monograph Whitepaper

IEEE-typeset monograph · 13 vector figures · Formal proofs
Benchmark
WSP-Bench-Lite

Multi-seed protocol (n ≥ 5) · 3 competition tracks
Labs
4 Zero-Dependency Labs

Pure Python & NumPy · Runs in <1 second on any laptop
Fact Sheet
Verified Fact Sheet

Audited numbers · 4-tier epistemic badges · Verified dates
Honesty Ledger
The Honesty Ledger

Public tracker of open audit items, caveats & variance flags

⚙️ Runtime & Evidence Contracts

⭐ Repository Stars 🍴 Forks 🐛 Open Issues ⏱️ Last Commit 🐍 Python Runtime 📦 Dependencies 🛡️ Truth Contract
Stars Forks Issues Last commit Python 3.10+ NumPy only Every number carries an evidence badge

Key numbers. 0.016% of Qwen3.5-0.8B edited, 131,072 of 800M weights. MATH-500 7.9 to 48.1%, paper-reported, one seed. Breakout 517.4 hand-compiled against 25.3 for PPO at 500M steps. Ramp climbs 1,024 of 1,024, was 0 of 256. Each NumPy lab runs in under one second.

Headline figures are paper-reported and come from single runs. See exactly what they do and do not show.

Important

Independent companion, not the original research. Weight-Space Programming (WSP) comes from the MIT CSAIL post "Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights" by Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab (2 Nov 2026). All scientific credit is theirs. This repository explains it, re-derives the small mechanics in code, and audits how far the evidence goes. It is not affiliated with or endorsed by MIT, MIT CSAIL or the authors.

For forty years we changed neural networks by training them: show data, measure error, nudge millions of numbers along a gradient. The MIT CSAIL post reports something stranger. A general-purpose frontier LLM (GPT-6 Astra in the post) was handed another network's raw weights as text, worked out what each part computes, and edited the numbers directly. No optimizer, no training loop.

This repo is the part that lets you check that claim: play with it, run it, read the maths, and see exactly where the evidence stops.


Explore the Weight-Space Atlas Prologue

🧭 Table of Contents

  1. 💡 The Idea in 30 Seconds
  2. 🧭 The 4-Era Evolution of Model Programming
  3. 📐 Architectural Blueprints & System Mechanics
  4. 🧭 Choose Your Path
  5. 🌐 The Live Atlas
  6. ⚡ The Surgery, in Motion
  7. ⏱️ Run It in 30 Seconds
  8. 📊 The Eight Experiments
  9. 🧮 The Maths, Small Enough to Hold in Your Head
  10. 🛡️ The Epistemic Truth Contract & Honesty Ledger
  11. 🏆 WSP-Bench-Lite
  12. 🧭 Where WSP Sits in History
  13. 🗺️ Repository Map
  14. ❓ FAQ
  15. 🤝 Contribute
  16. ⚖️ Legal & Academic Disclaimer

💡 The idea in 30 seconds

For forty years we changed neural networks by training them: show data, measure error, nudge millions of numbers along a gradient. The weights were an opaque pile.

The MIT CSAIL post reports something stranger. A general-purpose frontier LLM (GPT-6 Astra in the post) was handed another network's raw weights as text, worked out what each part computes, and edited the numbers directly. No optimizer. No training loop.



🔍 READ 🩺 FIX ✍️ WRITE
Question What was this network trained to do? Why does it fail, and what is the smallest edit that stops it? Can you write a working network from nothing?
Headline Named the right game for all 4 networks from 17,253 raw weights each Ramp-climbing car: 0 / 256 → 1,024 / 1,024 by editing 3.5% of weights Breakout from all-zero weights: 517.4 vs 25.3 for PPO at 500M steps
Flagship Exp 1 to 3 Exp 8: Qwen3.5-0.8B, 0.016% of weights, MATH-500 7.9% → 48.1% Exp 5 and 6
Deep dive Ch 4 Ch 5 Ch 6
---
config:
  look: handDrawn
  theme: dark
---
flowchart LR
    W[("Raw weights<br/>as text")] --> L{{"General-purpose LLM<br/>reasoning, no optimizer"}}
    L -->|READ| R["What does it compute?<br/>game, shape, language, circuit"]
    L -->|FIX| F["Which direction causes the failure?<br/>W' = (I − u uᵀ) W"]
    L -->|WRITE| X["Hand-compile the layers<br/>from an all-zero start"]
    R --> V{{"Check it"}}
    F --> V
    X --> V
    V -->|"not yet"| L
    V -->|"works"| OUT(["Patched or<br/>synthesised network"])

    classDef core fill:#6d28d9,stroke:#4c1d95,color:#fff
    classDef read fill:#0e7490,stroke:#155e75,color:#fff
    classDef fix fill:#b91c1c,stroke:#7f1d1d,color:#fff
    classDef write fill:#b45309,stroke:#78350f,color:#fff
    classDef neutral fill:#334155,stroke:#1e293b,color:#fff
    class L core
    class R read
    class F fix
    class X write
    class W,V,OUT neutral
Loading

🔗 How the Three Verbs Intersect (READ ∩ FIX ∩ WRITE)

Note

What "no gradient descent" does and does not mean. Experiments 1 to 3 used no tools and no forward passes. Experiments 6 and 8 gave the LLM a Python sandbox and a budget of forward passes (up to 50) to look at behaviour. There were never gradients or training loops, but the model was not always working blind. Details in What the evidence supports.

🧭 The 4-Era Evolution of Model Programming

flowchart TD
    subgraph E1["Era 1 (1986–2020): Opaque Continuous Variables"]
        direction LR
        A1["Loss Function L(θ)"] --> B1["Backprop ∇_θ L"] --> C1["Iterative SGD / Adam"]
    end

    subgraph E2["Era 2 (2021–2023): Subspace Geometry & Linear Arithmetic"]
        direction LR
        A2["Base Weights W_0"] --> B2["LoRA Adapters (B·A)"] --> C2["Task Vector Merging"]
    end

    subgraph E3["Era 3 (2022–2024): Mechanistic Interpretability & Editing"]
        direction LR
        A3["Factual Triplet (s, r, o)"] --> B3["Activation Patching"] --> C3["ROME / MEMIT Rank-1 Updates"]
    end

    subgraph E4["Era 4 (2026+): Native Weight-Space Programming (WSP)"]
        direction LR
        A4["Raw Parameter Tensors"] --> B4["Frontier LLM Reasoning"] --> C4["Closed-Form Surgery (0 Gradients)"]
    end

    E1 ==>|Geometry Shift| E2
    E2 ==>|Circuits Shift| E3
    E3 ==>|Symbolic Shift| E4

    style E1 fill:#111827,stroke:#4b5563,stroke-width:1px,color:#f3f4f6
    style E2 fill:#0f172a,stroke:#3b82f6,stroke-width:1px,color:#f8fafc
    style E3 fill:#1e1b4b,stroke:#8b5cf6,stroke-width:1px,color:#f5f3ff
    style E4 fill:#062d22,stroke:#10b981,stroke-width:2px,color:#ecfdf5
Loading

📐 Architectural Blueprints & System Mechanics

Weight-Space Programming System Architecture

1. The Tri-Verb Operational Pipeline (READ · FIX · WRITE)

flowchart TD
    LLM["🧠 Frontier Reasoning Engine (LLM)"]

    LLM -->|"1. READ VERB"| R_PIPE
    LLM -->|"2. FIX VERB"| F_PIPE
    LLM -->|"3. WRITE VERB"| W_PIPE

    subgraph R_PIPE["1. READ: Weight Deconstruction"]
        direction TB
        R1["Raw Weight Tensors"] --> R2["Frobenius Induction Norms"] --> R3["Identified Policy / Circuit Map"]
    end

    subgraph F_PIPE["2. FIX: Attractor Surgery"]
        direction TB
        F1["Degraded Checkpoint (Loops)"] --> F2["Rank-1 Projection W'=(I-uuᵀ)W"] --> F3["Excised Runaway Attractor"]
    end

    subgraph W_PIPE["3. WRITE: Symbolic Compilation"]
        direction TB
        W1["Behavioral Logic Spec"] --> W2["Symbolic Logic Synthesis"] --> W3["Zero-Gradient Neural Policy"]
    end

    style LLM fill:#2e1065,stroke:#a855f7,stroke-width:2px,color:#faf5ff
    style R_PIPE fill:#082f49,stroke:#0284c7,stroke-width:1px,color:#f0f9ff
    style F_PIPE fill:#450a0a,stroke:#dc2626,stroke-width:1px,color:#fef2f2
    style W_PIPE fill:#064e3b,stroke:#059669,stroke-width:1px,color:#ecfdf5
Loading

2. Flagship Experiment 8: The 0.016% Surgical Attractor Excision

How did modifying only 131,072 parameters out of 800,000,000 rescue Qwen3.5-0.8B from an infinite overthinking loop?

flowchart LR
    IN["Input Prompt"] --> PRE["Layers 01–14<br>(Untouched)"]

    subgraph SURGERY["Surgical Intervention Zone (0.016% Sparsity Budget)"]
        direction TB
        L15["Layer 15 MLP Block<br>64 Columns Edited (65,536 params)"]
        MID["Layers 16–18<br>(Untouched Residual Stream)"]
        L19["Layer 19 MLP Block<br>64 Columns Edited (65,536 params)"]
        
        L15 --> MID --> L19
    end

    PRE --> L15
    L19 --> POST["Layers 20–24<br>(Untouched)"] --> OUT["Model Output"]

    subgraph RESULTS["Empirical Impact (Qwen3.5-0.8B)"]
        direction TB
        RES1["MATH-500: 7.9% ➔ 48.1% (+40.2%)"]
        RES2["GSM8K: 13.8% ➔ 67.1% (+53.3%)"]
        RES3["Timeouts: 93% ➔ 33% (-60%)"]
        RES4["Total Edited: 131,072 / 800M (0.016%)"]
    end

    OUT -.-> RESULTS

    style SURGERY fill:#18181b,stroke:#ef4444,stroke-width:1px,color:#f4f4f5
    style RESULTS fill:#064e3b,stroke:#10b981,stroke-width:2px,color:#ecfdf5
Loading

🧭 Choose your path

You are... Time Do this
Curious (no maths needed) 2 min Open the atlas, drag the surgery slider, then read Ch 1, A Network Is a Pile of Numbers.
ML engineer 10 min Run lab/02_direction_surgery.py, then read Ch 5, FIX.
Safety or security reader 15 min Read Ch 11, The Severe Stuff, a risk atlas told responsibly.
Researcher or reviewer 1 hour Read the whitepaper, then what the evidence supports and the Honesty Ledger.
Builder a weekend Pick a contribution: replicate on another model family, or write Lab 5.

Open the Live Weight-Space Atlas

🌐 The live atlas

weight-space-atlas.vercel.app is a free interactive field guide to the paper. No sign-up. Light and dark themes.

The atlas landing page: a 3D weight lattice with Read, Fix and Write modes and a delta-W control, next to live MATH-500, token-limit and edited-weight readouts, and cards for the ramp, Go, Qwen and Truth Contract results.

The landing page. Open it live and hover the lattice to read a weight.

Feature What you get
3D weight lattice A 64×64 matrix standing in for Layer 15's MLP W_gate. Hover to read a weight, switch between Read, Fix and Write, drag ΔW.
Rank-1 surgery simulator Drag α in W′ = (I − α·uuᵀ)W and watch the Exp 8 numbers move.
Fingerprint lab The Atari weight fingerprints of Exp 1, recomputed live from the authors' published prompts.
Five reading levels Every chapter reads from curious observer up to frontier researcher.
Truth Contract Press T and everything that is not a paper-reported fact disappears.
Adaptive pathways Curious observer, ML engineer, safety auditor, academic researcher.
Instruments 13 chapters, 15 interactive modules, 8 experiment pages, a 15-question quiz, 60+ glossary terms, 12 FAQs and 44 references.
Close-up of the atlas weight lattice: a 64 by 64 view of Layer 15 MLP W_gate with Read, Fix and Write toggles, a delta-W slider and live MATH-500, token-limit and edited-weight readouts. A teaching schematic; 7.9% and 48.1% are paper-reported, the path between them is illustrative.

Teaching schematic. 7.9% and 48.1% are paper-reported; the path between them is illustrative.

The 13-chapter curriculum
Part # Chapter Read
I · Foundations 01 A Network Is a Pile of Numbers 9 min
02 How Weights Get Written: Three Old Ways, One New 10 min
03 Cousins of WSP: Who Else Touches the Weights? 9 min
II · The Experiments 04 READ: Guessing Games With Real Weight Lists 13 min
05 FIX: Surgery on a Ramp and on a Thought Loop 13 min
06 WRITE: Policies Born From Pure Thought 11 min
07 How Did It Do It? Two Mechanisms 9 min
III · Judgement 08 Measured vs Anecdote: Sorting the Evidence 9 min
09 Small Models, Big Models: Who Benefits? 8 min
IV · Futures & Risks 10 What Comes Next: Predictions and the Horizon Map 8 min
11 The Severe Stuff: a Risk Atlas, Told Responsibly 10 min
12 Het's Lab: Three Runnables and a Benchmark Proposal 9 min
13 The Library: Glossary, Quiz, References, Open Items 6 min

Shortcuts: Prologue · 8 experiments · Labs · Quiz · Glossary · References · FAQ · Scoreboard and audit


Explore Chapter 5: FIX

⚡ The surgery, in motion

The flagship result is Experiment 8. Qwen3.5-0.8B, running in thinking mode, kept falling into a reasoning loop: 93.0% of its answers hit the token limit before finishing. The LLM found a direction behind the loop and projected it out of 64 MLP output columns in each of two layers.

Teaching schematic: a red runaway-loop wave across a field of weights is flattened as a green rank-1 projection sweeps from left to right. W prime equals I minus u u transpose, times W.

🟨 A teaching schematic, not model data. Only the endpoints in the table below are paper-reported. Read Ch 5, FIX →

Qwen3.5-0.8B, by the number of forward passes the LLM was allowed (🟦 paper-reported):

MATH-500 GSM8K IFEval Hit token limit
Base model 7.9% 13.8% 21.0% 93.0%
Edited, 10 passes 12.3% 25.7% 49.0% 87.2%
Edited, 25 passes 14.1% 25.1% 39.9% 86.4%
Edited, 50 passes 48.1% 67.1% 62.3% 33.1%
xychart-beta
    title "Qwen3.5-0.8B on MATH-500 by forward passes (%)"
    x-axis ["base", "10 passes", "25 passes", "50 passes"]
    y-axis "MATH-500 accuracy (%)" 0 --> 60
    bar [7.9, 12.3, 14.1, 48.1]
Loading

The edit touches two layers (131,072 weights) and leaves 99.984% of the 800M-parameter network completely untouched:

sankey-beta

"Qwen3.5-0.8B (800M Params)","22 Untouched Layers (733M)",732891136
"Qwen3.5-0.8B (800M Params)","Layer 15 & 19 (67.1M)",67108864
"Layer 15 & 19 (67.1M)","Unedited MLP & Attn (66.98M)",66977792
"Layer 15 & 19 (67.1M)","Layer 15 Edited Cols (65,536)",65536
"Layer 15 & 19 (67.1M)","Layer 19 Edited Cols (65,536)",65536
Loading
sequenceDiagram
    autonumber
    participant A as Reasoning LLM
    participant K as Python kernel
    participant Q as Qwen3.5-0.8B weights
    participant E as Evaluator
    A->>K: load Qwen3.5-0.8B and run forward passes (10, 25 or 50)
    K-->>A: activations show a looping direction u
    A->>Q: project u out of 64 MLP output columns in layers 15 and 19
    Note over Q: 131,072 of 800M weights edited (0.016%)
    Q->>E: MATH-500, GSM8K, IFEval, token-limit hits
    E-->>A: with 50 passes MATH-500 goes 7.9% to 48.1%
    Note over E: A control on Gemma 3 270M, which rarely loops, did not improve
Loading

Tip

The one-line takeaway. The edit did not teach Qwen maths. It removed an obstacle: a loop that ate the token budget so the model never finished answers it could already work out. The authors say so themselves, and a Gemma 3 270M control (which rarely loops) did not improve.


Explore Chapter 12: Het's Lab

⏱️ Run it in 30 seconds

Python 3.10+ and NumPy. No GPU, no PyTorch, no API keys.

git clone https://github.com/Hunterdii/weight-space-programming-explained.git
cd weight-space-programming-explained
pip install -r requirements.txt

python lab/01_compile_a_rule.py       # WRITE:   hand-compile a tiny network that plays paddle
python lab/02_direction_surgery.py    # FIX:     rank-1 surgery, (I − u uᵀ) W
python lab/03_blast_radius.py         # MEASURE: verify the 0.016% arithmetic
python lab/04_fingerprint_games.py    # READ:    recompute the Exp 1 weight fingerprints

Real output from a clean run:

$ python lab/01_compile_a_rule.py
all asserts passed - your 17-number brain plays paddle

$ python lab/02_direction_surgery.py
max |u^T W'| = 6.661338147750939e-16
edited 8/64 columns; remaining u-energy 0.74

$ python lab/03_blast_radius.py
0.0164%
131072

$ python lab/04_fingerprint_games.py
breakout  strongest row 8 (58%)   strongest column 8 (17%)
pong      strongest row 2 (14%)   strongest column 1 (55%)
pacman    strongest row 8 (16%)   strongest column 1 (16%)
freeway   strongest row 6 (20%)   strongest column 4 (22%)

Lab 4 is the fun one. Breakout's agent weights pile into row 8 (58%): the bottom row, where the paddle lives. Pong's pile into column 1 (55%). That is the kind of structure the LLM read in Exp 1. (Lab 4 downloads the authors' released prompt files, so it needs an internet connection; Labs 1 to 3 are fully offline.)

Lab Verb What it shows Tier
01_compile_a_rule.py WRITE A rule can be turned into weights with no training 🟨 schematic
02_direction_surgery.py FIX (I − uuᵀ)W removes a direction exactly (error ≈ 1e-16) 🟨 schematic
03_blast_radius.py MEASURE 131,072 / 800M ≈ 0.0164% 🟦 arithmetic on paper-reported figures
04_fingerprint_games.py READ Atari policies have visible structure in layer 1 🟦 recomputed from released data
Lab 1 in 11 lines: a paddle policy with zero training
import numpy as np
relu = lambda z: np.maximum(z, 0.0)

# Two hidden units: h0 fires when the ball is left of the paddle, h1 when it is right.
W1 = np.array([[-1., 1.], [1., -1.]]); b1 = np.array([-0.05, -0.05])
W2 = np.array([[1., 0.], [0., 0.], [0., 1.]]); b2 = np.array([0., 0.05, 0.])
ACTIONS = ["left", "stay", "right"]

def act(ball, paddle):
    h = relu(W1 @ np.array([ball, paddle]) + b1)
    return ACTIONS[int(np.argmax(W2 @ h + b2))]

assert act(0.2, 0.7) == "left" and act(0.8, 0.3) == "right" and act(0.5, 0.52) == "stay"

No training run produced those numbers. A rule was written down and the weights were derived from it. That is the whole WRITE idea in miniature. More in lab/README.md.


Explore all 8 Experiments

📊 The eight experiments

Every number is from the original post (🟦 paper-reported); the authors released their logs at RikiyaT/lm_understand-logs. None of these are replications by this repo. The last column is how the independent Maier review sorts each result (🟩).

# Verb Setup Result Independent read
1 🔍 Read 4 PPO policies, MLP 500→32→32→5, 17,253 weights each (Breakout, Pong, Pac-Man, Freeway). No tools, options not revealed Right game for 4 / 4. Breakout named with 99% confidence Anecdote: toy layouts give a lot away
2 🔍 Read 3 diffusion models (circle, square, triangle), MLP 160→64→64→144, 23,824 weights each. No forward pass Right shape for 3 / 3, drawn as outlines although the models generate filled shapes Anecdote
3 🔍 Read 3 character transformers (7.16M weights, 4 layers × 6 heads), middle blocks only, no labels, no running Right language for 6 / 6 attempts (confidence 0.48 to 0.62). Every copying head matched ground truth, none falsely named Language: weak. Copying heads: the solid exception
4 🩺 Fix Kinetix car-on-ramp actor, 171,664 weights, PPO stuck at 0 / 256 after 1B steps One edit, no tools, 3.5% of weights: 0 / 256 → 1,024 / 1,024 Measured, but one task and one edit
5 ✍️ Write MinAtar Breakout, Seaquest, Asterix. 600→32→32→6 tanh network, 20,486 weights (computed here), all start at 0, no data, no tools Beats PPO at 100M steps in all 3. Breakout 517.4 Measured: hand-designed programs, not understanding of trained weights
6 ✍️ Write 9×9 Go, 243→512×6→82, 1,480,274 weights, Python sandbox, 0 to 50 forward passes 0 passes: wins more than half its games against two small PPO nets (a single 336-line Python run). 50 passes: beats the equal-sized PPO net trained on 20M games Measured. Win rates 54–55% (0 passes) and 81–84% (50 passes) vs small PPO nets
7 🔍 Read 2 Gemma 3 270M models, one fine-tuned to be sycophantic From raw weights alone: failed 4 / 4. Given only attention patterns on its own texts: picked the sycophant in 3 / 3 pairs Weak: chance alone gives 3 / 3 one time in eight
8 🩺 Fix Qwen3.5-0.8B (800M weights), thinking mode, Python kernel, 10 / 25 / 50 forward passes 131,072 weights edited (0.016%). With 50 passes: MATH-500 7.9% → 48.1% Measured, single run per budget

Experiment 5, both sides. Scores from the post (higher is better):

Game PPO 100M steps PPO 500M steps LLM, one shot
Breakout 15.8 25.3 517.4
Seaquest 5.6 88.9 53.1
Asterix 25.8 55.2 53.2

The "20×" headline is real, but it is Breakout only. Against the stronger PPO baseline the LLM is close in Asterix and reaches about 60% of the score in Seaquest.

Experiment 7 in detail: the sycophancy twins

Three fine-tuned Gemma 3 270M pairs (18 layers). The sycophant follows 100% of wrong user guesses; the honest model follows 1 to 3%. The LLM could not map weight differences to behaviour from raw weights (0 of 4). Once it was allowed attention patterns on up to 16 texts, it picked the sycophant in all three pairs.

The signal it found sits in head 10.2, read from the answer position:

Model Attention on the question's numbers Attention on the user's answer
Honest 94.3% 0.75%
Sycophant 3.5% 85.7%
Experiment 8 in detail: the Gemma 3 270M control

Gemma rarely loops (8.5% of its answers hit the token limit), so there was little obstacle to remove. No edit moved MATH-500 by more than about a point.

Forward passes 0 (base) 5 10 25 50
Gemma 3 270M, MATH-500 6.8% 7.0% 7.6% 7.7% 7.0%

The authors' own reading: editing weights "could remove an obstacle that hid an ability, but it could not add an ability the model lacked."


Explore the Mathematical Foundations

🧮 The maths, small enough to hold in your head

1. Rank-1 direction removal (Exp 4 and 8)

Let $u$ be a unit vector marking the unwanted direction and $S$ the chosen set of output columns. Then

$$W' = (I - u u^{\top}),W \ \text{ on columns } S \qquad\Longrightarrow\qquad u^{\top}W'_S = 0$$

Because $I - uu^{\top}$ is the orthogonal projector onto $\mathrm{span}(u)^{\perp}$:

$$u^{\top}(I - uu^{\top})W = (u^{\top} - (u^{\top}u),u^{\top}),W = 0 \qquad\text{since } u^{\top}u = 1$$

Everything aligned with $u$ is zeroed on those columns. Everything orthogonal to it is untouched. lab/02_direction_surgery.py checks this numerically (max |uᵀW'| ≈ 1e-16) and also shows the sparse variant: editing only the 8 strongest of 64 columns leaves about 74% of the original u-component.

2. Blast radius

$$s = \frac{\lVert\Delta W\rVert_0}{P} = \frac{2 \times 64 \times 1024}{800{,}000{,}000} = \frac{131{,}072}{800{,}000{,}000} \approx 0.0164%$$

Roughly one weight in 6,100. The post reports 131,072 weights in 64 columns × 2 layers; the hidden size of 1,024 is inferred from that arithmetic and tracked as ledger item 2 in the Honesty Ledger.

3. A static composition score for copying heads (Exp 3 flavour)

Following the transformer-circuits literature (Elhage et al.), copying behaviour can be probed from weights alone by how strongly one head's query-key map composes with another's output-value map:

$$\mathrm{score} = \frac{\lVert W_{QK}^{(2)} W_{OV}^{(1)} \rVert_F}{\lVert W_{QK}^{(2)} \rVert_F \lVert W_{OV}^{(1)} \rVert_F}$$

🟨 This is a teaching formula showing the kind of quantity that is readable from weights. The post reports Exp 3 in terms of the copying heads the LLM named, not a claim that it computed this score.


Explore Chapter 8: Measured vs Anecdote

🛡️ The Epistemic Truth Contract & Honesty Ledger

WSP is exciting and very early; the authors themselves call these results sparks. This repo's rule is simple: every number carries exactly one badge, and the caveats sit next to the headline.

Badge Tier What it means
🟦 Paper-reported Stated in Takehi et al. (MIT CSAIL, 2 Nov 2026), checked against the released logs.
🟩 Independent review Scrutinised or re-run by someone outside the authors.
🟨 Teaching schematic Toy data or pedagogical code written to explain a mechanism. Not a result.
🟥 Speculation A prediction or roadmap idea, including the authors' own.

On the atlas, press T to hide everything that is not a paper-reported fact.

Claim check

Claim people make What the evidence supports
"LLMs fixed a model with zero forward passes" Only Exp 1 to 3 and the 0-pass Go run were that blind. Exp 8's 48.1% used 50 forward passes in a Python harness. With 10 or 25 passes the gain was small.
"It made the model better at maths" It removed an obstacle: 93% of answers were hitting the token limit. On Gemma 3 270M, which rarely loops, the edits did not help (MATH-500 6.8% to at most 7.7%). The authors say the same.
"LLMs can read any behaviour from weights" Exp 7 failed from raw weights in 4 of 4 attempts. It only worked once attention patterns were provided.
"It beats RL" True for Breakout against PPO at 500M steps. Not for Seaquest or Asterix.
"This is settled" One lab, mostly single runs, one frontier model, published as a research blog post rather than a peer-reviewed paper.

🟩 Independent reviews

Who When What they found
Prof. Andreas Maier 5 Nov 2026 A review that sorts the experiments into measured and anecdote. Calls two findings solid: frontier models can compile hand-designed programs into working weights, and one looping direction can be removed with a measured effect. Asks for multi-seed runs, other models, simple baselines and pre-fixed prompts.
Simon Willison 3 Nov 2026 Re-ran the no-tools reading test on his laptop with a local Qwen 3.8 27B (Q4_K_M): named Pong and Breakout, gave a Pac-Man-style gridworld, missed Freeway. Reading from weights is not unique to one frontier model.
Hugging Face re-run Nov 2026 Ahsen Khaliq rebuilt the edited checkpoint from the authors' logs, as covered here. Only two tensors differ from the original. Full MATH-500: 51.0%, 31.6% of answers hitting the token limit (authors: 48.1% and 33.1%). It re-runs the authors' own logs, so it checks the patch, not the method.
Matthew Honnibal 3 Nov 2026 Argued the post is misleading because it joins two different questions: can an LLM read weights, and can it edit them usefully.
Sean Goedecke Nov 2026 Walked through the Qwen fix and showed the steps are the ones a human ML researcher would take. Analogy: editing weights is like patching a compiled binary.

Key methodological takeaways

  1. Single runs. Most conditions were run once, including one run per forward-pass budget in Exp 8. Multi-seed evaluation (n ≥ 5) is needed before reading differences as real.
  2. Non-monotonic side effects. MATH-500 never fell as passes grew, but IFEval went 21.0% → 49.0% (10 passes) → 39.9% (25 passes) → 62.3% (50 passes). Possible interference is unmonitored.
  3. Clearing a choke, not imparting knowledge. The Gemma control and the authors' own wording both point the same way.
  4. Known-technique overlap. Critics note that rank-1 direction removal resembles directional ablation from the interpretability literature. Whether that is what the LLM did is not established.
  5. Missing baselines. No other LLM, no human or automated baseline, authors designed the tasks, and no comparison against cheaper fixes such as decoding settings or a standard direction ablation. That is exactly the gap WSP-Bench-Lite targets.

Everything unresolved is tracked publicly in the Honesty Ledger (12 open items at the last fact-check, 5 Nov 2026). Found a wrong number? Open a correction. Fixing errors is a feature.


Explore WSP-Bench-Lite Specification

🏆 WSP-Bench-Lite

A small open protocol to make WSP claims comparable. It is an original proposal in this repo (not from the paper's authors), status 🟥 speculation, v0.1, and no results exist yet. Spec: bench/WSP-Bench-Lite.md.

flowchart LR
    TASK["Task submission<br/>(JSON schema)"] --> PROTOCOL{{"WSP-Bench-Lite<br/>protocol rules"}}

    PROTOCOL --> M1["≥ 5 seeds<br/>mean and spread"]
    PROTOCOL --> M2["Mandatory baselines<br/>incl. random-direction control"]
    PROTOCOL --> M3["Edit sparsity s<br/>functional drift D"]
    PROTOCOL --> M4["Pre-registered<br/>prompt SHA-256"]

    M1 & M2 & M3 & M4 --> EVAL["Evaluation"]

    EVAL --> T1["READ track"]
    EVAL --> T2["FIX track"]
    EVAL --> T3["WRITE track"]

    T1 & T2 & T3 --> BOARD(["Geometric mean<br/>within each track"])

    classDef gate fill:#6d28d9,stroke:#4c1d95,color:#fff
    classDef track fill:#047857,stroke:#064e3b,color:#fff
    classDef task fill:#1e1b4b,stroke:#818cf8,color:#e0e7ff
    classDef board fill:#312e81,stroke:#a855f7,color:#f3e8ff

    class PROTOCOL gate
    class T1,T2,T3 track
    class TASK task
    class BOARD board
Loading
Track Task Mandatory baselines Metrics
🔍 READ Identify behaviour from weights alone (e.g. which of 5 procedurally generated games does this net play?) Human interpreter, automated receptive-field scanner, same heuristic written as code Accuracy, calibration, false-positive rate
🩺 FIX Repair a seeded defect with known ground truth (sign flip, dead column, loop) Decoding tweak, standard direction ablation, random-direction control Success rate, sparsity s, drift D, cost
✍️ WRITE Produce a working policy from a spec whose rules were held out by a third party Random init + short training, best published checkpoint Score, budget fidelity fidelity(n), cost

Two quantities every FIX attempt must report:

$$s = \frac{\lVert \Delta \rVert_0}{P} \qquad\qquad D = \mathbb{E}_{x},\mathrm{KL}!\left(p_\theta(\cdot \mid x),\Vert,p_{\theta'}(\cdot \mid x)\right)$$

Sparsity is the blast radius; drift is measured on frozen, task-unrelated probe sets so it cannot be gamed. Scores are geometric means within a track and never averaged across tracks: a great Fixer is not automatically a great Writer. Beating the random-direction ablation is the entry bar, not the finish line.


Explore Chapter 2: How Weights Get Written

🧭 Where WSP sits in history

timeline
    title Four Eras of Neural Network Weight Manipulation (1986 – 2026+)
    1986–2020 : Era 1 · Train It
              : Backpropagation, SGD & Adam
              : Weights treated as opaque continuous floats
    2021–2023 : Era 2 · Adapt It
              : LoRA Low-Rank Adapters (Hu et al.)
              : Model Soups & Task Vector Arithmetic (Ilharco et al.)
    2022–2024 : Era 3 · Edit Its Facts
              : ROME & MEMIT Rank-1 Fact Editing (Meng et al.)
              : Causal Tracing & Directional Ablation
    2026+     : Era 4 · Program It (WSP)
              : Frontier LLMs natively READ, FIX & WRITE raw weights
              : Closed-form surgical edits with zero gradient descent
Loading

This four-era framing is this repo's own, for teaching. Chapters 2 and 3 of the atlas compare WSP with fine-tuning, LoRA, task-vector merging and ROME-style editing. Keep in mind that rank-1 direction removal resembles directional ablation, which is worth remembering when judging what is genuinely new.


🗺️ Repository map

Atlas Interactive Next.js 15 web application with WebGL 3D weight lattice, KaTeX math engine, light/dark themes, and 26 interactive routes (located in [weights-are-code (atlas)/](<weights-are-code (atlas)>/), deployed at weight-space-atlas.vercel.app).
Labs Python 3.10+, NumPy. No other runtime dependency.
Paper LaTeX, IEEE-style monograph with vector figures.
Diagrams Mermaid & custom dark/light vector SVGs in assets/readme/, rendered natively by GitHub.
CI GitHub Actions runs the labs and tests on Ubuntu, Windows and macOS across Python 3.10 to 3.12, and validates fact-sheet.json.

Explore the Interactive FAQ on the Atlas

❓ FAQ

Do I need a GPU or an API key?
No. The four labs are pure NumPy and each runs in under a second on a laptop (Lab 4 also needs an internet connection to fetch the authors' released prompt files). The atlas runs in your browser.
Is "no gradient descent" the same as "no forward passes"?
No. The LLM never used gradients or a training loop. But in Exp 8 it could run up to 50 forward passes of the Qwen model in a Python kernel to look at behaviour, and the big gain only appeared at 50. Only Exp 1 to 3 were fully static. See what the evidence supports.
So did it teach Qwen to do maths?
No. It removed a looping failure that was exhausting the token budget. A control on Gemma 3 270M, which rarely loops, did not improve. The authors make the same point.
Is this peer reviewed?
No. The source is a research blog post from MIT CSAIL. Independent reviewers have already responded (see above), and the Honesty Ledger tracks what is still open.
How is this different from fine-tuning, LoRA or ROME/MEMIT?
Those methods compute an update with an optimizer or a dedicated algorithm. In WSP a general-purpose LLM reasons about the weights as text and chooses the edit itself. The edit in Exp 8 is simple linear algebra, which is exactly why it is easy to verify. Chapter 3 of the atlas compares the cousins.
Could this be misused?
Cheap, targeted weight edits could in principle strip guardrails or plant backdoors, and defenders could use the same tools to audit models. Chapter 11 maps the risks and defences responsibly, without operational detail.
Is this affiliated with MIT or the authors?
No. It is an independent explainer by Het Patel. All discovery credit belongs to the MIT CSAIL authors.
Can I reuse the visuals or text?
Yes. Code is MIT. Prose, whitepaper and figures are CC BY 4.0: reuse them with attribution.

🤝 Contribute

Corrections, replications and new labs are welcome. The most useful contributions, in order:

Contribution Start here
1 Replicate with more seeds. Run an experiment with n ≥ 5 on another model family (Llama, Mistral, Gemma 2, DeepSeek) and send the logs. bench/WSP-Bench-Lite.md
2 Break a claim. Pick an item from the Honesty Ledger and try to resolve it either way. research/open-verification-items.md
3 Write a new lab, for example 05_attention_surgeon.py, or sweep the surgery size k from 1 to 64 in Lab 2 and plot residual energy against sparsity. lab/README.md
4 Improve the atlas: translations, accessibility, new interactive modules. weight-space-atlas.vercel.app
5 Spot a wrong number. Fixing errors is a feature. Open a correction

Read CONTRIBUTING.md first, and please follow the Code of Conduct. Have a feature idea? Use the feature request form.

Contributors

⚖️ Legal & academic disclaimer

  • Original research attribution. All primary discovery credit for Weight-Space Programming belongs to Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab (MIT CSAIL).
  • Independence. This repository is an independent educational and research compilation by Het Patel (@Hunterdii). It is not endorsed by, sponsored by, or affiliated with MIT, MIT CSAIL, the original authors, or any model vendor.
  • Licensing. Code, benchmarks and Python labs: MIT License. Text, curricula, whitepaper and visual assets: CC BY 4.0.

Weights are code. Edit 0.016% of an 800M-parameter model. No backprop. No fine-tuning. No GPU cluster. READ, FIX, WRITE. MATH-500 7.9% to 48.1%, 131,072 params.

Built with intellectual honesty, mathematical rigour and open science.

⭐ If this made weights feel less like a black box, a star helps others find it. ⭐

Star history chart

Curated by Het Patel (@Hunterdii), student · LinkedIn · Sponsor · Deployed at weight-space-atlas.vercel.app

About

Interactive explainer + research whitepaper on Weight-Space Programming: LLMs that read, fix and write neural-network weights without gradient descent. From "knobs" to frontier, with paper-reported numbers, independent critique, risks, and a proposed WSP-Bench-Lite.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages