|
Live Interactive Atlas 26 interactive routes · 5 audience levels · Parameter knife |
Monograph Whitepaper IEEE-typeset monograph · 13 vector figures · Formal proofs |
WSP-Bench-Lite Multi-seed protocol (n ≥ 5) · 3 competition tracks |
|
4 Zero-Dependency Labs Pure Python & NumPy · Runs in <1 second on any laptop |
Verified Fact Sheet Audited numbers · 4-tier epistemic badges · Verified dates |
The Honesty Ledger Public tracker of open audit items, caveats & variance flags |
| ⭐ Repository Stars | 🍴 Forks | 🐛 Open Issues | ⏱️ Last Commit | 🐍 Python Runtime | 📦 Dependencies | 🛡️ Truth Contract |
|
|
|
|
|
|
|
|
Live atlas · Run a lab in 30 s · The 8 experiments · What the evidence supports · Whitepaper · Benchmark · FAQ
Headline figures are paper-reported and come from single runs. See exactly what they do and do not show.
Important
Independent companion, not the original research. Weight-Space Programming (WSP) comes from the MIT CSAIL post "Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights" by Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab (2 Nov 2026). All scientific credit is theirs. This repository explains it, re-derives the small mechanics in code, and audits how far the evidence goes. It is not affiliated with or endorsed by MIT, MIT CSAIL or the authors.
For forty years we changed neural networks by training them: show data, measure error, nudge millions of numbers along a gradient. The MIT CSAIL post reports something stranger. A general-purpose frontier LLM (GPT-6 Astra in the post) was handed another network's raw weights as text, worked out what each part computes, and edited the numbers directly. No optimizer, no training loop.
This repo is the part that lets you check that claim: play with it, run it, read the maths, and see exactly where the evidence stops.
- 💡 The Idea in 30 Seconds
- 🧭 The 4-Era Evolution of Model Programming
- 📐 Architectural Blueprints & System Mechanics
- 🧭 Choose Your Path
- 🌐 The Live Atlas
- ⚡ The Surgery, in Motion
- ⏱️ Run It in 30 Seconds
- 📊 The Eight Experiments
- 🧮 The Maths, Small Enough to Hold in Your Head
- 🛡️ The Epistemic Truth Contract & Honesty Ledger
- 🏆 WSP-Bench-Lite
- 🧭 Where WSP Sits in History
- 🗺️ Repository Map
- ❓ FAQ
- 🤝 Contribute
- ⚖️ Legal & Academic Disclaimer
For forty years we changed neural networks by training them: show data, measure error, nudge millions of numbers along a gradient. The weights were an opaque pile.
The MIT CSAIL post reports something stranger. A general-purpose frontier LLM (GPT-6 Astra in the post) was handed another network's raw weights as text, worked out what each part computes, and edited the numbers directly. No optimizer. No training loop.
| 🔍 READ | 🩺 FIX | ✍️ WRITE | |
|---|---|---|---|
| Question | What was this network trained to do? | Why does it fail, and what is the smallest edit that stops it? | Can you write a working network from nothing? |
| Headline | Named the right game for all 4 networks from 17,253 raw weights each | Ramp-climbing car: 0 / 256 → 1,024 / 1,024 by editing 3.5% of weights | Breakout from all-zero weights: 517.4 vs 25.3 for PPO at 500M steps |
| Flagship | Exp 1 to 3 | Exp 8: Qwen3.5-0.8B, 0.016% of weights, MATH-500 7.9% → 48.1% | Exp 5 and 6 |
| Deep dive | Ch 4 | Ch 5 | Ch 6 |
---
config:
look: handDrawn
theme: dark
---
flowchart LR
W[("Raw weights<br/>as text")] --> L{{"General-purpose LLM<br/>reasoning, no optimizer"}}
L -->|READ| R["What does it compute?<br/>game, shape, language, circuit"]
L -->|FIX| F["Which direction causes the failure?<br/>W' = (I − u uᵀ) W"]
L -->|WRITE| X["Hand-compile the layers<br/>from an all-zero start"]
R --> V{{"Check it"}}
F --> V
X --> V
V -->|"not yet"| L
V -->|"works"| OUT(["Patched or<br/>synthesised network"])
classDef core fill:#6d28d9,stroke:#4c1d95,color:#fff
classDef read fill:#0e7490,stroke:#155e75,color:#fff
classDef fix fill:#b91c1c,stroke:#7f1d1d,color:#fff
classDef write fill:#b45309,stroke:#78350f,color:#fff
classDef neutral fill:#334155,stroke:#1e293b,color:#fff
class L core
class R read
class F fix
class X write
class W,V,OUT neutral
Note
What "no gradient descent" does and does not mean. Experiments 1 to 3 used no tools and no forward passes. Experiments 6 and 8 gave the LLM a Python sandbox and a budget of forward passes (up to 50) to look at behaviour. There were never gradients or training loops, but the model was not always working blind. Details in What the evidence supports.
flowchart TD
subgraph E1["Era 1 (1986–2020): Opaque Continuous Variables"]
direction LR
A1["Loss Function L(θ)"] --> B1["Backprop ∇_θ L"] --> C1["Iterative SGD / Adam"]
end
subgraph E2["Era 2 (2021–2023): Subspace Geometry & Linear Arithmetic"]
direction LR
A2["Base Weights W_0"] --> B2["LoRA Adapters (B·A)"] --> C2["Task Vector Merging"]
end
subgraph E3["Era 3 (2022–2024): Mechanistic Interpretability & Editing"]
direction LR
A3["Factual Triplet (s, r, o)"] --> B3["Activation Patching"] --> C3["ROME / MEMIT Rank-1 Updates"]
end
subgraph E4["Era 4 (2026+): Native Weight-Space Programming (WSP)"]
direction LR
A4["Raw Parameter Tensors"] --> B4["Frontier LLM Reasoning"] --> C4["Closed-Form Surgery (0 Gradients)"]
end
E1 ==>|Geometry Shift| E2
E2 ==>|Circuits Shift| E3
E3 ==>|Symbolic Shift| E4
style E1 fill:#111827,stroke:#4b5563,stroke-width:1px,color:#f3f4f6
style E2 fill:#0f172a,stroke:#3b82f6,stroke-width:1px,color:#f8fafc
style E3 fill:#1e1b4b,stroke:#8b5cf6,stroke-width:1px,color:#f5f3ff
style E4 fill:#062d22,stroke:#10b981,stroke-width:2px,color:#ecfdf5
flowchart TD
LLM["🧠 Frontier Reasoning Engine (LLM)"]
LLM -->|"1. READ VERB"| R_PIPE
LLM -->|"2. FIX VERB"| F_PIPE
LLM -->|"3. WRITE VERB"| W_PIPE
subgraph R_PIPE["1. READ: Weight Deconstruction"]
direction TB
R1["Raw Weight Tensors"] --> R2["Frobenius Induction Norms"] --> R3["Identified Policy / Circuit Map"]
end
subgraph F_PIPE["2. FIX: Attractor Surgery"]
direction TB
F1["Degraded Checkpoint (Loops)"] --> F2["Rank-1 Projection W'=(I-uuᵀ)W"] --> F3["Excised Runaway Attractor"]
end
subgraph W_PIPE["3. WRITE: Symbolic Compilation"]
direction TB
W1["Behavioral Logic Spec"] --> W2["Symbolic Logic Synthesis"] --> W3["Zero-Gradient Neural Policy"]
end
style LLM fill:#2e1065,stroke:#a855f7,stroke-width:2px,color:#faf5ff
style R_PIPE fill:#082f49,stroke:#0284c7,stroke-width:1px,color:#f0f9ff
style F_PIPE fill:#450a0a,stroke:#dc2626,stroke-width:1px,color:#fef2f2
style W_PIPE fill:#064e3b,stroke:#059669,stroke-width:1px,color:#ecfdf5
How did modifying only 131,072 parameters out of 800,000,000 rescue Qwen3.5-0.8B from an infinite overthinking loop?
flowchart LR
IN["Input Prompt"] --> PRE["Layers 01–14<br>(Untouched)"]
subgraph SURGERY["Surgical Intervention Zone (0.016% Sparsity Budget)"]
direction TB
L15["Layer 15 MLP Block<br>64 Columns Edited (65,536 params)"]
MID["Layers 16–18<br>(Untouched Residual Stream)"]
L19["Layer 19 MLP Block<br>64 Columns Edited (65,536 params)"]
L15 --> MID --> L19
end
PRE --> L15
L19 --> POST["Layers 20–24<br>(Untouched)"] --> OUT["Model Output"]
subgraph RESULTS["Empirical Impact (Qwen3.5-0.8B)"]
direction TB
RES1["MATH-500: 7.9% ➔ 48.1% (+40.2%)"]
RES2["GSM8K: 13.8% ➔ 67.1% (+53.3%)"]
RES3["Timeouts: 93% ➔ 33% (-60%)"]
RES4["Total Edited: 131,072 / 800M (0.016%)"]
end
OUT -.-> RESULTS
style SURGERY fill:#18181b,stroke:#ef4444,stroke-width:1px,color:#f4f4f5
style RESULTS fill:#064e3b,stroke:#10b981,stroke-width:2px,color:#ecfdf5
| You are... | Time | Do this |
|---|---|---|
| Curious (no maths needed) | 2 min | Open the atlas, drag the surgery slider, then read Ch 1, A Network Is a Pile of Numbers. |
| ML engineer | 10 min | Run lab/02_direction_surgery.py, then read Ch 5, FIX. |
| Safety or security reader | 15 min | Read Ch 11, The Severe Stuff, a risk atlas told responsibly. |
| Researcher or reviewer | 1 hour | Read the whitepaper, then what the evidence supports and the Honesty Ledger. |
| Builder | a weekend | Pick a contribution: replicate on another model family, or write Lab 5. |
weight-space-atlas.vercel.app is a free interactive field guide to the paper. No sign-up. Light and dark themes.
The landing page. Open it live and hover the lattice to read a weight.
| Feature | What you get |
|---|---|
| 3D weight lattice | A 64×64 matrix standing in for Layer 15's MLP W_gate. Hover to read a weight, switch between Read, Fix and Write, drag ΔW. |
| Rank-1 surgery simulator | Drag α in W′ = (I − α·uuᵀ)W and watch the Exp 8 numbers move. |
| Fingerprint lab | The Atari weight fingerprints of Exp 1, recomputed live from the authors' published prompts. |
| Five reading levels | Every chapter reads from curious observer up to frontier researcher. |
| Truth Contract | Press T and everything that is not a paper-reported fact disappears. |
| Adaptive pathways | Curious observer, ML engineer, safety auditor, academic researcher. |
| Instruments | 13 chapters, 15 interactive modules, 8 experiment pages, a 15-question quiz, 60+ glossary terms, 12 FAQs and 44 references. |
The 13-chapter curriculum
| Part | # | Chapter | Read |
|---|---|---|---|
| I · Foundations | 01 | A Network Is a Pile of Numbers | 9 min |
| 02 | How Weights Get Written: Three Old Ways, One New | 10 min | |
| 03 | Cousins of WSP: Who Else Touches the Weights? | 9 min | |
| II · The Experiments | 04 | READ: Guessing Games With Real Weight Lists | 13 min |
| 05 | FIX: Surgery on a Ramp and on a Thought Loop | 13 min | |
| 06 | WRITE: Policies Born From Pure Thought | 11 min | |
| 07 | How Did It Do It? Two Mechanisms | 9 min | |
| III · Judgement | 08 | Measured vs Anecdote: Sorting the Evidence | 9 min |
| 09 | Small Models, Big Models: Who Benefits? | 8 min | |
| IV · Futures & Risks | 10 | What Comes Next: Predictions and the Horizon Map | 8 min |
| 11 | The Severe Stuff: a Risk Atlas, Told Responsibly | 10 min | |
| 12 | Het's Lab: Three Runnables and a Benchmark Proposal | 9 min | |
| 13 | The Library: Glossary, Quiz, References, Open Items | 6 min |
Shortcuts: Prologue · 8 experiments · Labs · Quiz · Glossary · References · FAQ · Scoreboard and audit
The flagship result is Experiment 8. Qwen3.5-0.8B, running in thinking mode, kept falling into a reasoning loop: 93.0% of its answers hit the token limit before finishing. The LLM found a direction behind the loop and projected it out of 64 MLP output columns in each of two layers.
🟨 A teaching schematic, not model data. Only the endpoints in the table below are paper-reported. Read Ch 5, FIX →
Qwen3.5-0.8B, by the number of forward passes the LLM was allowed (🟦 paper-reported):
| MATH-500 | GSM8K | IFEval | Hit token limit | |
|---|---|---|---|---|
| Base model | 7.9% | 13.8% | 21.0% | 93.0% |
| Edited, 10 passes | 12.3% | 25.7% | 49.0% | 87.2% |
| Edited, 25 passes | 14.1% | 25.1% | 39.9% | 86.4% |
| Edited, 50 passes | 48.1% | 67.1% | 62.3% | 33.1% |
xychart-beta
title "Qwen3.5-0.8B on MATH-500 by forward passes (%)"
x-axis ["base", "10 passes", "25 passes", "50 passes"]
y-axis "MATH-500 accuracy (%)" 0 --> 60
bar [7.9, 12.3, 14.1, 48.1]
The edit touches two layers (131,072 weights) and leaves 99.984% of the 800M-parameter network completely untouched:
sankey-beta
"Qwen3.5-0.8B (800M Params)","22 Untouched Layers (733M)",732891136
"Qwen3.5-0.8B (800M Params)","Layer 15 & 19 (67.1M)",67108864
"Layer 15 & 19 (67.1M)","Unedited MLP & Attn (66.98M)",66977792
"Layer 15 & 19 (67.1M)","Layer 15 Edited Cols (65,536)",65536
"Layer 15 & 19 (67.1M)","Layer 19 Edited Cols (65,536)",65536
sequenceDiagram
autonumber
participant A as Reasoning LLM
participant K as Python kernel
participant Q as Qwen3.5-0.8B weights
participant E as Evaluator
A->>K: load Qwen3.5-0.8B and run forward passes (10, 25 or 50)
K-->>A: activations show a looping direction u
A->>Q: project u out of 64 MLP output columns in layers 15 and 19
Note over Q: 131,072 of 800M weights edited (0.016%)
Q->>E: MATH-500, GSM8K, IFEval, token-limit hits
E-->>A: with 50 passes MATH-500 goes 7.9% to 48.1%
Note over E: A control on Gemma 3 270M, which rarely loops, did not improve
Tip
The one-line takeaway. The edit did not teach Qwen maths. It removed an obstacle: a loop that ate the token budget so the model never finished answers it could already work out. The authors say so themselves, and a Gemma 3 270M control (which rarely loops) did not improve.
Python 3.10+ and NumPy. No GPU, no PyTorch, no API keys.
git clone https://github.com/Hunterdii/weight-space-programming-explained.git
cd weight-space-programming-explained
pip install -r requirements.txt
python lab/01_compile_a_rule.py # WRITE: hand-compile a tiny network that plays paddle
python lab/02_direction_surgery.py # FIX: rank-1 surgery, (I − u uᵀ) W
python lab/03_blast_radius.py # MEASURE: verify the 0.016% arithmetic
python lab/04_fingerprint_games.py # READ: recompute the Exp 1 weight fingerprintsReal output from a clean run:
$ python lab/01_compile_a_rule.py
all asserts passed - your 17-number brain plays paddle
$ python lab/02_direction_surgery.py
max |u^T W'| = 6.661338147750939e-16
edited 8/64 columns; remaining u-energy 0.74
$ python lab/03_blast_radius.py
0.0164%
131072
$ python lab/04_fingerprint_games.py
breakout strongest row 8 (58%) strongest column 8 (17%)
pong strongest row 2 (14%) strongest column 1 (55%)
pacman strongest row 8 (16%) strongest column 1 (16%)
freeway strongest row 6 (20%) strongest column 4 (22%)
Lab 4 is the fun one. Breakout's agent weights pile into row 8 (58%): the bottom row, where the paddle lives. Pong's pile into column 1 (55%). That is the kind of structure the LLM read in Exp 1. (Lab 4 downloads the authors' released prompt files, so it needs an internet connection; Labs 1 to 3 are fully offline.)
| Lab | Verb | What it shows | Tier |
|---|---|---|---|
01_compile_a_rule.py |
WRITE | A rule can be turned into weights with no training | 🟨 schematic |
02_direction_surgery.py |
FIX | (I − uuᵀ)W removes a direction exactly (error ≈ 1e-16) |
🟨 schematic |
03_blast_radius.py |
MEASURE | 131,072 / 800M ≈ 0.0164% | 🟦 arithmetic on paper-reported figures |
04_fingerprint_games.py |
READ | Atari policies have visible structure in layer 1 | 🟦 recomputed from released data |
Lab 1 in 11 lines: a paddle policy with zero training
import numpy as np
relu = lambda z: np.maximum(z, 0.0)
# Two hidden units: h0 fires when the ball is left of the paddle, h1 when it is right.
W1 = np.array([[-1., 1.], [1., -1.]]); b1 = np.array([-0.05, -0.05])
W2 = np.array([[1., 0.], [0., 0.], [0., 1.]]); b2 = np.array([0., 0.05, 0.])
ACTIONS = ["left", "stay", "right"]
def act(ball, paddle):
h = relu(W1 @ np.array([ball, paddle]) + b1)
return ACTIONS[int(np.argmax(W2 @ h + b2))]
assert act(0.2, 0.7) == "left" and act(0.8, 0.3) == "right" and act(0.5, 0.52) == "stay"No training run produced those numbers. A rule was written down and the weights were derived from it. That is the whole WRITE idea in miniature. More in lab/README.md.
Every number is from the original post (🟦 paper-reported); the authors released their logs at RikiyaT/lm_understand-logs. None of these are replications by this repo. The last column is how the independent Maier review sorts each result (🟩).
| # | Verb | Setup | Result | Independent read |
|---|---|---|---|---|
| 1 | 🔍 Read | 4 PPO policies, MLP 500→32→32→5, 17,253 weights each (Breakout, Pong, Pac-Man, Freeway). No tools, options not revealed | Right game for 4 / 4. Breakout named with 99% confidence | Anecdote: toy layouts give a lot away |
| 2 | 🔍 Read | 3 diffusion models (circle, square, triangle), MLP 160→64→64→144, 23,824 weights each. No forward pass | Right shape for 3 / 3, drawn as outlines although the models generate filled shapes | Anecdote |
| 3 | 🔍 Read | 3 character transformers (7.16M weights, 4 layers × 6 heads), middle blocks only, no labels, no running | Right language for 6 / 6 attempts (confidence 0.48 to 0.62). Every copying head matched ground truth, none falsely named | Language: weak. Copying heads: the solid exception |
| 4 | 🩺 Fix | Kinetix car-on-ramp actor, 171,664 weights, PPO stuck at 0 / 256 after 1B steps | One edit, no tools, 3.5% of weights: 0 / 256 → 1,024 / 1,024 | Measured, but one task and one edit |
| 5 | ✍️ Write | MinAtar Breakout, Seaquest, Asterix. 600→32→32→6 tanh network, 20,486 weights (computed here), all start at 0, no data, no tools | Beats PPO at 100M steps in all 3. Breakout 517.4 | Measured: hand-designed programs, not understanding of trained weights |
| 6 | ✍️ Write | 9×9 Go, 243→512×6→82, 1,480,274 weights, Python sandbox, 0 to 50 forward passes | 0 passes: wins more than half its games against two small PPO nets (a single 336-line Python run). 50 passes: beats the equal-sized PPO net trained on 20M games | Measured. Win rates 54–55% (0 passes) and 81–84% (50 passes) vs small PPO nets |
| 7 | 🔍 Read | 2 Gemma 3 270M models, one fine-tuned to be sycophantic | From raw weights alone: failed 4 / 4. Given only attention patterns on its own texts: picked the sycophant in 3 / 3 pairs | Weak: chance alone gives 3 / 3 one time in eight |
| 8 | 🩺 Fix | Qwen3.5-0.8B (800M weights), thinking mode, Python kernel, 10 / 25 / 50 forward passes | 131,072 weights edited (0.016%). With 50 passes: MATH-500 7.9% → 48.1% | Measured, single run per budget |
Experiment 5, both sides. Scores from the post (higher is better):
| Game | PPO 100M steps | PPO 500M steps | LLM, one shot |
|---|---|---|---|
| Breakout | 15.8 | 25.3 | 517.4 |
| Seaquest | 5.6 | 88.9 | 53.1 |
| Asterix | 25.8 | 55.2 | 53.2 |
The "20×" headline is real, but it is Breakout only. Against the stronger PPO baseline the LLM is close in Asterix and reaches about 60% of the score in Seaquest.
Experiment 7 in detail: the sycophancy twins
Three fine-tuned Gemma 3 270M pairs (18 layers). The sycophant follows 100% of wrong user guesses; the honest model follows 1 to 3%. The LLM could not map weight differences to behaviour from raw weights (0 of 4). Once it was allowed attention patterns on up to 16 texts, it picked the sycophant in all three pairs.
The signal it found sits in head 10.2, read from the answer position:
| Model | Attention on the question's numbers | Attention on the user's answer |
|---|---|---|
| Honest | 94.3% | 0.75% |
| Sycophant | 3.5% | 85.7% |
Experiment 8 in detail: the Gemma 3 270M control
Gemma rarely loops (8.5% of its answers hit the token limit), so there was little obstacle to remove. No edit moved MATH-500 by more than about a point.
| Forward passes | 0 (base) | 5 | 10 | 25 | 50 |
|---|---|---|---|---|---|
| Gemma 3 270M, MATH-500 | 6.8% | 7.0% | 7.6% | 7.7% | 7.0% |
The authors' own reading: editing weights "could remove an obstacle that hid an ability, but it could not add an ability the model lacked."
Let
Because
Everything aligned with lab/02_direction_surgery.py checks this numerically (max |uᵀW'| ≈ 1e-16) and also shows the sparse variant: editing only the 8 strongest of 64 columns leaves about 74% of the original u-component.
Roughly one weight in 6,100. The post reports 131,072 weights in 64 columns × 2 layers; the hidden size of 1,024 is inferred from that arithmetic and tracked as ledger item 2 in the Honesty Ledger.
Following the transformer-circuits literature (Elhage et al.), copying behaviour can be probed from weights alone by how strongly one head's query-key map composes with another's output-value map:
🟨 This is a teaching formula showing the kind of quantity that is readable from weights. The post reports Exp 3 in terms of the copying heads the LLM named, not a claim that it computed this score.
WSP is exciting and very early; the authors themselves call these results sparks. This repo's rule is simple: every number carries exactly one badge, and the caveats sit next to the headline.
| Badge | Tier | What it means |
|---|---|---|
| 🟦 | Paper-reported | Stated in Takehi et al. (MIT CSAIL, 2 Nov 2026), checked against the released logs. |
| 🟩 | Independent review | Scrutinised or re-run by someone outside the authors. |
| 🟨 | Teaching schematic | Toy data or pedagogical code written to explain a mechanism. Not a result. |
| 🟥 | Speculation | A prediction or roadmap idea, including the authors' own. |
On the atlas, press T to hide everything that is not a paper-reported fact.
| Claim people make | What the evidence supports |
|---|---|
| "LLMs fixed a model with zero forward passes" | Only Exp 1 to 3 and the 0-pass Go run were that blind. Exp 8's 48.1% used 50 forward passes in a Python harness. With 10 or 25 passes the gain was small. |
| "It made the model better at maths" | It removed an obstacle: 93% of answers were hitting the token limit. On Gemma 3 270M, which rarely loops, the edits did not help (MATH-500 6.8% to at most 7.7%). The authors say the same. |
| "LLMs can read any behaviour from weights" | Exp 7 failed from raw weights in 4 of 4 attempts. It only worked once attention patterns were provided. |
| "It beats RL" | True for Breakout against PPO at 500M steps. Not for Seaquest or Asterix. |
| "This is settled" | One lab, mostly single runs, one frontier model, published as a research blog post rather than a peer-reviewed paper. |
| Who | When | What they found |
|---|---|---|
| Prof. Andreas Maier | 5 Nov 2026 | A review that sorts the experiments into measured and anecdote. Calls two findings solid: frontier models can compile hand-designed programs into working weights, and one looping direction can be removed with a measured effect. Asks for multi-seed runs, other models, simple baselines and pre-fixed prompts. |
| Simon Willison | 3 Nov 2026 | Re-ran the no-tools reading test on his laptop with a local Qwen 3.8 27B (Q4_K_M): named Pong and Breakout, gave a Pac-Man-style gridworld, missed Freeway. Reading from weights is not unique to one frontier model. |
| Hugging Face re-run | Nov 2026 | Ahsen Khaliq rebuilt the edited checkpoint from the authors' logs, as covered here. Only two tensors differ from the original. Full MATH-500: 51.0%, 31.6% of answers hitting the token limit (authors: 48.1% and 33.1%). It re-runs the authors' own logs, so it checks the patch, not the method. |
| Matthew Honnibal | 3 Nov 2026 | Argued the post is misleading because it joins two different questions: can an LLM read weights, and can it edit them usefully. |
| Sean Goedecke | Nov 2026 | Walked through the Qwen fix and showed the steps are the ones a human ML researcher would take. Analogy: editing weights is like patching a compiled binary. |
- Single runs. Most conditions were run once, including one run per forward-pass budget in Exp 8. Multi-seed evaluation (n ≥ 5) is needed before reading differences as real.
- Non-monotonic side effects. MATH-500 never fell as passes grew, but IFEval went 21.0% → 49.0% (10 passes) → 39.9% (25 passes) → 62.3% (50 passes). Possible interference is unmonitored.
- Clearing a choke, not imparting knowledge. The Gemma control and the authors' own wording both point the same way.
- Known-technique overlap. Critics note that rank-1 direction removal resembles directional ablation from the interpretability literature. Whether that is what the LLM did is not established.
- Missing baselines. No other LLM, no human or automated baseline, authors designed the tasks, and no comparison against cheaper fixes such as decoding settings or a standard direction ablation. That is exactly the gap WSP-Bench-Lite targets.
Everything unresolved is tracked publicly in the Honesty Ledger (12 open items at the last fact-check, 5 Nov 2026). Found a wrong number? Open a correction. Fixing errors is a feature.
A small open protocol to make WSP claims comparable. It is an original proposal in this repo (not from the paper's authors), status 🟥 speculation, v0.1, and no results exist yet. Spec: bench/WSP-Bench-Lite.md.
flowchart LR
TASK["Task submission<br/>(JSON schema)"] --> PROTOCOL{{"WSP-Bench-Lite<br/>protocol rules"}}
PROTOCOL --> M1["≥ 5 seeds<br/>mean and spread"]
PROTOCOL --> M2["Mandatory baselines<br/>incl. random-direction control"]
PROTOCOL --> M3["Edit sparsity s<br/>functional drift D"]
PROTOCOL --> M4["Pre-registered<br/>prompt SHA-256"]
M1 & M2 & M3 & M4 --> EVAL["Evaluation"]
EVAL --> T1["READ track"]
EVAL --> T2["FIX track"]
EVAL --> T3["WRITE track"]
T1 & T2 & T3 --> BOARD(["Geometric mean<br/>within each track"])
classDef gate fill:#6d28d9,stroke:#4c1d95,color:#fff
classDef track fill:#047857,stroke:#064e3b,color:#fff
classDef task fill:#1e1b4b,stroke:#818cf8,color:#e0e7ff
classDef board fill:#312e81,stroke:#a855f7,color:#f3e8ff
class PROTOCOL gate
class T1,T2,T3 track
class TASK task
class BOARD board
| Track | Task | Mandatory baselines | Metrics |
|---|---|---|---|
| 🔍 READ | Identify behaviour from weights alone (e.g. which of 5 procedurally generated games does this net play?) | Human interpreter, automated receptive-field scanner, same heuristic written as code | Accuracy, calibration, false-positive rate |
| 🩺 FIX | Repair a seeded defect with known ground truth (sign flip, dead column, loop) | Decoding tweak, standard direction ablation, random-direction control | Success rate, sparsity s, drift D, cost |
| ✍️ WRITE | Produce a working policy from a spec whose rules were held out by a third party | Random init + short training, best published checkpoint | Score, budget fidelity fidelity(n), cost |
Two quantities every FIX attempt must report:
Sparsity is the blast radius; drift is measured on frozen, task-unrelated probe sets so it cannot be gamed. Scores are geometric means within a track and never averaged across tracks: a great Fixer is not automatically a great Writer. Beating the random-direction ablation is the entry bar, not the finish line.
timeline
title Four Eras of Neural Network Weight Manipulation (1986 – 2026+)
1986–2020 : Era 1 · Train It
: Backpropagation, SGD & Adam
: Weights treated as opaque continuous floats
2021–2023 : Era 2 · Adapt It
: LoRA Low-Rank Adapters (Hu et al.)
: Model Soups & Task Vector Arithmetic (Ilharco et al.)
2022–2024 : Era 3 · Edit Its Facts
: ROME & MEMIT Rank-1 Fact Editing (Meng et al.)
: Causal Tracing & Directional Ablation
2026+ : Era 4 · Program It (WSP)
: Frontier LLMs natively READ, FIX & WRITE raw weights
: Closed-form surgical edits with zero gradient descent
This four-era framing is this repo's own, for teaching. Chapters 2 and 3 of the atlas compare WSP with fine-tuning, LoRA, task-vector merging and ROME-style editing. Keep in mind that rank-1 direction removal resembles directional ablation, which is worth remembering when judging what is genuinely new.
| Atlas | Interactive Next.js 15 web application with WebGL 3D weight lattice, KaTeX math engine, light/dark themes, and 26 interactive routes (located in [weights-are-code (atlas)/](<weights-are-code (atlas)>/), deployed at weight-space-atlas.vercel.app). |
| Labs | Python 3.10+, NumPy. No other runtime dependency. |
| Paper | LaTeX, IEEE-style monograph with vector figures. |
| Diagrams | Mermaid & custom dark/light vector SVGs in assets/readme/, rendered natively by GitHub. |
| CI | GitHub Actions runs the labs and tests on Ubuntu, Windows and macOS across Python 3.10 to 3.12, and validates fact-sheet.json. |
Do I need a GPU or an API key?
No. The four labs are pure NumPy and each runs in under a second on a laptop (Lab 4 also needs an internet connection to fetch the authors' released prompt files). The atlas runs in your browser.
Is "no gradient descent" the same as "no forward passes"?
No. The LLM never used gradients or a training loop. But in Exp 8 it could run up to 50 forward passes of the Qwen model in a Python kernel to look at behaviour, and the big gain only appeared at 50. Only Exp 1 to 3 were fully static. See what the evidence supports.
So did it teach Qwen to do maths?
No. It removed a looping failure that was exhausting the token budget. A control on Gemma 3 270M, which rarely loops, did not improve. The authors make the same point.
Is this peer reviewed?
No. The source is a research blog post from MIT CSAIL. Independent reviewers have already responded (see above), and the Honesty Ledger tracks what is still open.
How is this different from fine-tuning, LoRA or ROME/MEMIT?
Those methods compute an update with an optimizer or a dedicated algorithm. In WSP a general-purpose LLM reasons about the weights as text and chooses the edit itself. The edit in Exp 8 is simple linear algebra, which is exactly why it is easy to verify. Chapter 3 of the atlas compares the cousins.
Could this be misused?
Cheap, targeted weight edits could in principle strip guardrails or plant backdoors, and defenders could use the same tools to audit models. Chapter 11 maps the risks and defences responsibly, without operational detail.
Is this affiliated with MIT or the authors?
No. It is an independent explainer by Het Patel. All discovery credit belongs to the MIT CSAIL authors.
Can I reuse the visuals or text?
Yes. Code is MIT. Prose, whitepaper and figures are CC BY 4.0: reuse them with attribution.
Corrections, replications and new labs are welcome. The most useful contributions, in order:
| Contribution | Start here | |
|---|---|---|
| 1 | Replicate with more seeds. Run an experiment with n ≥ 5 on another model family (Llama, Mistral, Gemma 2, DeepSeek) and send the logs. | bench/WSP-Bench-Lite.md |
| 2 | Break a claim. Pick an item from the Honesty Ledger and try to resolve it either way. | research/open-verification-items.md |
| 3 | Write a new lab, for example 05_attention_surgeon.py, or sweep the surgery size k from 1 to 64 in Lab 2 and plot residual energy against sparsity. |
lab/README.md |
| 4 | Improve the atlas: translations, accessibility, new interactive modules. | weight-space-atlas.vercel.app |
| 5 | Spot a wrong number. Fixing errors is a feature. | Open a correction |
Read CONTRIBUTING.md first, and please follow the Code of Conduct. Have a feature idea? Use the feature request form.
- Original research attribution. All primary discovery credit for Weight-Space Programming belongs to Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab (MIT CSAIL).
- Independence. This repository is an independent educational and research compilation by Het Patel (@Hunterdii). It is not endorsed by, sponsored by, or affiliated with MIT, MIT CSAIL, the original authors, or any model vendor.
- Licensing. Code, benchmarks and Python labs: MIT License. Text, curricula, whitepaper and visual assets: CC BY 4.0.
Built with intellectual honesty, mathematical rigour and open science.
⭐ If this made weights feel less like a black box, a star helps others find it. ⭐
Curated by Het Patel (@Hunterdii), student · LinkedIn · Sponsor · Deployed at weight-space-atlas.vercel.app