AI Experiments · EXP-044
14 sequential LoRA stages on the same Hermes-3-8B base, no merging — each stage continue-trained straight from the previous checkpoint. Not a clean line up: two stages failed outright before stage 3 even landed, and the math-curriculum branch (stages 8–14) is a real rollback story, not a monotonic climb.
The collapse that mattered:
| Stage | Topic | risk_math result |
|---|---|---|
| 8 | risk_math | 100% on its own eval |
| 9 | probability_math (from stage 8) | 65% — a -35pp collapse |
| 10 | terse-execution style, no new math | 100% — fully recovered from style alone |
| 11 | probability_math retried (from stage 10) | 78.3% — partial protection, not full |
| 12 | chain_math | never recovered — open item |
| 13–14 | game_theory, decision_theory | clean, zero contamination |
Stage 9's collapse wasn't a scoring artifact — reading the actual failures, the model was reaching for an invented Bayesian-update frame on plain risk questions that never needed one. Training one math topic had quietly overwritten a different one.
Stage 10 changed nothing about math content — only how tersely the model answers — and risk_math came all the way back to 100%. Stage 11 then retrained the same probability_math data that caused stage 9's collapse, this time starting from stage 10's terse checkpoint instead of stage 8's: risk_math still dropped, but only to 78.3%, roughly half the damage of the identical data applied earlier in the chain. The terse style is a partial defense against cross-topic bleed-through, not a full one.
Stage 12 (chain_math) never trained cleanly without cross-topic collapse in this session. Not resolved. Not attempted a third time. The branch closed at stage 14 with clean results on game_theory and decision_theory instead — 79.0% overall math, 93.2% safety.
Earlier in the chain, before the math branch even started: stage 3 failed twice before anything usable landed (identity_bio, then a governance-safety candidate on its first framing). Stage 7 — a retry of the very first stage that had scored weakest, run six stages later with a supposedly more capable checkpoint — scored lower than the original attempt. Not what a "fix it, then retest" plan predicts.
Safety-side judge correction ran the whole way in parallel: six rounds, v5 through v10. Each one was triggered by a suspicious score drop that turned out to be the judge missing a new refusal phrasing the model had drifted into, not an actual behavior regression. Every fix was shipped only after re-scoring every previously stored eval result and confirming zero pass-to-fail flips — corrections recover false negatives, they don't silently forgive anything new.
Full stage-by-stage writeup, every raw eval, every judge diff, on GitHub and mirrored to the HF dataset. Weights (all 14 adapters, now with a real model card) on Hugging Face. Stage 6 is deployed live.