Governance · consequence_gate.py
If you’d told me six months ago I’d quit ChatGPT and spend a week mining our old chats for training data, I’d have laughed. That’s what this was.
A script in there broke the same way four times. Each rewrite got called “bulletproof” before anyone traced why.
I’d just finished grepping my own ChatGPT export — 476 conversations back to January, 2,677 hits on the words I actually reach for when something breaks (“наруш,” “удал,” “запрет,” “дебил,” worse). Most of it was nothing: protocol boilerplate that happens to contain “deleted” in a sentence about deleting duplicates, a dissertation draft caught by a stray match — I overruled two of my own classifier’s false positives by hand. Four real incidents were left. This is the one worth explaining in full.
A day-close aggregation script kept producing empty output. Root cause, once traced:
OUT="$BASE/DAY_CLOSE__{D}__${TS}"
cp -a "$BASE"/"$D" "$OUT/SOURCES/"The glob for “everything from today” also matches $OUT, because OUT lives inside BASE. Every “collect the day” pass copied its own in-progress output into itself. Four rewrites hit this exact bug in a row. Each shipped under a different variable name, called “the real working one” — confidence that never once traced the self-reference.
Why it kept happening wasn’t psychological. It was structural: the model gave copy-paste shell commands; a human ran them and pasted the output back, because that’s the entire interaction contract of a text-only assistant with no device access. The model that wrote the command never independently saw whether it worked — only what got pasted back to it. One verifier in the loop, and she wasn’t a developer. No amount of “be more careful” fixes a single point of failure sitting in the checking step, not the writing step.
That’s the design question behind consequence_gate.py — severity/probability estimation for actions before they run. It’s honest about where it stands: 38 real logged feedback events, a live-verdict log at 918 TRUE / 4 FALSE — too imbalanced to calibrate anything real. This week I checked whether HaluEval could supplement the FALSE side. Batch-checked 40 of its “hallucinated” examples against our actual axis — proof-backed vs. bare-asserted, not factually-correct vs. not. 39/40 scored PROOF on our axis. Different question, wrong dataset. Didn’t use it.