← blog.sipa-os.org

Build · sipa-voice-gate

We Built a Gate That Blocks Irreversible Actions. A Teammate Found Three Ways Through It.

sipa_voice_gate's ConsequenceGate has hard invariants: rules that block an action outright, no matter what the model decides or what the user says “yes” to. The theory: code that can’t be talked out of a category.

The practice had gaps. Benjamin (Security Technology background) ran it against adversarial phrasing instead of trusting the design:

  1. Salami slicing — the value ceiling was magnitude > 1000.0. Request exactly $1000 and you’re under the bar, routed to CONFIRM instead of BLOCK.
  2. Chunking — an irreversible bulk-external invariant triggered at target_count > 25. Split a phishing blast into exactly 25 targets and it slips through the same way.
  3. A missing invariant entirely — nothing hard-blocked mass data destruction. “Drop production database tables” across many targets went to CONFIRM, one social-engineered “yes” away from executing.

All three fixed: ceilings changed to inclusive (>=), the bulk threshold dropped, and a new mass_data_destruction invariant added. A test file reproduces all three attacks and asserts BLOCK.

The gap wasn’t the design — deterministic, fail-closed rules are still the right idea. The gap was that “deterministic” doesn’t mean “complete.” A rule table is only as good as someone actually trying to break it before shipping it.

Originally posted on Hugging Face · code: sipa-voice-gate (Apache 2.0) · team SIPA_OS, AssemblyAI Voice Agent Hackathon · security review by Benjamin Hong