One flipped bit in the weights: which checks notice?

01Why it matters

Weights are files that can be corrupted or tampered with. Re-reading the answer is the cheap check; checking the computation is the expensive one. We wanted to know how much the cheap one misses.

02Method, step by step

1. Take one pruned BitNet-2B checkpoint and one prompt, 64 greedy tokens.
2. Flip single bits in the weights: Experiment 1 used 50 trials, Experiment 2 used 295 trials spread across tensor classes.
3. Run three kinds of check on each corrupted run and count the misses, with 95% Wilson intervals.
4. Recount everything from the raw per-trial logs on a second host.

03Results, exact and sourced

1. R1Experiment 2 (295 trials): of 247 trials where the output changed, the text-replay column (A) missed 182. In the source's Experiment 2 definitions, A is 'inference ran to completion without erroring'.

   | text/argmax replay (A) | 182 / 247 | [0.679, 0.788] |

2. R2Experiment 2: the top-k / receipt-style check (B) missed 82 of the same 247. In Experiment 2 B means the receipt verifier accepted the run, not top-5 replay.

   | top-k / receipt-style check (B) | 82 / 247 | [0.276, 0.393] |

3. R3The witness digest row is 0 of 247. The document says this is exact by construction, since a digest miss and a C-fail are the same event.

   | CIS witness digest (C) | 0 / 247 | [0.000, 0.015] |

4. R4Experiment 1 (50 trials): text replay passed 3 corruptions that the digest caught (trials 16, 25, 47). Top-5 replay also failed on all 3, so its miss count was 0 at this sample size.

   | A pass AND C fail (text replay misses a corruption the digest catches) | 3 |

5. R5Null result: 43 flips (37 of 70 SCALE_FFN and 6 of 35 TRIT_FFN trials) left the witness digest identical to the clean baseline. The digest by design stays silent on these, and a byte-level check would flag them.

   A byte-level integrity check would flag these 43 cases; the CIS digest, by design, does not, because it attests to computation, not storage.

6. R6The write-up corrected its own original report after the independent recount: trial 47 was a bit-7 flip from the random set, not one of the 5 dedicated high-bit control trials.

   Correction from the independent recount: the original report used trial 47 as "the" high-bit example;

05Scope and caveats

• The digest's 0/247 is exact by construction (C is defined as a digest or baseline-output mismatch); the informative numbers are how often A and B miss.
• Experiment 2's definitions differ from the entry's headline labels. A there is 'inference completed without error' and C is 'output matches baseline', so the 182/247 figure is not a pure text-replay measurement. The source's own tables relabel the columns, and readers should be told.
• Experiment 1's random flips were low-mantissa bits 0 to 4 only (plus 5 forced bit-7 flips). Flips were chosen by the harness, not an adversary.
• B means different things in Experiment 1 (top-5 logit set) and Experiment 2 (receipt accepted). The two are not merged.
• In Experiment 1 every method caught all 5 forced high-bit flips. In Experiment 2 the EMBED class (80 trials) was missed by both A and B 80 of 80 times.
• Embedding result: EMBED C-fail was 80/80 for the fixed prompt only. The unused-embedding-row case was not tested.
• No adversarial (optimised) tampering was tested. No timing claims are made.
• One pruned 2B checkpoint, one prompt, one hardware family; second host recomputed counts from logs only. The checkpoint is not linked publicly, so outsiders may not be able to rerun as-is.
• Run by us; no unaffiliated reproduction.

06Outcome

07What we learned

In the larger run, among the 247 trials where the output actually changed, the text-replay column missed 182 and the top-k/receipt-style check missed 82. Some flips (43) changed nothing in the computation. A check on the computation itself is by definition silent only on flips that changed nothing, but that 0/247 is true by construction, so the informative finding is how often the weaker checks miss. Single prompt and model, harness-chosen flips, no adversary.

08Check it yourself

• cis2-spec / 2026-09-13-BITFLIP-HEADTOHEAD.mdhttps://github.com/Aefinity-AI/cis2-spec/blob/main/docs/results/2026-09-13-BITFLIP-HEADTOHEAD.md
• alice-aegis / commit 14ef1a1https://github.com/Aefinity-AI/alice-aegis/commit/14ef1a1
• alice-aegis / commit 1567ee1https://github.com/Aefinity-AI/alice-aegis/commit/1567ee1

Read the result page and the two public harness commits (links above).