CIS-2: the same AI output, bit for bit, on different machines

01Why it matters

If an AI's output can differ depending on which computer ran it, nobody can audit a result by re-running it. A written specification that makes the bits identical would turn 'trust us' into 'run it yourself'.

02Method, step by step

1. Write the arithmetic down: the order in which numbers are added, no fused multiply-add, how tiny numbers are flushed, the sin/cos polynomials and the position table.
2. Pick one pinned test: the SmolLM2-135M model, the prompt 'Once upon a time', 16 greedy tokens, fp32.
3. Hash the full output scores (the logits) into a SHA-256 chain. That digest is the receipt.
4. Run the self-check on different machines and compare the digests with the pinned ones in EXPECTED_DIGESTS.md.

03Results, exact and sourced

1. R1A free Lightning AI cloud Studio (Xeon Platinum 8488C, CPU not owned by us) matched all four pinned digests using the narrower scripts/self_check.sh, not the 9-check selfcheck.sh. Driven by the maintainer.

   **Result: PASS — all four digests match the pinned test vector.**

2. R2The README's documented expected selfcheck.sh output (shown with an i5-5200U fingerprint, the i5-5200U AVX2 row) ends in 9 passed, 0 failed. This is shown as expected output; the machine table separately lists the i5-5200U host with its log not published.

   == SUMMARY: 9 passed, 0 failed ==

3. R3The aarch64 Android phone reproduced the pinned digests through a different code path; the self-check script itself was not run on it.

   `selfcheck.sh` not run on the device; the pinned §13.1 digests were reproduced on it through the KV-cache `verify3/` path

05Scope and caveats

• Run by us on every machine, including the cloud workspace; the README says: No unaffiliated party has reproduced it yet.
• One model (SmolLM2-135M), one prompt, 16 tokens, fp32 greedy decoding only (no sampling, batching or quantization), mostly CPU; production bf16/GPU serving is outside the spec today.
• Full logs for most machine-table rows are published as 'log not published'.
• The Lightning run used scripts/self_check.sh (four digests), not the 9-check selfcheck.sh; the 9-passed line is the README's expected-output block (i5-5200U fingerprint), not a separately published log.
• The Lightning Studio is a free tier workspace, not a rented one; no timing numbers reported.
• One machine-table host (one of our own machines) had weights copied over the LAN because its resolver could not reach the Hugging Face CDN; the pinned sha256 of all three artifacts was still verified.
• Shows the bits reproduce; says nothing about whether the model is correct, safe or fair.
• Conformance-suite internals are not public; only the vectors and spec are.

06Outcome

07What we learned

With the arithmetic pinned in writing, one fp32 digest reproduced across AVX2 and scalar x86, an aarch64 phone and an AVX-512 cloud CPU, in our own runs. Until an outsider runs it, the claim stays 'reproduced by us'.

08Check it yourself

• cis2-spec / README.mdhttps://github.com/Aefinity-AI/cis2-spec/blob/main/README.md
• cis2-spec / EXPECTED_DIGESTS.mdhttps://github.com/Aefinity-AI/cis2-spec/blob/main/EXPECTED_DIGESTS.md
• cis2-spec / CIS2_SPEC_v0.3b.mdhttps://github.com/Aefinity-AI/cis2-spec/blob/main/docs/CIS2_SPEC_v0.3b.md
• cis2-spec / 2026-09-25-LIGHTNING-STUDIO-SELFCHECK.mdhttps://github.com/Aefinity-AI/cis2-spec/blob/main/docs/results/2026-09-25-LIGHTNING-STUDIO-SELFCHECK.md
• Hugging Face / cis2-conformancehttps://huggingface.co/datasets/aefinityAIINC/cis2-conformance

Reproduce the headline check with the command the repository README gives:

$ git clone https://github.com/Aefinity-AI/cis2-spec && cd cis2-spec && ./selfcheck.sh

The README also lists a narrower local script, scripts/self_check.sh, run after the same git clone. The README says it needs a Rust toolchain (cargo) and network access to download the pinned SmolLM2-135M weights.