RANGER Phase 1: the quantization ideas on real models (SmolLM2-135M, Qwen3-0.6B)

01Why it matters

Synthetic results do not count until they survive on real models.

02Method, step by step

1. Measure perplexity on a WikiText-2 slice for SmolLM2-135M, replicate on Qwen3-0.6B.
2. Fix predictions and kill gates before each run; claim effects only above 3x run noise.
3. Re-test small effects on disjoint slices.

03Results, exact and sourced

1. R1Our earlier E7 null result was retracted as an artifact of how it was built; nothing about super-weight protection can be concluded from E7.

   E7's null result was an artifact by construction — retract it.

2. R2On SmolLM2-135M (fp32), 4-bit per-token activation quantization collapses perplexity from 16.14 to 13,773; a folded rotation sandwich recovers 92.3% of the log-PPL collapse, leaving PPL at 27.11 (still above the 16.14 baseline). On Qwen3-0.6B the recovery was 92.5% (34.23 to 146,845 collapse). Fake-quant is an upper bound on deployable A4.

   A4 per-token quantization collapses PPL 16.14 → 13,773; a per-linear folded rotation sandwich recovers **92.3%** of the log-PPL collapse (→ 27.11)

3. R3Refuted: the 'skip the rotation' upgrade. Static exemption of 8 census channels recovered only 59.1% at 135M (25% with activation-measured channels at 0.6B; the census-derived set did nothing on Qwen) versus 92%+ for rotation.

   Pillar 1's "skip the rotation" upgrade is refuted

4. R4Null at 135M: no super-weight lever over an honest g=128 4-bit baseline, and MSE clipping hurts there (24.896 to 26.917). The 'rescues 0.38 of clip damage' sub-result was refuted as noise on a disjoint slice (-0.015). At 0.6B, clipping helps (48.06 to 41.82) and the coordinate increment is small but real (+0.45, then +0.24).

   At 135M there is no exploitable super-weight lever over an honest W4 baseline — and MSE-clipping actively hurts.

5. R5Across models, the overlap between the weight census and activation-outlier channels fell from 6/10 (SmolLM2-135M) to 2/10 (Qwen3-0.6B); the census still found the single top channel on both.

   Census→activation overlap collapses: 6/10 → 2/10.

05Scope and caveats

• Two small models, CPU-only, one WikiText-2 slice (plus one disjoint confirmation slice each); Qwen absolute perplexity is chat-tuned (34.23 bf16) and not comparable to SmolLM2; slice-to-slice magnitude varies a lot, direction is the replicated quantity.
• Per-token dynamic fake-quant upper-bounds deployable 4-bit activations; rotation used a QR orthogonal, not a fast Hadamard; Qwen rotation arm ran in bf16 with storeback rounding noted.
• Super-weight effects are documented in the literature from about 1B parameters up; scale-gated pending larger runs.
• Rotation sandwich W4A4 end-to-end point is 53.868 PPL vs 16.14 baseline: recovery is of the collapse, not full parity.
• The source document describes itself with a superlative ordinal; we do not repeat that wording.
• Run by us; no unaffiliated reproduction.

06Outcome

07What we learned

A prior null result was retracted publicly. Rotation recovery replicated across two architectures (92.3% and 92.5%), while the census-replaces-rotation shortcut was refuted, and a super-weight lever is null at 135M and only small at 0.6B. One of our own sub-results (0.38 rescue) was refuted by our confirmation pass.

08Check it yourself

• ranger / RESULTS-phase1.mdhttps://github.com/Aefinity-AI/ranger/blob/main/research/phase1/RESULTS-phase1.md

Read the results file (link above).