Aefinity AI

L25 Mixed Q9 · Shrinking models

RANGER mock run: low-bit quantization ideas tested on synthetic tensors

QuestionDo popular tricks for shrinking AI models to very few bits actually work, when tested where they can fail?

Run by us; no unaffiliated reproduction

01Why it matters

Low-bit compression ideas often look good in prose. We tested them where they could fail.

02Method, step by step

  1. Build synthetic tensors with documented outlier patterns, fixed seed 20260714.
  2. Run tests E0 to E5 plus a follow-up E6, pure numpy.
  3. Rebuild any test that did not compute its target.

Method as recorded

Pure numpy, fixed seed 20260714, no GPU, no real model; tests E0 to E5 (plus an E6 follow-up) on engineered tensors reproducing documented LLM outlier pathologies (massive-activation channels, attention sinks, super-weights). Run by us.

03Results, exact and sourced

  1. R1On synthetic tensors, bits saved per width doubling came out at -0.60 against a predicted -0.50 (the file labels this a pass; it came only after the author rebuilt a first test that was not computing the target).

    **−0.60 bits per 2× width** (predicted −0.50)

    Verbatim source ranger / RESULTS.md

  2. R2Falsified sub-hypothesis: on synthetic tensors, rotation helps at 4-bit (1.23x) but is slightly worse than plain per-token quantization at 3- and 2-bit (0.92x).

    wins at W4A4 (1.23×) but is slightly *worse* than plain per-token quant at W3A3 and W2A2 (0.92×)

    Verbatim source ranger / RESULTS.md

  3. R3In the first combined test, rotation smeared the super-weights, so protecting the top rotated weights in sparse form failed. The fix was to split them off before rotating.

    rotation delocalizes the super-weights

    Verbatim source ranger / RESULTS.md

04What it does NOT show

Real-model quality, quantization-aware training, or outlier patterns beyond the hand-set ones.

05Scope and caveats

  • Synthetic tensors, not a real model; tests mechanisms only, not end-to-end LLM quality, per the file's own limitations section.
  • Outlier model (gain, count, sink strength) is hand-set; real distributions vary by model family.
  • No quantization-aware training loop was run.
  • The author states the first width test was wrong (it did not compute the target) and rebuilt it; the combined-stack test also failed at v1 and was redesigned.
  • A follow-up test (E6, same file) found rotation and width are largely redundant for outliers: rotation adds only +0.1 dB on top of width. This corrects an earlier theory claim, and its design also needed two redesigns.
  • The file's 'below ~3 bits you need QAT' remark rests on an outside paper (ParetoQ); it was not tested here.
  • Run by us; no unaffiliated reproduction.

06Outcome

Mixed

Some parts held, some did not, or we corrected ourselves.

07What we learned

On synthetic data, one sub-hypothesis was falsified and two design constraints emerged: rotate only after splitting off the sparse outliers, and rotation helps 4-bit activations but not below 4 bits. A later test showed width and rotation overlap on outliers. These are mechanism-level findings, not claims about real models.

08Check it yourself

  • ranger / RESULTS.mdhttps://github.com/Aefinity-AI/ranger/blob/main/research/experiments/RESULTS.md

Read the results file (link above). It is pure numpy with a fixed seed.