Aefinity AI

L30 Negative Q8 · Speed numbers

The '16% slower' lookup-table kernel claim, retracted

QuestionDoes a performance number copied from a source comment hold up when actually measured?

Run by us; no unaffiliated reproduction

01Why it matters

Numbers that were never measured spread easily.

02Method, step by step

  1. Put all arms in one binary and interleave them in one loop on identical packed weights.
  2. Make three logged runs.
  3. Compare the f32 arm and the pshufb arm.

Method as recorded

All arms in one binary, interleaved in one loop, on identical packed weights; three logged runs on a Chromebook crosvm guest (i5-10210U), git_head a249b2c. Run by us.

03Results, exact and sourced

  1. R1The earlier '16% slower' claim does not reproduce: the pshufb arm was bimodal (8.2 vs ~11.2 GMAC/s effective, a 36% swing) while the f32 arm moved only 4.7%. In the faster mode pshufb was 1.19-1.21x faster than the f32 matvec kernel; in run 1 it lost at 1.09x, so the ordering flips between runs.

    "16% SLOWER" is NOT REPRODUCIBLE.

    Verbatim source alice-aegis / lut_mpgemm_ab_findings_2026-07-30.md

  2. R2The decision to reject the kernel still stands, on different grounds. The memory-traffic ratio is derived arithmetic (1.670x per token), not a measurement. The derivation uses prior-log figures to reach a projected ~28% slower decode in the bandwidth-bound regime.

    The rejection verdict STANDS, on memory-traffic grounds, not kernel throughput.

    Verbatim source alice-aegis / lut_mpgemm_ab_findings_2026-07-30.md

04What it does NOT show

End-to-end decode speed (the 28% figure is derived) or any VNNI machine.

05Scope and caveats

  • Three logged runs on a virtualised machine (crosvm guest, i5-10210U); the pshufb arm was bimodal. A fourth spot-run was unlogged and the source says it substantiates nothing.
  • The bimodality explanation (working set straddling the 6 MB LLC) is an untested hypothesis in the source.
  • The original number came from a source-file comment, which our own provenance rule does not admit; the bench had never measured the f32 arm it claimed to lose to.
  • The benchmarked pshufb kernel is numerically incorrect (madd pairing artifact, flagged in-source); repairing it would erode the faster-mode gain.
  • The 28% slower figure is a derivation from earlier logs, not a measured end-to-end result.
  • No VNNI (vpdpbusd) machine was measured; that revisit is still open.
  • Run by us; no unaffiliated reproduction.

06Outcome

Negative

The claim or approach failed, or we retracted it.

07What we learned

A number cited from a source file but never measured was wrong in the form claimed. The conclusion survived, for a different reason.

08Check it yourself

Read the findings (link above).