L16 Mixed Q6 · Do our tests test anything
E26: sin/cos at very long contexts — the sampled answer was wrong
QuestionDoes the pinned sin/cos stay accurate at the huge angles a million-token context produces, and can spot-checking tell you?
Run by us; no unaffiliated reproduction
01Why it matters
Spot-checking a function is cheap and can be reassuring and wrong.
02Method, step by step
- Run a sampled sweep of 4,096 arguments per binade.
- Replace it with an exhaustive sweep of every f32 in [0, 2^20) against a float64 oracle.
- Check symmetry and the rotary angle sets for 131,072 positions on both models.
Method as recorded
First a sampled sweep of 4,096 arguments per binade (reported max 1 ULP for sin and cos over [2^0, 2^16)); then an exhaustive sweep of every f32 in [0, 2^20) against a float64 host-libm oracle, plus symmetry checks and the RoPE angle set for 131,072 positions on both models (SmolLM2-135M and Qwen2.5-0.5B), on one VM (ChromeOS Crostini, x86_64) under the pinned FTZ+DAZ floating-point environment. Run by us.
03Results, exact and sourced
-
R1The first, sampled answer was wrong.
The first answer was wrong, and the way it was wrong is the point
Verbatim source cis2-spec / E26_TRIG_RANGE.md -
R2The sample never landed on the hard cases.
4096 samples out of 8,388,608 per binade simply never landed near a zero of
Verbatim source cis2-spec / E26_TRIG_RANGE.md -
R3Exhaustive sweep below 2^16, measured in ULP of the result: sin 66, cos 2283 (large near zeros of sin and cos). The sampled sweep had reported at most 1 ULP (one 2-ULP cos) over [2^0, 2^16). Over the full [0, 2^20) sweep the ULP maxima are sin 2283, cos 2617.
TRIG-X EXHAUSTIVE |x|<2^16 max|ulp| sin=66 (at 4.4767695e3) cos=2283 (at 5.2516434e4)
Verbatim source cis2-spec / E26_TRIG_RANGE.md -
R4The full sweep covered 1,233,125,376 arguments.
Every f32 in `[0, 2^20)` — **1,233,125,376 arguments**, subnormals included:
Verbatim source cis2-spec / E26_TRIG_RANGE.md -
R5Measured as absolute error, the worst case over [0, 2^20) is 9.4218e-8, which is 0.790 ULP-of-unity.
over the entire range is 0.790 ULP-of-unity
Verbatim source cis2-spec / E26_TRIG_RANGE.md
04What it does NOT show
A proof, behaviour at |x| of 2^31 or more, or that the ULP figure measures a defect: it is inflated near zeros of sin and cos.
05Scope and caveats
- The large ULP figures are ULP of the result near zeros of sin/cos, where relative error is inflated; the absolute-error figure is the one about the reduction (at the worst ULP point the absolute error is 1.86e-11).
- The accuracy oracle is the host libm (glibc f64 sin/cos rounded to f32); the spec pins bits, not accuracy, and a more accurate reduction would be non-conforming. Double rounding can shift a reported ULP count by one.
- Source: an exhaustive enumeration on one host, not a proof; nothing is claimed for |x| >= 2^31. Worst absolute error below 2^31 is 2.0925e-7 (1.756 ULP-of-unity).
- The earlier sampled program is kept in the repo marked SUPERSEDED, with its numbers not to be quoted.
- No timing figures (VM).
- Run by us on one machine; no unaffiliated reproduction.
06Outcome
Some parts held, some did not, or we corrected ourselves.
07What we learned
4,096 samples per binade never hit the worst points among about 8.4 million, so sampling gave a comfortable answer that was wrong. An exhaustive sweep replaced it. Absolute error stays flat while ULP error grows near zeros of sin/cos, so ULP alone is the wrong figure of merit for these functions.
08Check it yourself
- cis2-spec / E26_TRIG_RANGE.mdhttps://github.com/Aefinity-AI/cis2-spec/blob/main/docs/E26_TRIG_RANGE.md
Read the page (link above); the earlier sampled program is marked SUPERSEDED.