Fused decode kernels, checked in your browser
This page runs a JavaScript port of the fp32 reference numerics of every thox-edge-ops v2 operator against known-answer vectors produced by the PyTorch reference, and charts the measured operator latencies from the paper. It does not run the native kernels; they need a GPU or a CPU extension. Your AI. Your Data. Your Rules.™
1. Numerics: known-answer tests
| Operator | max abs error | result |
|---|
Tolerance 2e-6 for fp32 paths; the fused residual stream must be bit-identical (tolerance 0). Inputs: 3 rows of 16 for norms, 3 rows of 2 x 8 for gated activations, head size 8 and a 32-position cos/sin cache for rotary paths, Gemma offset 1 for the fused prologue.
2. Try the operators
Norm weight is all ones (Gemma: all zeros, so the scale is 1). Gated activations split the vector into gate and up halves. Rotary uses position 5 of the demo cache with head size equal to the vector length (even lengths up to 8).
3. Measured speedups over eager PyTorch
| Operator | Shape | eager us | kernel us | speedup |
|---|
4. What is measured and what is not claimed
Measured: operator-level median latency over 30 interleaved trials of 50 calls, on one NVIDIA RTX 4060 Ti under WSL2 and one x86 cloud CPU, with the shapes of Qwen3-0.6B, Gemma 3 1B and Gemma 3 4B. Speedups are ratios of medians; both paths include Python dispatch.
Not claimed: model-level throughput, tokens per joule, Jetson Orin or Apple silicon results for v2, and any configuration not in the chart. Where the kernel is slower than eager the bar shows it.