Skip to content

THOX.ai Research · Technical report THOX-TR-2026-11-03 · v0.2.0 · 2026-11-04

Preprint, not peer reviewed

CUDA Rust on ThoxOS

An integration review of NVIDIA’s two Rust GPU kernel tracks, now with the first THOX device qualification on an RTX 4060 Ti, a same-day comparison against the production CUDA C++ kernel, and the ThoxOS GPU layer that acts on both.

Tommy Xaypanya

Chief Technology Officer & Co-Founder (corresponding author)

ORCID 0009-0000-4425-9565

THOX.ai LLC · Cedar Park, Texas · November 4, 2026

DOI 10.5281/zenodo.23955185

Abstract

NVIDIA’s CUDA Rust offers two ways to write GPU kernels in Rust: cuda-oxide for SIMT and cutile-rs for Tile. We review both, map them onto ThoxOS™ hosts, and test the Tile track twice: on a host with no GPU, and on a THOX RTX 4060 Ti workstation (sm_89). Without a GPU, cutile 0.4.0 builds on 3 of 3 stable Rust toolchains given CUDA 13 headers including cuRAND, rejects 4 of 4 aliasing misuse cases at compile time, and exits with a typed driver error instead of panicking when no driver is present. On the workstation, our FP32 Tile RMSNorm kernel matches the CPU reference on all 6 model shapes we test. Against the production CUDA C++ kernel on the same host and day, the result splits by shape: on the full launch path the C++ side is 2.96x faster at 1 x 2048 and the Tile side 1.52x faster at 128 x 4096. Both figures are dominated by allocation, launch and copy-back, so we make no kernel-versus-kernel speed claim. A CUDA 13.0 toolkit builds the crate and passes host tests but cannot launch a Tile kernel, because the Tile IR assembler ships only with CUDA 13.2 and later. We describe the ThoxOS GPU layer that turns these results into a fallback order and test it on 8 virtual hosts. The Tile track is adopted for new NVIDIA kernel authoring behind that fallback; the production default stays on CUDA C++, Jetson is not yet run, and the SIMT track is reviewed from source only.

6/6

model shapes where the Tile RMSNorm kernel matches the FP32 CPU reference on sm_89 (tolerance 1.0e-5)

Measured on device

Split

launch path, same host and day: C++ 2.96x faster at 1 x 2048, Tile 1.52x faster at 128 x 4096; no kernel-vs-kernel claim

Calculated from measured medians

4/4

aliasing misuse cases rejected at compile time by cutile 0.4.0

Measured, GPU-less host

8/8

virtual hosts on which the ThoxOS GPU layer binds the expected kernel

Simulated

Download the paper (PDF)Read the paper (HTML) Browser demo

Contents

  1. Executive summary
  2. 1. Introduction
  3. 2. Background and related work
  4. 3. System and threat model
  5. 4. Design
  6. 5. Implementation
  7. 6. Evaluation
  8. 7. Discussion and limitations
  9. 8. Conclusion
  10. 9. Roadmap
  11. Appendix A: reproducibility
  12. Appendix B: notation
  13. Appendix C: device qualification protocol
  14. Appendix D: document control
  15. References

Where the paper lives

Cite this paper

@techreport{xaypanya2026cudarust020,
  author      = {Tommy Xaypanya},
  title       = {{CUDA} {Rust} on {ThoxOS}: An Integration Review of {NVIDIA}'s Two {Rust} {GPU} Kernel Tracks, with the First {THOX} Device Qualification on an {RTX} 4060 {Ti}, a Same-Day Comparison Against the Production {CUDA} {C++} Kernel, and the {ThoxOS} {GPU} Layer},
  institution = {THOX.ai Research, THOX.ai LLC},
  address     = {Cedar Park, Texas},
  year        = {2026},
  month       = nov,
  version     = {0.2.0},
  doi         = {10.5281/zenodo.23955185},
  url         = {https://doi.org/10.5281/zenodo.23955185},
  note        = {Preprint, not peer reviewed. Technical report THOX-TR-2026-11-03. Zenodo concept DOI (all versions): 10.5281/zenodo.23955184. PDF SHA-256: 9aba898eea4591792e753692bb9f7776d829e8f2d8ec25dfdd2db21f01363b8f.}
}

Preprint, not peer reviewed; DOI 10.5281/zenodo.23955185.

Your AI. Your Data. Your Rules.™