NVIDIA Advances Native Rust Programming for GPUs
NVIDIA has announced a major initiative to enhance native GPU programming capabilities using Rust. While CUDA C++ and CUDA Python remain established, enterprise-grade toolchains, the company plans to significantly develop and mature CUDA Rust through 2027 and beyond.
The systems infrastructure surrounding artificial intelligence (AI)—including serving frameworks, drivers, and inference engines—is increasingly built using Rust. This language is valued for its ability to enforce memory safety at compile time without sacrificing performance. NVIDIA itself utilizes Rust in several core components, such as the Nova Linux driver and the NVIDIA Dynamo framework.
Historically, while Rust could initiate kernel execution, the kernel logic itself often required writing in another language. CUDA Rust aims to eliminate this gap by enabling developers to write the entire GPU kernel natively in Rust and compile it directly to PTX.
The framework supports two distinct programming models, mirroring the structure of CUDA itself: SIMT and Tile. SIMT is the traditional model, where the developer specifies the action of a single thread, and the system launches thousands of instances of that thread. Tile represents a newer model, also available in C++ and Python, which allows the developer to define the operation for a single tile of data, leaving the compiler to manage the complex mapping onto the specific hardware architecture. NVIDIA recommends starting with the Tile model, as the compiler handles architecture-specific mapping, simplifying the source code.
The SIMT Programming Track: cuda-oxide
The SIMT approach is implemented via the cuda-oxide project. This tool operates as a custom rustc codegen backend. It intercepts the compilation process, routing functions marked with #[kernel] through Rust’s MIR, the community Pliron IR framework, and LLVM IR before converting the output to PTX.
This track enforces memory safety by using concepts like DisjointSlice and launch contracts to prevent common errors such as memory aliasing. To use cuda-oxide, developers require Linux, a GPU with compute capability 8.0 or later, a CUDA toolkit of version 12.x or newer, and the pinned nightly toolchain.
cargo +nightly-2026-04-03 install –git https://github.com/NVlabs/cuda-oxide.git cargo-oxide
Developers can then set up a project and run a sample, such as a vector addition program:
cargo oxide new vecadd_demo
cd vecadd_demo
cargo oxide doctor
cargo oxide run
The source code demonstrates that the host and device code reside within a single file, built using one command, eliminating the need for separate kernel libraries. Key safety features include the use of DisjointSlice for the output buffer, which ensures each thread receives exclusive, non-overlapping mutable access to its single element.
Furthermore, the use of #[launch_contract] declares the intended kernel indexing (e.g., 1D indexing with 256-thread blocks). The prepare_vecadd function then validates the execution configuration against this contract and the live device limits, providing a safe token that the subsequent vecadd method requires.
The Tile Programming Track: cutile-rs
The cutile-rs project operates at a higher level of abstraction, allowing computations to be performed on “tiles” rather than individual scalars. In this model, each tile block executes the kernel body once as a single logical thread over a sub-tensor of data, and the compiler manages the necessary underlying GPU thread mapping.
The requirements for this track are less stringent than the SIMT model. cutile-rs requires Linux, a GPU with compute capability 8.0 or later, CUDA 13.3, and stable Rust 1.89 or newer, but it does not necessitate a nightly toolchain or custom LLVM setup.
Since cutile is published on crates.io, getting started is straightforward:
cargo new vecadd_demo
cd vecadd_demo
cargo add cutile
The example code for the vector addition shows how the kernel is defined using #[cutile::module]. The kernel runs once per mutable sub-tensor, treating the data as a tile. The key safety mechanism is the use of partitioning: api::zeros::(&[1024]).partition([128]). This function accomplishes three tasks: it grants exclusive ownership of the 128-element chunk to the tile, it defines the launch geometry (1024 divided by 128 equals 8 tiles), and it supplies the tile width (B).
A crucial aspect of cutile-rs is that all operations are lazy until the explicit call to .sync_on(&stream)?. Everything preceding this point merely builds a description of the computation, rather than submitting it to the GPU.
Advanced Memory Safety and Interoperability
Both projects achieve the same high level of memory safety, ensuring that inputs are shared and the output belongs exclusively to a single writer. However, they enforce this guarantee at different points. The SIMT model relies on checking launch calls, while the Tile model enforces ownership tracking across the entire launch boundary, which represents a stronger compile-time guarantee.
The ability of both frameworks to catch classic aliasing mistakes at compile time is a major technical achievement. In the SIMT example, attempting to pass the output buffer (c_dev) as an immutable input results in a compile error. Similarly, on the Tile side, the compiler prevents the use of a tensor that has been moved into the kernel call, enforcing ownership.
While the SIMT model retains manual control over shared memory and thread indexing—which is necessary for performance but requires unsafe code—the Tile model makes this control explicit and safe by construction, as each tile block is treated as a single logical thread.
Both cuda-oxide and cutile-rs are currently considered early-stage tools, with cuda-oxide being in early alpha. In contrast, cutile-rs is more advanced, already published on crates.io and utilized by external organizations, including HuggingFace’s Grout inference engine and mistral.rs. NVIDIA plans to ensure seamless inter-language interoperability among CUDA Rust, CUDA C++, and CUDA Python, ensuring developers are not locked into a single frontend.
Action Items for Developers
Developers can begin by running the SIMT example using cargo oxide new followed by cargo oxide run. For the Tile example, the process involves using the appropriate cutile-rs setup and running the sample code.
For detailed instructions and API references, developers are encouraged to consult the dedicated cuda-oxide book and the cuTile Rust documentation. Further engagement can occur by reporting issues or joining the community discussions on the respective GitHub repositories.
