Torch-TensorRT Tutorials¶

The user guide covers the basic concepts and usage of Torch-TensorRT. We also provide a number of tutorials to explore specific usecases and advanced concepts

Dynamo / `torch.compile`¶

Torch-TensorRT provides a backend for the new torch.compile API released in PyTorch 2.0. In the following examples we describe a number of ways you can leverage this backend to accelerate inference.

Compiling ResNet using the Torch-TensorRT torch.compile Backend: Compiling a ResNet model using the Torch Compile Frontend for torch_tensorrt.compile
Compiling a Transformer using torch.compile and TensorRT: Compiling a Transformer model using torch.compile
Torch Compile Advanced Usage: Advanced usage including making a custom backend to use directly with the torch.compile API
Torch Compile Stable Diffusion: Compiling a Stable Diffusion model using torch.compile
Torch Export with Cudagraphs: Using the Cudagraphs integration with ir=”dynamo”
Using Custom Kernels within TensorRT Engines with Torch-TensorRT: Creating a plugin to use a custom kernel inside TensorRT engines
Refit TenorRT Graph Module with Torch-TensorRT: Refitting a compiled TensorRT Graph Module with updated weights
Deploy Quantized Models using Torch-TensorRT: Compiling a VGG16 model with FP8 and PTQ using torch.compile