torch.compile and Diffusers: A Hands-On Guide to Peak Performance
Diffusers is the go-to library that provides a unified interface to cutting-edge and open diffusion models for image, video, and audio. Over the past few months, we have deepened its integration with torch.compile. By tailoring the compilation workflow to the diffusion model architecture, torch.compile delivers significant speed-ups with minimal impact on user experience. In this post, we will show how to unlock these gains. The target audience for this post is
torch.compile delivers its best gains when you know the model well enough to compile the right sub-modules. In this section, we’ll first outline the factors that shape the torch.compile user experience, then dissect diffusion architectures to pinpoint which components benefit most from compilation.
torch.compile turns a Python program into an optimized graph and then emits machine code, but the speedup and ease of use depends on the following factors:
