Torch Tweak supports multiple tuning backends, each with different characteristics and use cases. The backends align with a common interface for the build and inference process.
The TorchAO backend leverages PyTorch's AO (Accelerated Optimization) framework for model tuning.
from torch_tweak.torch.backend import TorchAOBackend
backend = TorchAOBackend()Guide: TorchAO Backend.
The Torch Inductor backend uses PyTorch's Inductor compiler for model tuning.
from torch_tweak.torch.backend import TorchInductorBackend
backend = TorchInductorBackend()Guide: Torch Inductor Backend.
The XPU graph backend records the module once while tuning and replays that recording for later inference. PyTorch chooses native or legacy recording. This backend uses whichever one that build records. A recording that fails, or that disagrees with eager, is rejected for that module and input.
from torch_tweak.torch.backend import XPUGraphBackend
backend = XPUGraphBackend()Guide: XPU Graph Backend.
The OpenVINO backend converts the module to an OpenVINO model and runs it on an Intel CPU or GPU.
from torch_tweak.torch.backend import OpenVINOBackend
backend = OpenVINOBackend()Conversion defaults to torch.export. For modules that torch.export cannot capture, switch to the
legacy trace-based conversion and optionally change the compile hint:
from torch_tweak.torch.backend import OpenVINOBackend, OpenVINOBackendConfig
backend = OpenVINOBackend(OpenVINOBackendConfig(use_dynamo=False, performance_hint="THROUGHPUT"))Guide: OpenVINO Backend.
The Torch Eager backend runs the module with no compilation and no conversion. It is the baseline HighestThroughputStrategy compares the other backends against.
from torch_tweak.torch.backend import TorchEagerBackend, TorchEagerBackendConfig
import torch
backend = TorchEagerBackend(TorchEagerBackendConfig(autocast_enabled=True, autocast_dtype=torch.float16))Guide: Torch Eager Backend.