Get the 2026 ML Training Cookbook | 52 recipes — GRPO, Flow Matching, World Models, and everything in between Download Now →
ONNX (Open Neural Network Exchange)
An open format for representing machine learning models that enables interoperability between training and inference frameworks.
ONNX (Open Neural Network Exchange) is an open standard format for representing machine learning models, designed to enable interoperability between different training and inference frameworks. A model trained in PyTorch, TensorFlow, or JAX can be exported to the ONNX format and then loaded by any ONNX-compatible runtime, decoupling the training framework from the deployment environment. This eliminates vendor lock-in and allows teams to choose the best inference engine for their hardware.
The ONNX format defines a computation graph as a series of standardized operators (such as Conv, MatMul, Softmax) with defined inputs, outputs, and attributes. Each operator has a versioned specification, ensuring that the semantics of a model remain consistent across runtimes. The ONNX Runtime — Microsoft's cross-platform inference engine — provides hardware-accelerated execution on CPU, GPU, and specialized accelerators (NPUs, TPUs) through execution providers.
ONNX is widely used in production deployment pipelines, particularly for edge and on-device inference where lightweight runtimes and hardware-specific optimizations are critical. Models can be further optimized through ONNX-specific graph transformations, operator fusion, and quantization to reduce model size and latency. The format supports both floating-point and quantized (INT8) models.
- ■Open standard format decoupling training frameworks from inference runtimes
- ■Defines computation graphs using versioned, standardized operators
- ■ONNX Runtime provides hardware-accelerated execution across CPU, GPU, and NPUs
- ■Supports graph optimizations, operator fusion, and INT8 quantization
- ■Widely used for edge, on-device, and production inference deployment
Exporting a PyTorch-trained compliance classification model to ONNX for deployment in an air-gapped environment using ONNX Runtime on CPU.