We're happy to provide AI research and engineering teams access to our PyTorch framework for applying quantization algorithms that are tightly aligned with NVIDIA's compiler for efficient inference. The framework also enables convenient research of new algorithms!
We hold several patents and have published articles, including a CVPR oral presentation on DNNs compression. We've built tools we enjoy using ourselves and hope they can benefit others too!
The framework supports various quantization setups:
We hold several patents and have published articles, including a CVPR oral presentation on DNNs compression. We've built tools we enjoy using ourselves and hope they can benefit others too!
The framework supports various quantization setups:
- Integer and Float quantization
- Symmetric and Asymmetric quantization
- Dynamic and Static quantization
- Multiple granularity options: per-tensor, per-channel, per-token, etc
- Pre-defined configuration schemas compatible with NVIDIA GPUs that are easy to set up and use