Huawei releases MACaQ, its internal quantizer for local inference
14 points by thefox96
Huawei has introduced MACaQ, an internal quantizer designed for model-agnostic quantization that enhances local inference capabilities. It supports various data types, allowing users to quantize and execute models for text, image, video, and audio using frameworks like llama.cpp, vLLM, Diffusers, and PyTorch.