Smaller, faster, safer: running Kimi and GLM at scale
Serving frontier models like Kimi and GLM means fighting for GPU memory. Here's how we quantize KV caches, compress model weights, and add integrity checks to serve them faster, cheaper, and safely.
Summary
Learn how to optimize GPU memory usage for advanced models like Kimi and GLM through techniques such as quantizing KV caches and compressing model weights, resulting in faster and more cost-effective deployment.
Comments (66 of ) ⌛