Compress any LLM 3x in one click,near-lossless,in under 50 MB RAM
26 points by anushka_gandhi
SPECTRA achieves a 3.31x compression of large language models, enabling near-lossless performance while using under 50 MB of RAM. The method employs calibration-free Group-32 microscaling and demonstrates a decoding speed of 82.53 tokens per second on 4GB consumer GPUs.