GitHub - liventruth/UL-SMF-Cache-Compression: The Unified Latent-State Memory Fabric (UL-SMF) is a hardware-software co-designed memory compression fabric that solves the memory bottleneck in long-context Transformer inference. By combining FSQ with dynamic 16-dimensional latent mapping, UL-SMF compresses Key-Value (KV) cache tensors by up to 384x while maintaining >94% semantic retention.

Show HN: UL-SMF – Open-source linear-complexity ~300x KV-cache compression

The Unified Latent-State Memory Fabric (UL-SMF) is a hardware-software co-designed memory compression fabric that solves the memory bottleneck in long-context Transformer inference. By combining FS...

13 points by liventruth 66 days ago github.com 1 comments

Summary not yet generated for this article.

Comments (1 of ) ⌛

Loading more comments... ⌛

Scroll down to load more comments

No comments available.