GitHub - SharpAI/SwiftLM: ⚡ Native MLX Swift LLM inference server for Apple Silicon. OpenAI-compatible API, SSD streaming for 100B+ MoE models, TurboQuant KV cache compression, + iOS iPhone app.

TurboQuant KV Compression and SSD Expert Streaming for M5 Pro and IOS

⚡ Native MLX Swift LLM inference server for Apple Silicon. OpenAI-compatible API, SSD streaming for 100B+ MoE models, TurboQuant KV cache compression, + iOS iPhone app. - SharpAI/SwiftLM

72 points by aegis_camera 203 days ago github.com 40 comments

Summary not yet generated for this article.

Comments (40 of ) ⌛

Loading more comments... ⌛

Scroll down to load more comments

No comments available.