$GitHub - carloslfu/slotstream: Run Qwen3.8-Flash-Next (125B MoE, 104 GB at 4-bit) on Macs with a fraction of that RAM by streaming experts from SSD. MLX + Swift, Ollama-compatible API.$

Show HN: Running 104GB Qwen3.8-Flash-Next on 48GB Mac with at ~12 tok/s

Run Qwen3.8-Flash-Next (125B MoE, 104 GB at 4-bit) on Macs with a fraction of that RAM by streaming experts from SSD. MLX + Swift, Ollama-compatible API. - carloslfu/slotstream

239 points by carloslfu 50 days ago github.com 118 comments

Summary

A developer has created Slotstream, enabling the use of a high-memory AI model on a Mac with only 48GB of RAM by leveraging expert-offloading and SSD streaming. The tool is designed for easy installation and includes an auto-mode for optimizing memory and speed, with future enhancements on the way.

Comments (118 of ) ⌛

Loading more comments... ⌛

Scroll down to load more comments

No comments available.