$GitHub - carloslfu/slotstream: Run Qwen3.8-Flash-Next (125B MoE, 104 GB at 4-bit) on Macs with a fraction of that RAM by streaming experts from SSD. MLX + Swift, Ollama-compatible API.$

Show HN: Slotstream, run Qwen3.8-Flash-Next 4-bit on a low-memory Mac

Run Qwen3.8-Flash-Next (125B MoE, 104 GB at 4-bit) on Macs with a fraction of that RAM by streaming experts from SSD. MLX + Swift, Ollama-compatible API. - carloslfu/slotstream

2 points by carloslfu 52 days ago github.com 0 comments

Summary not yet generated for this article.

Original Text

I built slotstream, a way to run Qwen3.8-Flash-Next 4-bit on a low-memory mac starting from 16GB, a 125B parameter model that would need 100GB+ memory/RAM, thanks to expert-offloading/ssd-streaming. Easy to install/update, and mac-native using MLX and Swift.

It ships with auto-mode, which makes a good tradeoff between memory usage and speed.

I'll be implementing and porting the MTP module for speculative decoding next

Local models really are the future of computing!

Comments (0 of ) ⌛

Loading more comments... ⌛

Scroll down to load more comments

No comments available.