Unified Memory, Explained: Why Mini PCs Can Run 70B Models a Big GPU Can't

One idea explains the whole mini PC category: unified memory lets a $2,000 box hold a 70B model no consumer GPU can fit, then decode it slowly. The roofline math, the prompt-processing catch, the NPU red herring, and the owner-measured speeds.
50 points by ermantrout 103 days ago vettedconsumer.com 39 comments
Summary not yet generated for this article.

Comments (39 of ) ⌛

Loading more comments... ⌛
Scroll down to load more comments
No comments available.
Type to search HN and engineering blog articles
Press Escape to close  |  ↑↓ to navigate  |  Enter to open