Llama-modchip: 284B DeepSeek on 64GB RAM, no GPU
5 points by joshcsimmons
The Llama-modchip enables the execution of the 284 billion parameter DeepSeek model and 30 billion MoEs on a system with only 64GB of RAM and no GPU, achieving processing speeds of up to 44.4 tokens per second for DeepSeek and 111 tokens per second for the MoEs. This setup demonstrates the capability of running large language models efficiently on CPU-only configurations.