Show HN: Taalas-style on-chip LLM weights on a $250 AMD FPGA (60k tok/s)
A 3.16M-parameter INT4 transformer running entirely in the on-chip memory of a Xilinx Kria KV260. Zero DRAM in the token loop, 59,965 tok/s on the fabric, bit-exact. Chat with it live.
Summary not yet generated for this article.
Comments (30 of ) ⌛