GateGPT: 56k tokens per second Transformer (KV cache) on FPGA at 80 MHz

56,000+ tokens/sec at just 80 MHz. 🤯

I burned a full Transformer with KV cache into a custom chip. Designed gate by gate as a 100% digital integrated circuit. Prototyped on a FPGA. (No GPU. No CPU)
Just pure digital silicon running @karpathy microGPT, spelling out names on a https://t.co/hXX8kKIxiA