1. Attention Through Arithmetic Intensity (changyi.fun)
The article breaks down the arithmetic intensity of various models, revealing that MHA consistently equals 1 while comparing GQA, MQA, and MLA. It also highlights a key difference between MLA and MTP, making it a valuable read for those interested in model efficiency.