Variants of the Attention Mechanism

From-scratch PyTorch implementations of attention variants — multi-head, grouped-query, flash, and linear attention. Each variant is self-contained with benchmarks and visualizations to compare compute, memory, and output quality. A reference repo for understanding how attention evolved from the original transformer to modern efficient variants.