-
Online Softmax in Attention Mechanism
How online softmax enables exact, memory-efficient blockwise attention for long sequences.
-
Backpropagation Through Matrix Multiplication
A derivation of backpropagation through matrix multiplication, from scalar intuition to batch-wise gradient formulas.