Beyond Self-Attention: How a Small Language Model Predicts the Next Token
A deep dive into the internals of a small transformer model to learn how it turns self-attention calculations into accurate predictions for the next token.
Source: shyam.blog