A Mechanistic Interpretability Analysis of Grokking

A significantly updated version of this work is now on Arxiv …

Related Keywords

Janos Kramar ,Martin Wattenberg ,Vikrant Varma ,Zac Kenton ,Chris Olah ,Jeff Wu ,Noa Nabeshima ,Jacob Steinhardt ,Evan Hubinger ,Jacob Hilton ,Arthur Conmy ,Tom Lieberum ,Michela Paganini ,John Wentworth ,Rohin Shah ,Kevin Wang ,Yalex Ray ,Nicholas Turner ,Nick Cammarata ,Tao Lin ,David Lindner ,Neel Nanda ,David Bau ,Lauro Langosco ,Eric Michaud ,Johannes Treutlein ,Xander Davies , ,Discrete Fourier Transforms ,Discrete Fourier Transform ,Induction Heads ,Large Language Models ,Repeated Subsequences ,Phase Changes ,Phase Changes Are Inherent ,Mathematical Framework ,Transformer Circuits ,Intuitive Explanation ,Zero Capabilities ,Alphazero Interpretability ,Discrete Fourier ,Fourier Components ,Fourier Basis ,Circuits During ,Slingshot Mechanism ,Vlad Mikulik ,Sid Black ,