The Illustrated GPT-2 (Visualizing Transformer Language Models)
Discussions: Hacker News (64 points, 3 comments), Reddit r/MachineLearning (219 points, 18 comments) Translations: Simplified Chinese, French, Korean, Russian, Turkish This year, we saw a dazzling application of machine learning. The OpenAI GPT-2 exhibited impressive ability of writing coherent and passionate essays that exceed what we anticipated current language models are able to produce. The GPT-2 wasn’t a particularly novel architecture – it’s architecture is very similar to the decoder-only transformer. The GPT2 was, however, a very large, transformer-based language model tra...
Source: jalammar.github.io
:focal(1368x938:1369x939)/https://tf-cmsv2-smithsonianmag-media.s3.amazonaws.com/filer_public/76/9c/769ce792-48aa-46a9-a89e-38f470c28cee/mack-at-typewriter.jpg)