Aware Transformer News Today

Many new Transformer architecture improvements have been proposed since my last post on “The Transformer Family” about three years ago. Here I did a big refactoring and enrichment of that 2020 post — restructure the hierarchy of sections and improve many sections with more recent papers. Version 2.0 is a superset of the old version, about twice the length. Notations Symbol Meaning $d$ The model size / hidden state dimension / positional encoding size.

The Critical Resemblance House is the main attraction of the Site of Reversible Destiny park in Yoro, Gifu Prefecture. The roof is shaped like a map of Gifu and there is a maze inside, with upside-down furniture hanging off the walls. If you look up at the ceiling, you see an identical world in reverse. (Photo by Lisa Vogt) Weird. Crazy. I wonder how much money was put into creating this mega... mega... park in Yoro, Gifu Prefecture. Park? I don’t know what to call it. I suppose it can be cal...