"Embedding the Self-Organisation of Deep Feature Maps in the Hamburger " by Jack Humphreys, Markus Hagenbuchner et al.
The popularity of neural networks that explicitly utilise the global correlation structure of their features have become vastly more popular ever since the Transformer architecture was developed. We propose to embed unsupervised Self-Organising Maps within neural networks as a means to model the global correlation structure. By enforcing topological preservation therein, such a neural network is able to represent more complex correlation structures and to produce interpretable visualisations as a byproduct. We validate this approach by comparing with the existing state-of-the-art attention sub...
Source: uow.edu.au
