Scaling Transformer to Output Over 2 Million Words With RMT
Recurrent Memory Transformer retains information across up to 2 million tokens (words). Applying Transformers to long texts does not necessarily require large
Source: nextbigfuture.com