Improving Language Models by Retrieving from Trillions of Tokens

We explore an alternate path for improving language models: we augment transformers with retrieval over a database of text passages including web pages, books, news and code.

Related Keywords

,Improving Language Models ,Retrieval Enhanced Transformer ,Retrieval Enhanced Transformers ,