Improving Language Models by Retrieving from Trillions of To

We explore an alternate path for improving language models: we augment transformers with retrieval over a database of text passages including web pages, books, news and code.

Related Keywords

, Improving Language Models , Retrieval Enhanced Transformer , Retrieval Enhanced Transformers ,