Separating Language Information News Today : Breaking News, Live Updates & Top Stories | Vimarsana

Discussion: Discussion Thread for comments, corrections, or any feedback.



Summary: The latest batch of language models can be much smaller yet achieve GPT-3 like performance by being able to query a database or search the web for information. A key indication is that building larger and larger models is not the only way to improve performance.



The last few years saw the rise of Large Language Models (LLMs) – machine learning models that rapidly improve how machines process and generate language. Some of the highlights since 2017 include:


The original Transformer breaks previous performance records for machine translation.
BERT popularizes the pre-training then finetuning process, as well as Transformer-based contextualized word embeddings. It then rapidly starts to power Google Search and Bing Search.
GPT-2 demonstrates the machine’s ability to write as well as humans do.
First T5, then T0 push the boundaries of transfer le ....

Discussion Thread , Large Language Models , Google Search , Improving Language Models , Separating Language Information , World Knowledge , Transformer Encoder , Illustrated Transformer , Chunked Cross Attention ,