fast.ai - Can LLMs learn from a single example?
We’ve noticed an unusual training pattern in fine-tuning LLMs. At first we thought it’s a bug, but now we think it shows LLMs can learn effectively from a single example.
Sebastian Ruder Radek Osmulski Leslie Smith Cyclical Learning Rates For Training Neural Networks Alignment Lab Ai Discord Large Language
Source: fast.ai