In-Context Learning
GPT-3 learns your task from examples in the prompt — zero retraining. Here's how that's even possible.
Grounded in the research on in-context learning
2020: GPT-3 shocks everyone
Tom Brown and OpenAI drop "Language Models are Few-Shot Learners" in May 2020. The headline: GPT-3 could do a brand-new task just by reading a few examples inside the prompt. No fine-tuning, no gradient updates, no touching the 175 billion weights. You paste three translation pairs, it translates the fourth. They named the trick in-context learning, and it didn't exist in smaller models.
What it actually is
Normal machine learning means changing the model's weights with training data. In-context learning skips all of that. The model's weights stay frozen. The "learning" happens live, at inference, purely from the tokens you stuffed into the context window. The task lives in the prompt, not the parameters. Close the chat and it's gone — nothing was saved. It's pattern-matching on steroids, performed on the fly.