Prompting & In-Context Learning
The prompt is the only steering wheel you have at inference time — no weights change — and by choosing what you put in the context, from a bare question to a few worked examples, you can dramatically shape what the model does through a phenomenon called in-context learning.
A trained LLM is frozen. Its billions of weights don't budge when you talk to it — every conversation starts from the exact same model. So how do you get it to translate, summarize, write SQL, or answer in a specific format, all without retraining anything?
The answer is the prompt. The prompt is the text you feed in, and at inference time it is the only thing you control. You aren't teaching the model new facts; you're arranging the context so that the most likely continuation is the answer you want. Learning to do that well is the single highest-leverage skill in using LLMs — and it's the whole subject of this lesson.
The model is a piano, the prompt is what you play
A piano already contains every song it will ever play — the notes are all built in. It doesn't learn a new song when you sit down; you select which keys to strike and in what order. The LLM is the same: all its capability is baked into the frozen weights. The prompt is you at the keyboard, choosing which of those capabilities to bring out. Change the prompt and you play a different song — with the exact same instrument.
#Zero-shot: just ask
The simplest prompt is zero-shot: you state the task in plain language and give zero examples. You're relying entirely on what the model absorbed during training — and for common tasks that's often plenty.
'Zero-shot' comes from machine learning, where a 'shot' means a labeled example. Zero shots = no examples, just the instruction.
Classify the sentiment of this review as positive or negative.
Review: "The battery dies in an hour and support ignored me."
Sentiment:#Few-shot: show, don't just tell
When zero-shot is shaky — the task is unusual, the format is picky, or the model keeps drifting — you switch to few-shot: you put a handful of worked examples right in the prompt before the real question. The model reads your examples, infers the pattern, and applies it to the new input.
The striking part: the model has not been retrained. Those examples live only in the context window for this one request, and they vanish the moment the request ends. Yet within that single prompt, the model behaves as if it just learned your task on the spot. That effect has a name.
Classify the sentiment as positive or negative.
Review: "Fast shipping and works great." Sentiment: positive
Review: "Broke after two days, total waste." Sentiment: negative
Review: "Exactly what I needed, love it." Sentiment: positive
Review: "The battery dies in an hour." Sentiment:In-context learning: 'learning' without learning
When a model picks up a task from examples in the prompt — no weight updates, no training run — that's in-context learning. It feels like learning because behavior changes, but nothing about the model is permanently altered: the 'lesson' exists only inside this one context window. It's less like a student who studied overnight and more like a session musician who nails the tune after hearing you hum a few bars — then walks out having changed nothing about themselves. Close the chat and the model is exactly as it was.
You give a model three input→output examples in your prompt and it suddenly formats its answer correctly. What actually changed inside the model?
#Roles: system, user, assistant
Chat models don't see one flat block of text — the conversation is structured into messages, each tagged with a role:
- system — sets the model's overall behavior, persona, and rules for the whole conversation. It's the standing instruction, written once, that everything else obeys.
- user — that's you: your questions and requests.
- assistant — the model's own replies. Previous assistant turns stay in the context so the model can see what it already said.
Under the hood these roles are just special tokens wrapped around the text, but they carry real weight: the system message is where you say 'You are a terse SQL assistant; reply with a query and nothing else,' and it colors every answer that follows.
[
{"role": "system", "content": "You are a terse assistant. Reply in one sentence, no preamble."},
{"role": "user", "content": "Why is the sky blue?"},
{"role": "assistant", "content": "Air scatters short blue wavelengths more than red ones."},
{"role": "user", "content": "And at sunset?"}
]#Prompt patterns that reliably work
A few habits do most of the heavy lifting. None require anything fancy — they just reduce the model's guesswork:
- Be specific. 'Summarize this' is vague. 'Summarize this in 3 bullet points for a busy executive, no jargon' pins down length, audience, and style.
- Show the format you want. If you need JSON, show one example of the exact JSON. Models mirror the shape you give them far more reliably than a shape you merely describe.
- Give relevant context. Paste the actual document, error message, or data. The model can only use what's in the window — it can't consult a file you didn't include.
- Break the task into steps. For anything with reasoning, ask the model to work through it step by step instead of blurting a final answer.
"Let's think step by step"
For multi-step problems — math, logic, debugging — telling the model to reason step by step before answering measurably improves accuracy. Because each token is predicted from the ones before it, letting the model 'write out its work' gives it intermediate tokens to build on, instead of forcing the whole answer into one leap. This is called chain-of-thought prompting. (Note: many 2026 reasoning models now do this internally by default, so you may not need to ask — but for standard chat models, the phrase still earns its keep.)
More examples isn't always better
Few-shot is powerful, but examples aren't free: each one eats context-window tokens, adds cost and latency on every call, and — worst of all — a sloppy example teaches the wrong pattern. Three clean, consistent examples usually beat ten noisy ones. And if a crisp zero-shot instruction already works, adding examples can be pure overhead. Reach for few-shot to fix a specific failure (wrong format, misread task), not as a reflex.
You want the model to always reply with strict JSON like {"name": "...", "age": 0}. Which prompt change is most likely to get reliable JSON?
Put it all together and the picture is simple: you can't change the model, but you own the context. Start zero-shot; if the model stumbles, add a few clean examples (few-shot / in-context learning); set the ground rules in the system message; and be specific about task, format, and steps. Everything else in prompt engineering is variations on those moves.
Key takeaways
- The model's weights are frozen at inference time — the prompt is the only thing you control, so prompting is the act of arranging context to make the answer you want the most likely continuation.
- Zero-shot means asking with no examples; few-shot means showing a handful of worked examples in the prompt so the model infers the pattern.
- In-context learning is the model picking up a task from prompt examples with no weight updates — the effect lives only in that one context window and vanishes when the request ends.
- Chat prompts are structured by role: the system message sets behavior and persona, user messages are your input, and assistant messages are the model's prior replies.
- Reliable patterns: be specific, demonstrate the exact output format, include the relevant context, and ask the model to reason step by step for multi-step problems.
This few-shot prompt establishes a pattern, then gives a new input. Given how in-context learning works, what is the model most likely to output for the last line?
Convert the word to its plural.
Word: cat Plural: cats
Word: bus Plural: buses
Word: box Plural: boxes
Word: dog Plural:A beginner wrote these notes about prompting. Exactly one line is a real misconception. Which line is WRONG?
# Notes on prompting an LLM:
1. Few-shot means putting a few examples in the prompt.
2. In-context learning permanently updates the model's weights.
3. The system message sets the model's overall behavior and persona.
4. Zero-shot means asking with no examples at all.Complete the sentences about prompting terminology.
Asking with no examples is called prompting. Showing a handful of worked examples in the prompt is called prompting. The role that sets behavior and persona for the whole conversation is the message.
You're designing a prompt to reliably extract structured data. Put these steps in the order that gives the model the least guesswork.
State the task specifically, including the exact output format you want
Set the standing rules and persona in the system message (e.g. 'output only JSON')
Add the real input last so the model continues the established pattern
Include the relevant context — the actual text to process — in the window
Provide a few clean input→output examples that demonstrate the format
A teammate is building a feature that extracts a person's name and city from a messy sentence and returns strict JSON like {"name": "...", "city": "..."}. Their current zero-shot prompt is:
> Extract the name and city: "Yesterday Priya flew from her home in Lagos to a conference."
The model keeps replying with chatty prose like "Sure! The person is Priya and the city is Lagos." instead of clean JSON, and sometimes it grabs the conference city instead of the home city.
Part A. Explain, in terms of what the model is doing, why the zero-shot prompt produces inconsistent, chatty output.
Part B. Rewrite the prompt using few-shot examples so the model reliably returns the exact JSON shape. Show at least two examples plus the real input.
Part C. Where would you put the rule "Always output only JSON, no explanation" — in a system message or a user message — and why? Is this few-shot fix changing the model's weights?
Try it yourself — a starting point to build on:
# Write your solution here