What Is an LLM? – Definition and How It Works

Learn what a large language model is, how training and inference work, what a token is, and where an LLM's knowledge stops without external data.

A large language model is a neural network trained to predict the next token in a sequence. That single objective, applied across a very large corpus, produces a system that can summarise, translate, write code, and follow instructions — none of which were programmed as separate features.

1. What Is a Large Language Model?

An LLM is a statistical model of text. It assigns a probability to every possible next token given the tokens before it, and generates output by sampling from that distribution one token at a time.

• Key idea: language understanding emerges from next-token prediction at scale, not from hand-written grammar rules.
• Mechanism: a transformer network with attention layers that let every token weigh every other token in the context.
• Goal: produce text that is plausible given the input, which in practice means answering, explaining, or continuing.

Example of Tokenisation

A token is roughly a word fragment. "Scraping" may split into scrap and ing; punctuation and spaces count too.

"web scraping api"  →  ["web", " scr", "aping", " api"]   # 4 tokens

Token counts matter because both pricing and context limits are measured in tokens, not words or characters.

2. How Training Works

Training happens in stages, and each stage changes what the model does rather than what it knows.

1. Pre-training. The model reads an enormous corpus and learns to predict the next token. This is where nearly all factual knowledge is absorbed.
2. Supervised fine-tuning. The model is shown examples of instructions paired with good responses, which teaches it to answer rather than merely continue text.
3. Preference tuning. Human or model-generated rankings of candidate answers push the model toward responses people prefer.

The result is a fixed set of weights. Nothing is written back to those weights when you use the model.

3. How Inference Works

Inference is the generation step, and it is worth understanding because it explains several familiar behaviours.

• The prompt is tokenised and fed through the network, producing a probability distribution over the next token.
• One token is selected — greedily, or by sampling with a temperature setting that controls randomness.
• The selected token is appended and the process repeats until a stop condition is hit.

Because each token is chosen from a distribution, the same prompt can produce different answers on different runs. Lower temperature narrows that variation; it does not eliminate it.

4. The Context Window

Everything the model can see at once — the system prompt, the conversation so far, any retrieved documents, and the answer being generated — must fit inside a fixed budget called the context window.

This is the practical boundary between what a model knows and what it can be told.

5. Where an LLM's Knowledge Stops

Two limits explain most surprising answers.

• The training cutoff. The weights reflect a corpus assembled at a point in time. Prices, rankings, release versions, and availability move afterwards, and the model has no way to notice.
• No inherent retrieval. A model with no tools cannot fetch a page, run a query, or check a fact. Asked about something outside its training data, it produces the most plausible-sounding continuation, which is where fabricated citations come from.

Both limits are addressed the same way: put the current information into the context window. That is what retrieval-augmented generation and tool calling exist to do.

6. Real-World Examples

• Summarising a long document works well because the source text sits in the context window and the task is compression rather than recall.
• Answering "what is the current price of X" fails without tools, because the answer is not in the weights and the model has no way to look it up.
• Writing code against a library is reliable for stable APIs and unreliable for APIs that changed after the training cutoff.
• Extracting structured fields from messy text is a strong fit: the input is supplied, and the model is doing transformation, not recall.

7. Summary

An LLM is a next-token predictor whose knowledge is frozen in its weights at training time and whose working memory is a fixed-size context window. It generates rather than looks up, which is why it is excellent at transforming text you give it and unreliable at recalling facts you do not.

Every serious production use of an LLM therefore involves getting the right text into the context window at the right moment — through retrieval, through tools, or through both.

Potential Related Glossary

• Unstructured Data
• Data Structure
• Parsing
• Data Wrangling