An LLM (large language model) is a neural network trained on vast amounts of text to predict the next piece of text, called a token, given everything that came before it. Repeating that prediction one token at a time is how models like Claude, GPT and Gemini write answers, code and summaries.
Anthropic's glossary puts it this way: "Large language models (LLMs) are AI language models with many parameters that are capable of performing a variety of surprisingly useful tasks." This guide explains how they are trained, how they produce text, what limits them, and how coding agents build on them.
What is a large language model, exactly?
At its core, an LLM is a very large function. It takes a sequence of tokens as input and returns a probability for every token in its vocabulary that could come next. "Large" refers to the number of parameters: the learned numbers inside the network, which run into the billions for current models.
The model does not store documents and look them up. During training it adjusts its parameters so its predictions match the text it sees. Patterns of grammar, facts, code style and reasoning steps end up encoded in those parameters, which is why a model can explain a regex or write a SQL query it has never seen in exactly that form.
Three terms are easy to mix up:
- The model is the trained network, such as a specific Claude or GPT release.
- The API is how you send it tokens and get tokens back.
- The app (a chat interface, an IDE plugin, a coding agent) adds prompts, history and tools around the model.
How are LLMs trained?
Training happens in stages.
Pretraining. The model reads a huge, mostly unlabeled body of text and learns to predict the next token. Anthropic's glossary describes pretraining as "the initial process of training language models on a large unlabeled corpus of text," and notes that the resulting models "are not inherently good at answering questions or following instructions." A pretrained model will happily continue your text, but it will not reliably act like an assistant.
Fine-tuning. The pretrained model is trained further on curated examples so it follows instructions and answers in a useful format. The glossary says fine-tuning "causes the model to start representing and mimicking the patterns and characteristics of the fine-tuning dataset."
Reinforcement learning from human feedback (RLHF). People rank several candidate answers, and training pushes the model toward the higher-ranked ones. Per the same glossary, RLHF trains a model "to behave in ways that are consistent with human preferences." Labs also use other feedback and reinforcement methods, but the goal is the same: turn a text predictor into a helpful, honest assistant.
Training is expensive and happens before release. Once a model ships, its parameters are fixed. It does not learn from your prompts while you use it.
How does an LLM generate text?
Generation is a loop:
- Your input is split into tokens, small chunks of text such as words or word pieces.
- The model reads all the tokens and outputs a probability for each possible next token.
- One token is picked from that distribution.
- The picked token is appended, and the model runs again on the longer sequence.
- This repeats until the model emits a stop token or hits a length limit.
Anthropic's glossary describes Claude's underlying model as autoregressive, "pretrained to predict the next word, given the previous context of text in the document." Each output token becomes part of the input for the next step, which is why long answers take longer and cost more.
How a token is picked depends on sampling settings such as temperature. Low temperature favors the most likely token; higher temperature allows less likely choices. Our LLM temperature guide covers which models still let you change it.
Token counts matter for cost and limits. For Claude, a token on models before Claude 4.7 is about 3.5 English characters, and the glossary says Claude 4.7 and later use a tokenizer that produces about 30 percent more tokens for the same text. Read what tokens are for how to count them.
What is a context window?
The context window is the most text a model can work with in one request. Anthropic's context window docs define it as "all the text a language model can reference when generating a response, including the response itself," and call it a "working memory" that is separate from the training data.
Everything you send counts: the system prompt, every earlier message, tool definitions, file contents and the model's own output. As of October 2026, current Claude models have a 1M-token context window, per the same page.
Bigger is not automatically better. The docs warn that "As token count grows, accuracy and recall degrade, a phenomenon known as context rot." Choosing what goes into the window matters as much as its size. Our context window guide compares sizes across vendors and explains what happens when it fills.
Why do LLMs hallucinate?
An LLM is optimized to produce plausible next tokens, not to check facts against a database. When the right answer is missing from its training data or its context, it can still produce fluent, confident text that is wrong. Anthropic's guide to reducing hallucinations describes the problem as text that is "factually incorrect or inconsistent with the given context."
Common triggers in developer work:
- Asking about an API version released after the model's training cutoff.
- Asking for a function or flag that sounds right but does not exist.
- Long contexts where the relevant detail is buried.
- Prompts that pressure the model to answer instead of admitting uncertainty.
Anthropic's basic fixes are to let the model say "I don't know," ground answers in direct quotes from supplied documents, and verify claims with citations. The same page cautions that these techniques "don't eliminate them entirely." See AI hallucinations for more detail, and give the model real docs or code to read instead of relying on memory.
How do coding agents use LLMs?
A plain LLM can only return text. A coding agent turns that text into action by giving the model tools and running it in a loop. Claude Code's How Claude Code works page describes the loop as three phases: "gather context, take action, and verify results."
The page splits the work cleanly: "models that reason and tools that act." Claude Code is "the layer around the model that provides the tools and manages the context the model sees," which it calls the agentic harness. When you ask it to fix failing tests, the model might decide to run the suite, read the error, search for the source file, edit it and rerun the tests, with each tool result fed back as new tokens in the context.
That design explains several agent behaviors:
- Context fills up. Every file read and command output lands in the context window, so long sessions need compaction or a fresh start.
- Instructions files matter. Files like CLAUDE.md or AGENTS.md are loaded into context so the model sees your conventions every session.
- Verification beats trust. Because the model can hallucinate, agents that run tests and read real output catch more of their own mistakes.
Our guide to AI coding agents compares how the main agents work.
What are the limits of LLMs?
Plan around these when you build with or rely on an LLM:
| Limit | What it means in practice |
|---|---|
| Training cutoff | The model does not know about libraries or events after its data ends unless you provide them. |
| Context window | It only sees what fits in one request; anything else is invisible. |
| Hallucination | Confident output can be wrong, so check facts and run code. |
| Non-determinism | The glossary notes that even at temperature 0, identical inputs might produce different outputs. |
| Cost and latency | Every input and output token is billed and takes time to process. |
None of these are reasons to avoid LLMs. They are reasons to give the model good context, ask for evidence, and keep a human or a test suite in the loop.
How do you get better results from an LLM?
Most quality gains come from the prompt and the context, not the model choice. Be specific about the task, include the relevant code or documents, state the output format, and tell the model what to do when it is unsure. Prompts that work are worth keeping. Kipt stores them as Markdown in a private GitHub repo you own, with fill-in blanks for the parts that change, so you can reuse the version that worked instead of rewriting it.
What should you read next?
Start with what tokens are to understand cost and limits, then context windows to see how much a model can hold. When you are ready to put a model to work on code, read what an AI coding agent is and what agentic AI means.