How does an AI answer get made?

How does an assistant turn a notice into an invitation? Start with what it has to work from. This fictional notice supplies the event details:

The neighborhood book swap is November 14, from 10 a.m. to noon at the community hall. Bring up to 5 books. Entry is free. Children must attend with an adult. The notice does not say whether refreshments will be available.

Then you ask: "Write a short summary for a neighbor."

Your request is a prompt. The notice is part of the information the assistant can use for this answer, called its context. Depending on the system, context can also include conversation messages, product instructions, and material supplied by tools.

A large language model, or LLM, is trained to work with language. Training develops patterns in its internal numerical settings from examples. Further training can shape how it follows instructions and responds.

When the model answers your question, it uses those learned patterns together with the information available now. Supplying the event notice gives it material to work from. It does not immediately retrain the underlying model or make the notice known to every other user.

The context has a size limit, called the context window. In a very long conversation, a product may drop or summarize earlier messages to make room, so a detail you gave at the start can fall out of view. Training also stops at some date. A model on its own may not know about recent events unless the product searches for them. Anthropic's guide to context windows shows the limit from the builder's side.

Whether the service stores your conversation or later uses it to improve models depends on its policies and settings. The privacy and actions guide explains those distinctions.

A language model processes text as tokens. A token can be a word, part of a word, or another unit of text. Picture the model producing the next piece, then the next, using the material available and what it has generated so far.

3Blue1Brown's short LLM explanation gives a visual introduction to this process. It simplifies some details to build intuition. A whole assistant product can also use search, calculations, and other operations around the model.

The process can produce useful explanations and drafts, along with details that sound plausible and have no support. People often call an invented detail a hallucination. The model is producing likely text, not looking the detail up, so a wrong detail can sound exactly as sure as a right one. Anthropic's note on incorrect answers gives the same advice: check what matters against a source. The sentences arrive without a record of where every fact came from. Read the answer against the source to see which details have support.

An assistant may use a search tool to find information, then use that information in its response. Finding material is often called retrieval. Generating an answer and retrieving a source are different steps.

For the book swap, the notice supports the time, place, and book limit. It does not tell us whether refreshments are available. More confident wording cannot fill that gap. Another source might help, or the organizer might need to answer.

The ChatGPT capabilities overview shows how one assistant combines language models with tools. Do not assume every product has those tools or uses them for every question.

Imagine the assistant searches for a newer event notice before writing its answer. Put these steps in order: generate a reply, receive your question, retrieve the notice, make the retrieved text available to the model.

Your question comes first. The search retrieves a notice, the system includes its text in the context, and the model generates a reply. A product can repeat these steps, and some replies use no search at all.

Now suppose the search finds last year's notice. Which step needs attention?

The retrieval found the wrong source. Better wording cannot make last year's event details current. Check the notice's date before checking whether the summary matches it. The answer-checking guide carries that check through a complete example.

The same problem appears with public services. In another authored example, someone asks when a permit application is due, and the assistant gives last year's date with complete confidence. The office's current page is the authority. Use the assistant to find the right page and the right question, then read the date there yourself.