How AI chatbots like ChatGPT and Claude actually work, explained

Hundreds of millions of people type questions into ChatGPT, Claude, Gemini or similar tools every day, but the software behind them is often described as a mysterious "black box." According to Google's Machine Learning Crash Course and an explainer by science writer Timothy B. Lee published on Understanding AI, the core mechanism is simpler to describe than it sounds: a large language model (LLM) is a neural network trained to predict the next small chunk of text, called a token, given everything that came before it.

From words to numbers: tokens and embeddings

Before an LLM can process text, it breaks a prompt into tokens — units that can be a whole word, part of a word, or a single character, according to Google's developer documentation. Each token is converted into a vector, a long list of numbers called an embedding, that is meant to capture something about the token's meaning. Understanding AI's explainer compares this to geographic coordinates: words with similar meanings end up positioned near each other in a vast numerical space, the way nearby cities share similar latitude and longitude.

Attention: how the model tracks context

The architecture behind virtually all modern LLMs is called the Transformer. Google's crash course explains that a Transformer relies on a mechanism called self-attention, which calculates how much every token in a piece of text should influence the interpretation of every other token. Understanding AI illustrates this with query and key vectors: each word effectively builds a checklist of the kind of other words it is looking for, and attention matches those checklists across the sentence so the model can, for example, work out which noun a pronoun like "it" refers to.

This step is followed by feed-forward layers that process each word's updated representation and push the model toward predicting what token should come next, Understanding AI notes. Because attention lets every token look directly at every other token regardless of distance, Transformers handle long, complex passages more consistently than the sequential models that came before them.

Training: billions of guesses, no human labels required

LLMs are trained mostly through unsupervised learning on masked or next-token prediction, meaning the system is not given hand-written "correct answers" for most of its training. Instead, Google's documentation describes a process in which tokens in real text are hidden and the model has to guess them; Understanding AI describes the same idea in terms of next-word prediction across huge collections of text scraped from books, websites and other sources. Parameters inside the network — Understanding AI cites GPT-3's 175 billion of them, trained on roughly 500 billion words — start out randomly set and are gradually adjusted, prediction by prediction, until the model's guesses line up with real text.

Turning a raw model into a helpful assistant

A model trained only to predict the next token is not automatically a chatbot that follows instructions politely. Hugging Face's technical explainer on Reinforcement Learning from Human Feedback (RLHF) describes the extra steps companies such as OpenAI use to convert a base model into an assistant. First, the pretrained model can be refined through supervised fine-tuning on example conversations. Second, human reviewers rank multiple model responses to the same prompt, and those rankings train a separate reward model that learns to score outputs the way a person would.

In the third stage, the original model is adjusted using reinforcement learning — commonly Proximal Policy Optimization — so that it produces responses the reward model scores highly, while a mathematical penalty keeps it from drifting so far from its original behavior that it starts generating nonsense purely to game the reward score, per Hugging Face's description of the method.

Why chatbots still make things up

A paper released by OpenAI researchers, "Why Language Models Hallucinate," available on arXiv, argues that fabricated answers are not simply a bug to be patched out. The paper attributes hallucinations to two linked causes: a statistical byproduct of how models learn to predict plausible-sounding continuations during pretraining, and the way models are evaluated afterward. Most benchmarks and leaderboards score an answer as either right or wrong and treat an admission of uncertainty — an "I don't know" — as a wrong answer, the paper states. That setup rewards confident guessing over honesty about uncertainty, which pushes models to keep producing fluent, specific-sounding statements even when they lack a reliable basis for them.

What the model is actually doing, in one sentence

Put together, the process described across these sources is this: a chatbot turns a prompt into tokens, converts them into numerical embeddings, uses self-attention layers to weigh how every token relates to every other, and repeatedly predicts the single most probable next token — a process shaped, after pretraining, by human feedback designed to make those predictions resemble helpful, instruction-following answers rather than raw internet text.