How Generative AI Works: Text, Image, and Code Generation Explained

Despite covering very different outputs, text generators, image generators, and coding assistants share a common foundation: a neural network trained on large amounts of data learns statistical patterns, then uses those patterns to build new content piece by piece rather than retrieving a stored answer.

Text generation: predicting the next token

Large language models such as those behind chatbots first break text into small units called tokens, which can be whole words or parts of words, according to IBM's technical explainer on transformer models, published March 28, 2025. Each token is converted into a numerical vector, called an embedding, that represents its meaning, and a second vector is added to encode the token's position in the sentence.

The core mechanism is self-attention: for every token, the model calculates how relevant every other token in the input is, using three learned vectors per token — a query, a key, and a value, per IBM. This lets the model weigh context from anywhere in the text at once, rather than reading word by word as older architectures did. Decoder-only transformers, the architecture behind models such as GPT, then generate text autoregressively — predicting one token at a time and feeding that token back in to predict the next one, IBM explains.

Research from Anthropic, published on its site as 'Tracing the thoughts of a large language model,' found that this next-token process can involve more planning than it appears from the outside. In one experiment, when its Claude model was writing a line of poetry, it activated internal features corresponding to a possible rhyming word before writing the rest of the line, then adjusted its wording to reach that word. In another test, the model answered a multi-step question — such as identifying the capital of the state containing Dallas — by first activating an internal representation for 'Texas' and then separately retrieving 'Austin,' rather than jumping straight to the answer, Anthropic reported.

Anthropic also found a gap between the model's internal computation and its own description of that computation: when asked how it performed a math calculation, Claude described a standard method, but the internal process Anthropic traced used a different, parallel approach, combining a rough estimate with separate tracking of the final digit.

Image generation: removing noise step by step

Most modern text-to-image tools, including Google's Imagen, use diffusion models, which work in the opposite direction from text generation. According to Google's Imagen research page, the system starts with a frozen large language model — T5-XXL — that converts the text prompt into an embedding, the same kind of numerical representation used in text models. Google's researchers note that scaling up the size of this text encoder improved image quality more than scaling up the image-generating part of the model.

A diffusion model then takes that text embedding and generates a small 64x64 pixel image. It does this by starting from random visual noise and running an iterative 'denoising' process, where the model repeatedly predicts and removes a bit of noise, guided by the text embedding, until a coherent low-resolution image remains, per Google's description. The image is then enlarged using two further diffusion models, first to 256x256 pixels and then to 1024x1024 pixels, a cascade Google calls super-resolution diffusion.

To allow the model to follow the text prompt more closely, Google's research page says Imagen introduces 'a new thresholding diffusion sampler,' used alongside a method known as classifier-free guidance, which together enable the use of much larger guidance weights. Google reported that in human evaluations, raters judged Imagen's output as comparable to real reference photos for matching the text prompt, and preferred it over other text-to-image systems, including DALL-E 2, on a benchmark Google built called DrawBench.

Code generation: suggestions built from context

AI coding assistants such as GitHub Copilot apply the same transformer-based, next-token prediction approach used for general text, but trained specifically on programming code and adapted to a developer's workspace. According to GitHub's own documentation, Copilot operates in two main modes: an 'assistive' mode that suggests code, answers questions, or explains code as a developer types, and an 'agentic' mode where a developer describes a goal and Copilot works through multiple steps — researching the repository, proposing a plan, editing files, and running tools to complete a task.

GitHub's documentation states that Copilot's capabilities also extend to customizations, such as repository-specific instructions and external tool integrations, and that some features work asynchronously, allowing a coding agent to make changes and open or update pull requests without constant supervision. GitHub does not publicly specify every model used or the full technical details of how context is selected and passed to the underlying model, according to its current documentation.

How the three compare

All three approaches depend on training data: the model's internal patterns come from the text, images, or code it was exposed to during training, and its output is a newly generated sequence or image shaped by those learned patterns rather than a copy of any single training example.