AI Glossary · Last reviewed August 2026

Inference

Hand-written by a real person. Reviewed against current practice in August 2026.

Definition

Running a trained model to get a prediction or response.

Why it matters

Inference is what happens every time you send a prompt and get a response. It is the production phase of AI - the part that costs money and determines speed. Faster inference means snappier tools; cheaper inference means lower subscription prices.

When comparing AI tools, inference speed and cost directly affect user experience and pricing. Tools that optimize inference can offer better performance at lower cost.

How it works

4 steps

STEP 01

Input is received

The model receives your prompt as a sequence of tokens.

STEP 02

Forward pass through layers

The tokens pass through the model's neural network layers, with each layer refining the prediction of what comes next.

STEP 03

Tokens generated one by one

The model predicts and outputs one token at a time, feeding each new token back as input for the next prediction.

STEP 04

Response is complete

Generation stops when the model produces a stop token or reaches the maximum length.

Frequently asked questions

What is inference in AI?+

Inference is the process where a trained AI model uses learned patterns to make predictions, generate content, or provide answers.

How does AI inference work?+

During inference, input data is processed by a trained model to produce an output based on its learned parameters.

What affects AI inference speed?+

Factors such as model size, hardware, optimization, and network conditions can affect inference speed.

New to Inference?

See the tools that use it.

The fastest way to understand Inference is to see it inside real products. Browse hand-reviewed tools that put it to work, each one checked by a person before it was listed.

Browse hand-reviewed AI tools