AI Glossary · Last reviewed August 2026

Transformer

Hand-written by a real person. Reviewed against current practice in August 2026.

Definition

The neural network architecture behind almost every modern LLM.

Why it matters

Transformers are the architecture behind virtually every modern AI model - GPT, Claude, Gemini, Llama, and more. The "T" in GPT stands for Transformer. This architecture revolutionized AI by enabling models to process entire sequences of text in parallel and understand relationships between distant words.

You do not need to understand the math, but knowing that transformers are the foundation helps you follow AI news and understand why certain breakthroughs matter.

How it works

4 steps

STEP 01

Input is tokenized and embedded

The input text is split into tokens and each token is converted into a vector with positional information added.

STEP 02

Self-attention runs

Each token attends to every other token in the sequence, calculating how relevant each word is to every other word. This is the core innovation.

STEP 03

Feed-forward layers process

The attention output passes through neural network layers that transform and refine the representation.

STEP 04

Layers repeat

Steps 2-3 repeat across many layers (GPT-4 has ~120 layers), with each layer building more abstract understanding.

Frequently asked questions

What is a transformer model?+

A transformer is a deep learning architecture that uses attention mechanisms to understand relationships in data.

Why are transformers important in AI?+

Transformers power many modern AI systems because they efficiently process large amounts of language and data.

Which AI models use transformers?+

Many large language models and generative AI systems are built using transformer architecture.

New to Transformer?

See the tools that use it.

The fastest way to understand Transformer is to see it inside real products. Browse hand-reviewed tools that put it to work, each one checked by a person before it was listed.

Browse hand-reviewed AI tools