A glossary of AI and machine learning terms

181 terms, each defined in a sentence or two without using itself, and each pointing at the section that teaches it properly. A definition is a door, not a destination.

A

A/B testonline experiment, randomised controlled experiment: Randomly assigning users to two versions and comparing an outcome metric between them. The only way to know whether a change helped real people rather than an offline dataset.
Accuracy: The fraction of predictions that were correct.
Adam: An optimiser keeping a running average of each parameter's gradient and of its squared gradient, and dividing one by the other. Parameters with small, consistent gradients get large steps and vice versa, which makes it forgiving about learning rate.
AdamW: Adam with weight decay applied directly to the weights instead of folded into the gradient. The default optimiser for transformers, because the original coupling made decay effectively vanish for exactly the parameters it should shrink.
Adapter: A small set of extra weights inserted into a frozen model and trained alone, so one base model can be specialised many ways without copying it. LoRA is the form almost everyone uses now.
Agent loop: The cycle of calling a model, executing whatever tool it asked for, appending the result to the conversation, and calling again until it stops asking. Nearly every agent is this loop plus a stopping rule.
Approximate nearest neighbour searchANN, vector index: Finding vectors close to a query without checking every one, trading a small chance of missing a true neighbour for orders of magnitude more speed.
Attentionattention mechanism: A layer in which every position pulls information from other positions, weighting each one by how relevant it is. The weights are computed from the input itself, so which positions matter changes with every sequence.
Autogradautomatic differentiation, autodiff: The machinery that records every operation as it runs and then walks that record backwards to produce gradients, so nobody has to derive derivatives by hand.
Autoregressive: Generating one element at a time and feeding each output back in as input for the next. It is why a reply's first token costs one pass over the prompt and every token after it costs another pass.

B

Backpropagationbackprop: Computing how much every parameter contributed to the loss by applying the chain rule backwards through the network, reusing intermediate results so the whole thing costs about as much as one forward pass.
Batch size: How many examples are processed before the parameters are updated. Bigger batches give less noisy gradients and better hardware use; smaller ones give more updates per unit of compute.
Bayes' theorem: The rule for updating a belief with evidence: multiply what you believed beforehand by how likely the evidence would be if it were true, then normalise. It is why a very accurate test for a very rare condition still gives mostly false positives.
Beam search: Keeping several candidate continuations alive at once and expanding the most promising, instead of committing to one token at a time. It helps where there is a single right answer, such as translation, and produces bland text on open-ended writing.
Benchmark contaminationtest set contamination, data contamination: Benchmark questions having appeared in a model's training data, so its score measures recall of the answers rather than the ability being tested.
Bi-encoderdual encoder: A model encoding query and document separately, so document vectors can be computed once and stored. This independence is what makes first-stage retrieval over millions of documents possible.
Bias-variance tradeoff: Error splits into being systematically wrong because the model is too simple, and being unstable because it follows the particular sample it saw. Reducing one usually raises the other.
BM25Okapi BM25: The standard keyword ranking function, scoring a document by how often the query's rare terms appear in it, adjusted for document length.
Byte pair encodingBPE: A way of building a vocabulary: start from single bytes and repeatedly merge the most frequent adjacent pair into a new symbol. Common words end up as one token, rare ones are spelled out in pieces, and nothing can ever be out of vocabulary.

C

Calibration: How well predicted confidence matches reality: among predictions made at 70% confidence, about 70% should turn out right.
Catastrophic forgetting: A model losing abilities it previously had when trained hard on something narrow. The reason a fine-tune that nails its task can become worse at everything around it.
Causal maskcausal masking, autoregressive mask: A constraint that stops each position attending to anything after it, applied by forcing those scores to negative infinity before the softmax. It is what lets a model generate: during training, position t only ever sees what preceded it.
Chain of thoughtCoT, step-by-step reasoning: Having a model write out intermediate steps before its answer, which gives it more computation and a place to keep partial results. It helps most on arithmetic, multi-step logic and anything with a chain of constraints.
Chain rule: The rule for differentiating composed functions: multiply the derivative of the outer by the derivative of the inner. Applied through a network's layers, it is exactly backpropagation.
Checkpoint: A saved snapshot of weights, and usually optimiser state and step count too, so a run can be resumed or an earlier model recovered.
Chunking: Cutting documents into passages small enough to embed and retrieve usefully. Chunk boundaries decide what a retriever can ever return, so bad splits cap the quality of everything downstream.
Class imbalance: One class making up a small fraction of the data, so a model minimising average error learns to ignore it. Fraud, faults and disease all look like this.
Clustering: Grouping items by similarity with no labels to learn from. Useful for exploring a dataset, and treacherous as a conclusion, because any method will return groups whether or not the data has any.
Cold start: The delay when a request arrives and no instance is warm: the container has to start, and for a model server the weights have to be loaded into GPU memory, which can take minutes.
Compute-bound: A workload limited by arithmetic throughput, with the memory system keeping up. Training and prefill are compute-bound, which is why batching helps them and faster chips do too.
Compute-optimalChinchilla-optimal: The split of a fixed training budget between model size and training tokens that reaches the lowest loss, roughly twenty tokens per parameter.
Concept drift: The relationship between inputs and the right answer changing, so a model can be wrong on data that looks entirely familiar. Fraud patterns adapting to a detector is the classic case.
Confusion matrix: A table of predicted class against actual class, so every kind of mistake is visible separately rather than blended into one score. The first thing to print when a classifier disappoints.
Containerisation: Packaging an application with its dependencies into an image that runs the same anywhere, isolated from the host but sharing its kernel.
Context management: Deciding what stays in the window as a conversation or agent run grows: what to summarise, what to drop, what to move to external storage and fetch back on demand.
Context windowcontext length: The maximum number of tokens a model can attend to in one call — system prompt, retrieved material, conversation history and the reply being generated, all counted together.
Continuous batchingin-flight batching: Letting new requests join a running batch as soon as earlier ones finish, instead of waiting for the whole batch to complete. Several times the throughput of static batching on mixed-length traffic.
Cosine similarity: The cosine of the angle between two vectors: 1 when they point the same way, 0 when perpendicular. The standard way to score how close two embeddings are, because it ignores magnitude and compares direction alone.
Cross-encoder: A model that reads query and document together in one pass and outputs a relevance score. Much more accurate than comparing separate embeddings, and far too slow to run over a whole corpus — hence its use as a reranker.
Cross-entropy losslog loss, negative log likelihood: The loss used for classification and next-token prediction: the negative logarithm of the probability the model assigned to the correct answer. Confident mistakes are punished far more heavily than uncertain ones.
Cross-validationk-fold cross-validation: Splitting the data into k parts, training k times with a different part held out each time, and averaging the scores. It uses every row for both training and evaluation, which matters when data is scarce.

D

Data leakageleakage, target leakage: Information that would not be available at prediction time finding its way into training, so offline results look excellent and production results do not.
Data parallelism: Putting a full copy of the model on every GPU, giving each a different slice of the batch, and averaging the gradients. The simplest way to scale, and the right one until the model stops fitting on one device.
Decode phasedecode step, generation phase: The token-at-a-time part of inference after prefill, where each step reads the whole KV cache to produce one more token. It is limited by memory bandwidth rather than arithmetic, and it sets how fast text appears.
Decoder-only modeldecoder-only transformer, causal language model: A transformer built from a single causally-masked stack that continues its input. Nearly every chat and completion model is one.
Distillationknowledge distillation: Training a small model to reproduce the outputs of a large one, rather than training it on the original labels. The teacher's full probability distribution carries more signal than a single correct answer.
Dot productinner product, scalar product: Multiplying two vectors elementwise and summing the result, giving one number that is large when they point the same way. Attention scores and similarity search are both built on it.
DPOdirect preference optimisation: Training directly on pairs of preferred and rejected responses with a classification-style loss, skipping the separate reward model and the reinforcement learning loop. Far simpler to run, and competitive for most preference tuning.
Driftdata drift, covariate shift: The inputs arriving in production moving away from the ones the model was trained on, so its performance quietly decays without anything breaking.
Dropout: Randomly zeroing a fraction of activations during training, so no single unit can be relied on and the network has to spread what it learns.

E

Early stopping: Halting training when validation loss stops improving, and keeping the best checkpoint rather than the last one. The cheapest regularisation there is.
Eigenvector: A direction a matrix only stretches or shrinks rather than rotating, and the factor it scales by is the eigenvalue. These directions describe what a transformation fundamentally does.
Embedding model: A model that turns a piece of text into one fixed-length vector capturing its meaning, so similarity can be measured by comparing vectors.
Encoder-only modelBERT-style model, bidirectional model: A transformer with no causal mask, so every position sees the whole input at once. Good for classification, embedding and reranking, and incapable of generating text left to right.
Entropy: The average surprise in a distribution — how many bits it takes to encode an outcome drawn from it. Zero when one outcome is certain, highest when everything is equally likely.
Epoch: One full pass over the training set.
Eval setevaluation set, eval suite: A fixed collection of inputs with known-good outcomes, run against the system to turn "this feels better" into a number that can be compared between versions.
Expectationexpected value, mean: The long-run average of a random quantity, each outcome weighted by its probability.
Exploding gradient: Gradients growing as they propagate backwards until a step throws the weights far out of range, usually showing up as a loss that becomes NaN. Clipping and a lower learning rate are the usual answers.

F

F1 score: The harmonic mean of precision and recall, collapsing both into one number. It punishes a bad score on either far more than an arithmetic mean would.
Featurepredictor, input variable: One measured input a model sees for each example: a column in a table, a pixel, a dimension of an embedding.
Feature engineering: Reshaping raw inputs into forms a model can use — ratios, aggregates, time differences, encodings of categories. On tabular problems it moves results more than the choice of algorithm.
Feed-forward networkFFN, MLP block: The per-position half of a transformer block: expand to roughly four times the model dimension, apply a non-linearity, project back. Attention moves information between positions; this is where it gets processed.
Few-shot promptingin-context learning, few-shot examples: Putting a handful of worked examples in the prompt so the model infers the pattern from them. It pins down format and edge-case handling far more reliably than describing them in prose.
Fine-tuning: Continuing to train an already-trained model on a smaller, narrower dataset so it fits a particular task, format or style.
FlashAttention: An attention implementation that never materialises the full attention matrix in GPU memory, computing it in tiles inside fast on-chip memory instead. Several times quicker, and a large part of why long contexts are affordable.
FLOPsfloating point operations: A count of arithmetic operations, used as the unit of compute for training and inference. A forward pass costs roughly two FLOPs per parameter per token, and training about three times that again.
FSDPfully sharded data parallel: PyTorch's implementation of sharded data parallelism: parameters live split across GPUs and are gathered layer by layer for each forward and backward pass, then released.
Function calling: The original name for tool calling, still used in several APIs and in most older documentation. The same mechanism: a schema of available functions in, a structured invocation out.

G

Golden datasetgold set, golden set: Examples whose correct answers have been carefully agreed by people, treated as the reference everything else is measured against.
GPUgraphics processing unit, accelerator: A processor with thousands of simple cores and very wide memory bandwidth, which makes it far faster than a CPU at the large matrix multiplies neural networks are made of.
Gradient: The vector of partial derivatives of a function with respect to every input, pointing in the direction of steepest increase. Training walks the opposite way.
Gradient accumulation: Summing gradients over several small batches before stepping the optimiser, to get the behaviour of a large batch on a GPU that cannot hold one.
Gradient boostingGBDT, XGBoost, LightGBM: Building an ensemble of small decision trees in sequence, each one fitted to the errors the ones before it left. Still the thing to beat on tabular data.
Gradient clipping: Capping the norm of the gradient before the optimiser step, so one freak batch cannot take a huge step and destroy the run. Standard practice in language model training.
Gradient descent: Repeatedly nudging every parameter a small step in the direction that most reduces the loss. Nearly all model training is some refinement of this one idea.
Greedy decoding: Always taking the highest-probability token. Repeatable, and prone to loops and flat prose on anything open-ended.
Grounding: Tying a model's answer to specific supplied text, so every claim can be traced to a passage that supports it.
Grouped-query attentionGQA: Several query heads sharing one set of key and value heads, shrinking the KV cache by the sharing factor. Most large open models use it, because the cache rather than the weights is what caps batch size.

H

Hallucinationconfabulation: A fluent, confident statement that is not true — an invented citation, a function that does not exist, a plausible number.
HBMhigh bandwidth memory: The stacked memory used on datacentre accelerators, offering terabytes per second of bandwidth. Moving data between it and the chip's small on-chip memory is what most kernel optimisation is actually about.
Held-out settest set: Data set aside and not looked at during development, used once to estimate how the system will do on inputs it has never seen.
HNSWhierarchical navigable small world: The dominant approximate search index: a layered graph where each search starts on a sparse top layer and descends, following edges towards the query. Fast and accurate, at the cost of an index that lives in memory and is expensive to build.
Hybrid search: Running keyword and vector retrieval together and merging the rankings. Each catches what the other misses: exact identifiers, names and rare words on one side, paraphrase and synonymy on the other.

I

Instruction tuningsupervised fine-tuning, SFT: Training a base model on examples of instructions paired with good responses, so it answers rather than continuing the text. It is the step that turns a text predictor into something that behaves like an assistant.
Inter-annotator agreement: How often independent human labellers give the same answer on the same item, usually reported as Cohen's or Fleiss' kappa, which correct for agreement by chance.

J

JAXjit, vmap, pmap: A numerical library built around transforming pure functions rather than objects that hold state: jit compiles one, grad differentiates it, vmap batches it, and pmap or shard_map spreads it across devices. Used heavily at some frontier labs, and the native stack for TPUs.
JSON modeconstrained decoding, grammar-constrained sampling: A serving-side feature that masks out any token which would break the required grammar, so the output is valid JSON by construction rather than by request.

K

k-means: Splitting points into k groups by repeatedly assigning each to the nearest centre and moving each centre to the mean of what it captured.
KL divergenceKullback-Leibler divergence, relative entropy: How much information is lost when one distribution is used in place of another, in bits or nats. Preference training uses it as a leash, penalising the tuned model for straying from the model it started as.
KV cachekey-value cache: The stored keys and values for tokens already processed, so generating the next token does not recompute attention over the whole prefix. It turns generation from quadratic work into linear work.

L

Labeltarget, ground truth: The correct answer attached to a training example, which the model learns to predict. Where labels come from, and how consistent they are, sets the ceiling on everything trained from them.
Latency: How long one request takes end to end. What a user experiences, as distinct from how much the system gets through overall.
Layer normalisationlayer norm, LayerNorm: Rescaling each vector so its values have zero mean and unit variance across the feature dimension, then applying a learned scale and shift. It keeps activations in a range where gradients stay usable as the stack deepens.
Learning rate: How far each optimiser step moves the parameters. The single hyperparameter most likely to be the reason a run failed.
Learning rate schedule: A plan for changing the learning rate over a run, usually a warmup followed by a slow decay to near zero. Large steps early to make progress, small steps late to settle.
LLM-as-judge: Using a model to grade outputs against a rubric, where no code check can decide correctness. Cheap and scalable, and itself something that has to be validated against human labels.
Logits: The raw scores a model produces before they become probabilities — one per vocabulary entry, unbounded, and not meaningful on their own.
LoRAlow-rank adaptation: Freezing the original weights and training a small pair of low-rank matrices whose product is added to each one. A few million trainable parameters can move behaviour as much as a full fine-tune, and the result is a file of tens of megabytes.
Loss functionobjective function, cost function: The single number saying how wrong the model's outputs were, which training exists to make smaller. Everything the model learns is downstream of what this number rewards.

M

Matrix: A rectangular grid of numbers, usually standing for a linear transformation from one space to another, or for a batch of vectors stacked together.
Matrix multiplicationmatmul: Combining two matrices so that each output entry is the dot product of a row of one and a column of the other. It is where essentially all of a model's arithmetic goes.
Maximum likelihood estimationMLE: Choosing the parameters under which the observed data would have been most probable. Minimising cross-entropy loss is exactly this, done by gradient descent.
MCPModel Context Protocol: An open protocol for exposing tools, data and prompts to any model client over a common interface, so one server works with every host that speaks it instead of being wired to one application.
Memory-bound: A workload limited by how fast data can be moved from memory rather than by arithmetic. Single-stream token generation is memory-bound, which is why a faster chip does not help it much but quantization does.
Mixed precision: Training with most arithmetic in 16-bit while keeping a master copy of the weights and the accumulations in 32-bit. Roughly halves memory and speeds up the maths without changing the result much.
Mixture of expertsMoE, sparse mixture of experts: A feed-forward layer split into many separate sub-networks, with a router sending each token to only one or two of them. Total parameters rise sharply while compute per token stays flat.
Multi-head attentionMHA: Several attention computations run in parallel over different learned projections of the same input, then concatenated and projected back. Each head can track a different kind of relationship.
Multi-query attentionMQA: Every query head sharing a single key and value head — grouped-query attention taken to its limit. The smallest possible cache, at some quality cost, which is why grouping is the usual compromise.

N

NCCL: NVIDIA's library for collective operations between GPUs — all-reduce, all-gather, broadcast — using the fastest link available. Every multi-GPU training framework sits on top of it.
Normal distributionGaussian distribution, bell curve: The symmetric, bell-shaped distribution that sums of many small independent effects tend towards, described entirely by its mean and variance.

O

Overfitting: Learning the training set itself, including its noise, instead of the pattern behind it. Training loss keeps falling while validation loss flattens and then rises.

P

PagedAttention: Storing the KV cache in fixed-size blocks that need not be contiguous, the way an operating system pages memory. It removes the fragmentation that otherwise wastes most of the cache, so many more requests fit at once.
Partial derivative: The rate a function changes as one of its inputs varies and the rest are held fixed.
PCAprincipal component analysis: Finding the directions along which data varies most, and describing each point by its position along the first few. Compresses many correlated columns into a handful of uncorrelated ones.
Perplexity: The exponential of average cross-entropy loss, read as the number of equally likely options a model is effectively choosing between at each token. Lower means less surprised by the text.
Pipeline parallelism: Giving each GPU a consecutive group of layers and passing activations along the chain. Communication is small, but devices sit idle waiting for work unless the batch is split into micro-batches.
Positional encodingposition embedding: Information about where a token sits in the sequence, mixed into its representation. Attention treats its input as a set, so without it "dog bites man" and "man bites dog" are the same input.
PR AUCarea under the precision-recall curve, average precision: Area under the precision-recall curve, which unlike ROC AUC takes no credit for correctly dismissing the abundant negative class. The right summary metric when positives are rare.
Precision: Of the items the model flagged, the fraction that were right. The metric to watch when acting on a false positive is expensive.
Prefillprompt processing: The first phase of inference, processing the entire prompt in one parallel pass to fill the KV cache and emit the first token. It is compute-bound, and its length is what sets time to first token.
Pretraining: The long first training run on a very large, general corpus, where a model learns language and most of what it knows. It is where nearly all the compute goes and where capability comes from.
Probability distribution: A description of how likely each possible outcome is, with the total across all of them equal to one. A model's next-token output is one over the vocabulary.
Prompt cachingprefix caching: Reusing the computed KV cache for a prompt prefix that has not changed, so a long fixed preamble is processed once and skipped thereafter. Large savings in latency and cost for anything with a stable system prompt.
Prompt injection: Text in content a model reads — a web page, an email, a document, a tool result — that the model follows as if it were an instruction from you.

Q

QLoRA: LoRA on top of a base model held in 4-bit, so a large model can be adapted on a single consumer GPU. Gradients flow through the quantized weights into the small trainable matrices, which stay at higher precision.
Quantization: Storing weights, and sometimes activations, at lower precision — 8-bit or 4-bit instead of 16-bit. Memory drops proportionally, and so does the memory traffic that limits generation speed.
Query, key and valueQ, K, V, QKV: The three vectors each position produces in attention: what it is looking for, what it offers to others, and what it passes on when chosen. A query scored against a key decides how much of that key's value is read.

R

Random forest: Many decision trees trained in parallel on different random samples of rows and columns, with their predictions averaged. Hard to overfit, almost free of tuning, and a good first baseline.
ReActreason and act: Interleaving a model's written reasoning with its tool calls, so each action follows an explicit thought and each observation feeds the next one. The pattern most agent frameworks are built around.
Recallsensitivity, true positive rate: Of the items that should have been flagged, the fraction the model found. The metric to watch when missing one is expensive.
Recall@k: The fraction of questions whose answer appears somewhere in the top k retrieved passages. The number that tells you whether a retrieval problem is worth working on at all.
Regression test: A check that something which used to work still works, run automatically on every change. In AI systems it usually means pinning the cases a past bug produced, so the same failure cannot return unnoticed.
Regularisation: Anything that deliberately makes fitting the training data harder, in exchange for better performance on data the model has not seen. Weight decay, dropout, early stopping and data augmentation are all forms of it.
Reranking: A second, more expensive scoring pass over the top few dozen retrieved candidates, reordering them before the best handful go into the prompt.
Residual connectionskip connection: Adding a sublayer's input to its output, so the sublayer only has to learn a correction rather than the whole mapping. Without it, gradients through a deep stack shrink to nothing and training stalls.
Residual stream: The running sum carried from the bottom of a transformer to the top, which every attention and feed-forward layer reads from and writes into. Treating it as a shared bus rather than a pipeline is what makes most interpretability work possible.
Retrieval-augmented generationRAG: Fetching relevant material at question time and putting it in the prompt, so the model answers from supplied text rather than from memory. Knowledge can then be updated, restricted and cited without retraining.
RLHFreinforcement learning from human feedback: Collecting human comparisons of model outputs, training a reward model to predict those preferences, then optimising the model against that reward. It is how helpfulness and tone get trained when nobody can write the target answer.
RMSNormroot mean square normalisation: Layer normalisation with the mean subtraction dropped: divide the vector by its root mean square and apply a learned scale. Slightly cheaper, works just as well, and so has replaced layer norm in most recent models.
ROC AUCarea under the ROC curve: The probability that a randomly chosen positive example is scored above a randomly chosen negative one. It measures ranking quality across all thresholds, independent of where you eventually set one.
Roofline model: A plot of achievable performance against arithmetic intensity, showing where a kernel stops being limited by memory bandwidth and starts being limited by compute. It tells you which of the two to fix before you optimise anything.
RoPErotary position embedding, rotary embeddings: Position encoded by rotating query and key vectors through an angle proportional to their index, so an attention score depends only on the distance between the two positions. Standard in modern open models, and the reason their context can often be stretched after training.

S

Scaling laws: The empirical finding that loss falls as a smooth power law in model size, data and compute, over many orders of magnitude. They let a large run's result be predicted from a series of small ones.
Self-attention: Attention where queries, keys and values all come from the same sequence, so each position looks at the other positions of that same input.
Singular value decompositionSVD: Factoring any matrix into a rotation, a scaling and another rotation, with the scale factors ordered by size. Keeping only the largest gives the best possible low-rank approximation, which is what PCA and low-rank adaptation both rest on.
Sliding-window attentionlocal attention: Restricting each position to attend only to the last w tokens instead of the whole prefix, making cost linear in sequence length rather than quadratic.
SMOTEsynthetic minority oversampling: Creating extra minority-class rows by interpolating between real ones and their neighbours, rather than duplicating them.
Softmax: A function turning a vector of arbitrary scores into positive numbers that sum to one. It exponentiates first, so it is far sharper than simply dividing by the total.
Speculative decodingdraft-and-verify decoding: A small fast model drafts several tokens, and the large model checks them all in a single pass, keeping the longest correct prefix. Output is distributed exactly as the large model's would be; only latency changes.
Statistical significance: The judgement that a measured difference is bigger than what chance alone would plausibly produce, given the sample size.
Stochastic gradient descentSGD: Gradient descent using one small random batch at a time rather than the whole dataset, so a step costs almost nothing and there are millions of them. The noise it introduces turns out to help generalisation.
Structured output: Making a model return data in a fixed shape — usually JSON matching a schema — so the next step in the program can rely on it rather than parsing prose.
Supervised learning: Learning a mapping from inputs to outputs from examples where the correct output is given. Classification and regression are its two halves.
SwiGLU: The feed-forward activation used in most current models: a gated linear unit with a swish activation, needing three weight matrices where a plain layer needs two. Slightly better loss per parameter than ReLU or GELU, at the same cost once widths are adjusted.
System prompt: Instructions supplied ahead of the conversation that set role, constraints and format. Models are trained to weight it more heavily than user turns, though not absolutely.

T

Tail latencyp95, p99, percentile latency: The latency at a high percentile: p99 is the figure 99% of requests come in under. It describes the slow requests that shape how a service is perceived.
Temperature: A divisor applied to the logits before the softmax, setting how much probability mass concentrates on the top candidates. Below 1 sharpens towards the likeliest token; above 1 flattens and lets unlikely ones through.
Tensor parallelismmodel parallelism: Splitting individual weight matrices across GPUs so each holds a slice of every layer. It needs a collective communication inside each layer, so it belongs within a fast-interconnect node rather than across nodes.
Throughput: How much work a system completes per unit of time — tokens per second, requests per minute. The number that decides cost per unit served.
Time per output tokenTPOT, inter-token latency: The gap between successive tokens once generation is under way. Combined with time to first token and the length of the answer, it fully predicts how long a response takes.
Time to first tokenTTFT: How long from sending a request until the first token comes back, dominated by prefill and therefore by prompt length. The number that decides whether a streaming interface feels responsive.
Token: The unit a model actually reads and writes: usually a common word, a word fragment, or a few bytes of text.
Tokenizer: The component converting text into a sequence of integers and back, using a fixed vocabulary learned before training. Every model family has its own, so counts do not transfer between them.
Tool calling: Giving a model a set of described functions it may ask for, so that instead of answering it can emit a structured request that your code executes and feeds back.
Top-k samplingtop-k: Keeping only the k highest-probability tokens, renormalising across them, and sampling from what is left. A fixed-size cutoff, regardless of how confident the model was.
Top-p samplingnucleus sampling, top-p: Keeping the smallest set of tokens whose probabilities add up to p, then sampling from that set. The cutoff moves with the model's confidence: a near-certain continuation leaves one candidate, an open-ended one leaves hundreds.
Train/validation/test split: Dividing data three ways: one part to fit on, one to make decisions against, and one untouched until the end to report an honest number.
Transformer: The architecture behind almost every current language model: a stack of identical blocks, each pairing attention with a per-position feed-forward network, wrapped in normalisation and residual connections.
Transpose: Flipping a matrix over its diagonal, so rows become columns. In model code it is usually there to line up the two dimensions a multiply needs to share.

U

Underfitting: A model too simple, too small or too briefly trained to capture the pattern, so it does poorly on training and validation data alike.
Unsupervised learning: Finding structure in data that carries no labels — groups, directions of variation, outliers — using only what is in the data itself.

V

Vanishing gradient: Gradients shrinking towards zero as they travel back through many layers, so early layers barely learn. Residual connections and careful normalisation are the standard fixes.
Variance: The average squared distance of a random quantity from its mean, measuring how spread out it is. Its square root, the standard deviation, is in the original units and easier to read.
Vector: An ordered list of numbers, which can be read as a point or a direction in as many dimensions as it has entries. Every representation inside a model is one.
Vector databasevector store: A store built to find the vectors nearest a query vector quickly, across millions of items, usually alongside metadata to filter on.
Vocabularyvocab size: The fixed set of tokens a model can read or emit, typically tens to hundreds of thousands of entries. Its size sets the width of the embedding table and of the final output layer.
VRAMGPU memory, device memory: The memory attached to a GPU, holding weights, activations, optimiser state and the KV cache. It is usually the constraint that decides what you can run at all.

W

Warmup: Starting a run at a tiny learning rate and raising it over the first few hundred or thousand steps. Early gradients are unreliable, and full-size steps taken against them can wreck a run before it starts.
Weight decayL2 regularisation: Shrinking every weight slightly towards zero at each step, so the model only keeps magnitude it is actually using.

Z

ZeROzero redundancy optimiser: Sharding optimiser state, then gradients, then parameters across data-parallel workers instead of replicating them, gathering each piece only when needed. Cuts memory per GPU by close to the number of workers.