Ship an AI feature and two vocabularies arrive together: the safety language from our responsible AI guide and the attacker language that fills hardening guides like this AgentCore walkthrough. This lesson separates the families and adds a third: the AI safety terms researchers use when they open a trained network and look at the wiring. Sixteen entries, grouped by what they defend against.
Safety and security are different jobs
Four terms that draw the lines between the two disciplines.
- AI safety: Harms a system causes while working exactly as built: biased outputs, confident misinformation, dangerous advice, misuse the operator never intended. Mitigation lives in training, policy, and product decisions, and progress is measured in incident reports rather than penetration tests.
- AI security: Harms from adversaries attacking the system on purpose: injected instructions, stolen weights, extracted training data. It borrows its methods from information security, and it is the half of the risk register that assumes someone out there is hostile.
- Red teaming: Attacking your own model or product before strangers do, using prompts, tools, and persistence. A red team finding is a bug report written by an enemy, and teams that skip the exercise tend to meet the same findings in production.
- Guardrails: Runtime checks wrapped around a model: input filters, output validators, policy classifiers, rate limits. The road barrier metaphor fits; the model can still steer badly, but the worst lanes are blocked before anyone gets hurt.
The attack vocabulary
Four terms for how people actually break language model systems.
- Prompt injection: Instructions hidden inside content the model reads, a web page, a document, an email, which the model then follows as if they came from its operator. It is the defining risk of agents that touch untrusted text, because the model cannot reliably tell data from commands.
- Jailbreaking: Crafted requests that talk a model past its refusal training, usually through persona play, hypothetical framing, or encoding tricks. The difference from injection is the source: a jailbreak comes from the user, while injection rides in on third-party content the user may never see.
- Data poisoning: Corrupting a training corpus so the trained model carries an implanted flaw or backdoor. The realistic modern route is the supply chain: scraped web data, purchased datasets, and third-party corpora are all places an attacker can reach before you do.
- Adversarial example: An input adjusted by small, often invisible amounts so a model flips its answer, the classic example being a stop sign the vision system reads as a speed limit. Language models face the text version, and both families share the uncomfortable property that perturbations transfer between models.
Privacy and provenance
Four terms from the data side of the risk register.
- PII: Personally identifiable information, the names, addresses, and identifiers that must not leak out of training data, logs, or generations. Handling it is a pipeline problem: redaction before training, filtering during generation, and access control everywhere else.
- Differential privacy: A mathematical guarantee that any single person’s data changes the trained model’s outputs by at most a small, chosen amount. The budget, usually called epsilon, is the dial: tighter privacy means more noise and less accuracy, and honest reporting means publishing the setting rather than the slogan.
- Watermarking: Embedding a statistical signature in generated text so later samples can be recognized as machine written. Detection stays probabilistic rather than certain, and the signature survives editing only partially, which keeps this an arms race rather than a solved problem.
- Model card: The documentation artifact that ships with a release: training data summary, intended uses, known limitations, evaluation results. Its job is to make claims checkable, and the habit has spread upstream to datasets as datasheets, the same idea one step earlier in the pipeline.
Opening the box
Four terms from interpretability, the research program that treats a trained network as an object you can read.
- Explainability: The applied half: giving humans a checkable account of why a system produced the output it did, using tools like attribution scores and surrogate models. The demand is regulatory as much as scientific, since deployment paperwork keeps asking exactly this question.
- Mechanistic interpretability: The research half: reverse engineering the network’s internals, mapping features onto the neurons and circuits that compute them, until behavior can be explained as mechanism rather than correlation. Slower, harder, and the only route to accounts that survive adversarial scrutiny.
- Sparse autoencoder: The probe technique behind most recent progress: train a small decoder to reconstruct activations from a wider, sparse code, and the code’s units often line up with single human-meaningful concepts, untangling neurons that mix several at once.
- Circuit: A subgraph of the network that carries out one identifiable computation, found by ablating and patching until the pieces stop working together. Cataloging circuits is the closest this field comes to reading source code, one verified subroutine at a time.
Common confusions
Three pairs that blur together in policy discussions.
- Safety vs security: Safety work assumes a functioning system and asks what it does wrong. Security work assumes an adversary and asks what breaks under attack. The controls differ accordingly: training and policy on one side, isolation and validation on the other.
- Prompt injection vs jailbreaking: Injection plants instructions in content the model reads from third parties. Jailbreaking is the user’s own request, shaped to slip past refusals. The first is defended at the architecture level, the second at the training level, and confusing the two produces the wrong fix.
- Explainability vs interpretability: Explainability produces after-the-fact accounts of a fixed model, often approximate ones. Interpretability aims to account for the model’s actual computation. Regulators mostly ask for the first; safety researchers mostly need the second.
Further reading
Three references that anchor the three families above.
- The OWASP Top 10 for LLM applications, the shared checklist for the attack vocabulary, maintained by practitioners.
- The EU AI Act portal, the regulation most deployment paperwork now cites, with the text and timelines in one place.
- Toy Models of Superposition, the paper that explains why networks cram in more concepts than they have neurons, and why untangling them needed new tools.
This is lesson 13 of the AI glossary. Lesson 12 assembled the LLM evaluation terms in Evaluation and Benchmarks, and red teaming is precisely an evaluation run by an enemy. For deployment practice, our responsible AI guide and the AgentCore security walkthrough put these AI safety terms to work. The next lesson moves back inside the model: reasoning and test-time compute.