🧭 Lesson Overview
Day 7 introduced OWASP LLM Top 10 as a practitioner threat reference and positioned it within the broader framework stack. Today you go deep on all ten items — not just what they are, but how each attack actually works, what a real compromise looks like, which defenses address it, and critically, which defenses sound good but don't work.
This lesson covers the 2025 edition, which introduced two genuinely new categories (System Prompt Leakage and Vector & Embedding Weaknesses) and renamed or expanded three others (Excessive Agency, expanded from the 2023 entry; Misinformation, refocused from 2023's Overreliance; and Unbounded Consumption, expanded from 2023's Model Denial of Service) — changes that reflect what practitioners are actually encountering in production LLM deployments, not theoretical future risks.
A building inspector doesn't invent what to check on each inspection visit. They work from a code book — a ranked list of the failure modes that have actually caused buildings to collapse, catch fire, or flood, organized by frequency and severity. The OWASP LLM Top 10 is that code book for LLM application security.
Every item on the list represents a failure mode that practitioners are actually observing in deployed LLM applications. The ranking reflects community consensus on impact, exploitability, and prevalence. When a security team pen tests an LLM application, they work down this list — it's the agreed checklist that ensures they don't miss the failures that matter most in production.
Day 8 Learning Objectives
- Explain what the OWASP LLM Top 10 (2025) is, how it differs from the original OWASP Top 10, and what's new in the 2025 edition.
- For each of the 10 items: describe the vulnerability, explain the attack path, and identify the primary defense.
- Match real-world AI security incidents to specific OWASP LLM Top 10 categories.
- Distinguish between effective and ineffective defenses for each category.
- Map OWASP LLM items to ATLAS techniques and NIST AI 100-2 attack categories (cross-framework literacy).
- Build a prioritized defense matrix for a specified LLM application using the OWASP Top 10 as the organizing structure.
All 10 Items at a Glance — Click Any Card to Jump
🔓 Module 1 · Why OWASP LLM Top 10 Exists — and What's New in 2025
The traditional OWASP Top 10 for web applications (SQL injection, broken access control, XSS, and their relatives) was built around a fundamental assumption: the application executes code, and the code has bugs. Fix the bugs, secure the code, ship the application. LLMs violate this assumption completely — the "code" is the model's learned behavior, and you cannot inspect, verify, or patch it the way you patch a SQL query.
| Dimension | Traditional OWASP Top 10 | OWASP LLM Top 10 (2025) |
|---|---|---|
| Root cause | Code bugs — deterministic, reproducible, patchable | Model behavior — probabilistic, emergent, not directly patchable |
| Attack channel | HTTP requests, SQL queries, form inputs, file uploads | Natural language prompts — instructions and data share the same channel |
| Testing methodology | Fuzz inputs, check for injection, test access controls | Adversarial prompts, red-team role-play, systematic jailbreak testing |
| Defense philosophy | Input validation, output encoding, parameterized queries | Defense in depth — model + application + infrastructure layers |
| Supply chain | Third-party libraries with known CVEs | Foundation model training data, model weights, fine-tuning datasets — no CVE system |
| Memory/context | Applications are stateless or have explicit session state | LLMs have context windows that accumulate across turns — injection can persist |
| Output behavior | Outputs are data — attacker manipulates inputs to get malicious outputs | Outputs are natural language that downstream systems execute — model output is code |
What Changed: 2023 → 2025 Edition
Demoted: Improper Output Handling (#2 → #5)
New in 2025: Excessive Agency (LLM06), System Prompt Leakage (LLM07), Vector & Embedding Weaknesses (LLM08), Misinformation (LLM09), Unbounded Consumption (LLM10)
Why these five? They reflect production realities: agentic AI deployments (LLM06), RAG & system-prompt risks (LLM07, LLM08), trust in AI outputs (LLM09), and pay-per-use cost attacks (LLM10).
| OWASP LLM Item | ATLAS Technique | NIST AI 100-2 Category | Deployment Type Most at Risk |
|---|---|---|---|
| LLM01 — Prompt Injection | AML.T0051, AML.T0054 | Evasion (GenAI) | Any customer-facing LLM |
| LLM02 — Sensitive Info Disclosure | AML.T0035, AML.T0024 | Privacy (training data extraction) | Fine-tuned LLMs on private data |
| LLM03 — Supply Chain | AML.T0048, AML.T0020 | Poisoning (supply chain) | Any LLM using external models or data |
| LLM04 — Data & Model Poisoning | AML.T0020, AML.T0019 | Poisoning (data) | Fine-tuned and RAG-based LLMs |
| LLM05 — Improper Output Handling | AML.T0043 | Evasion → downstream injection | LLMs integrated with code/DB systems |
| LLM06 — Excessive Agency | AML.T0086, AML.T0110 | No direct NIST category | Agentic AI with tool access |
| LLM07 — System Prompt Leakage | AML.T0024 | Privacy (model extraction) | Enterprise LLMs with custom prompts |
| LLM08 — Vector & Embedding Weaknesses | AML.T0054, AML.T0020 | Poisoning (RAG) | RAG-based applications |
| LLM09 — Misinformation | No direct ATLAS technique | Misuse (GenAI) | High-stakes information domains |
| LLM10 — Unbounded Consumption | AML.T0029 | Availability attack | Pay-per-use API deployments |
🔴 Module 2 · LLM01–LLM05 — Flip Cards & Deep Dives
Flip each card to see the real-world attack scenario. Then expand each item below for the full technical detail, real incident, defenses, and what doesn't work.
🟣 Module 3 · LLM06–LLM10 — Flip Cards & Deep Dives
The five new additions in 2025 reflect real production deployment patterns — agents with tool access, RAG pipelines, system prompt governance, and resource cost attacks. Flip and expand each.
🛡️ Module 4 · Defense-in-Depth Architecture for LLM Applications
No single control addresses all 10 OWASP LLM items. Effective LLM security requires layering defenses across three distinct layers — each addressing specific items and filling gaps the other layers cannot cover.
| Defense Layer | What It Covers | Primary Controls | OWASP Items Addressed |
|---|---|---|---|
| Model Layer | Controls built into the model itself — safety training, RLHF, alignment | Safety fine-tuning, adversarial training, output classifiers on generation, RLHF guardrails | LLM01 (partial), LLM09 (partial) |
| Application Layer | Controls in the code around the model — input validation, output handling, access controls | Input sanitization, instruction-data separation, output schema validation, privilege reduction, system prompt hardening, content filters, rate limiting | LLM01, LLM02, LLM05, LLM06, LLM07, LLM08, LLM10 |
| Infrastructure Layer | Controls in the deployment environment — API security, monitoring, supply chain, data governance | Vendor assessment, AIBOM, model provenance, API rate limiting, usage monitoring, DLP, incident response playbook | LLM03, LLM04, LLM06, LLM10 |
A system prompt instruction is like telling a brand-new employee: "Don't share confidential information." That's useful guidance for honest interactions. But it provides zero protection against a sophisticated social engineer. The actual controls — badge readers, locked filing cabinets, monitored exits, visitor logs — are in the physical infrastructure around the employee, not in the employee's verbal instructions.
LLM application security works the same way: the model is the employee, not the security system. The application layer provides the badge readers. The infrastructure layer provides the monitors. Defense-in-depth means not relying on the model to enforce its own security boundaries.
⚡ Activity A · OWASP Attack Scenario Lab
For each scenario, click the OWASP LLM item that best applies. Instant feedback explains the reasoning. Some scenarios are designed to be tricky — they may superficially resemble another item.
🔍 Activity B · Real Incident Case Study Matcher
Each entry describes a real or realistic documented AI security incident. Select the OWASP LLM Top 10 (2025) category that best classifies it. Click "Check All" when done.
🛡️ Activity C · Defense Builder — MVCC Enrollment Chatbot
For each OWASP LLM item, a set of proposed controls is shown. Check every control that effectively addresses that specific risk. Some controls sound reasonable but don't actually work for that item — identify which.
📝 Assessment Artifact
Open-Book Quiz: "Given This Scenario, Which OWASP LLM Category Applies and Why?"
The assessment artifact for Day 8 is the Day 8 Knowledge Check quiz below — completed open-book, submitted with explanations. For each question where you select an answer, write one sentence explaining why that category applies and one sentence identifying the primary defense. Submit your answers and explanations as a document by end of class.
Additionally: On a half-sheet, rank the OWASP LLM Top 10 items by priority for the MVCC Enrollment Chatbot (from highest to lowest) with a one-sentence justification for your top 3 choices. This ranking feeds into Day 11's secure architecture design.
Grading: Daily assignment (25% of course grade). Evaluated on accuracy of category identification, quality of defense reasoning, and appropriateness of the MVCC priority ranking.