🧭 Lesson Overview
Every risk framework you've used in traditional cybersecurity — threat modeling, risk registers, CIA triad — was built around a core assumption: the systems you're protecting are deterministic, their inputs are controlled, and the threats against them come from outside. AI breaks all three assumptions simultaneously.
Today you build the AI-specific risk vocabulary that makes the rest of this course operational. The five AI risk categories (model, data, operational, ethical, third-party) give you a taxonomy that maps to NIST AI RMF MAP outputs. Shadow AI — the hidden AI already operating inside your organization — gives you the most urgent practical threat to address before any of the other categories matter.
An organization that believes it has "five AI systems in production" almost certainly has dozens. The five visible ones are the tip above water — approved, registered, reviewed. Below the surface are: the ChatGPT subscription on the marketing team's credit card, the GitHub Copilot running on the developer's personal laptop, the AI-powered browser extension installed by 23% of staff, and the AI features that silently activated in the SaaS tools you renewed last quarter.
Shadow AI is the submerged portion of the iceberg. Traditional IT governance sees only what's above water. AI governance requires sonar — the detection, policy, and monitoring tools that surface what's below.
Day 6 Learning Objectives
- Explain why the CIA triad is necessary but insufficient for AI risk management.
- Define and differentiate the five AI risk categories: model, data, operational, ethical, and third-party.
- Apply each risk category to a real AI deployment scenario with specific examples.
- Describe the three forces (velocity, pervasiveness, personalization) that create the AI security gap.
- Define shadow AI, identify its common patterns, and distinguish it from traditional shadow IT.
- Apply shadow AI detection methods across network, SaaS, endpoint, and behavioral signals.
- Select appropriate governance responses (block, channel, register, monitor) for different shadow AI scenarios.
🔺 Module 1 · Why the CIA Triad Is Necessary but Insufficient
The CIA triad — Confidentiality, Integrity, Availability — is the most foundational model in information security. It maps to every control, every regulation, and every risk assessment in traditional infosec. For AI systems, it still applies — and it still misses the most consequential AI-specific risks entirely.
| CIA Property | Traditional Meaning | AI Extension Required | What CIA Misses |
|---|---|---|---|
| Confidentiality | Unauthorized parties cannot read the data | Training data, prompt content, model weights, and intermediate embeddings can all expose sensitive information | Model memorization — a model can "leak" training data in its outputs even with no direct data access by the attacker |
| Integrity | Data and systems are not modified without authorization | Training data poisoning modifies model behavior without modifying any file or database record in the traditional sense | Behavioral integrity — the model produces correct code 99.9% of the time, then produces subtly wrong code for one specific function. No hash check detects this. |
| Availability | Systems are accessible when needed | Unbounded token consumption (OWASP LLM10) and adversarial prompts can degrade AI service availability | Quality availability — the system is up and responding, but producing degraded, biased, or harmful outputs. Traditional availability monitoring shows green while the system is actively causing harm. |
2. Behavioral risk — CIA covers whether data flows to unauthorized parties and whether systems are running. It has no concept for "the system is running correctly but producing outputs that discriminate against a protected demographic group."
3. Emergent harm — AI systems can cause harm through their designed, intended outputs — outputs that are not a breach, not unauthorized, and not unavailable. CIA has no category for harm-from-correct-operation.
🗂️ Module 2 · Five AI Risk Categories — Overview
The five AI risk categories provide the vocabulary for MAP worksheets, risk registers, and governance discussions that go beyond what CIA triad supports. Each category has distinct characteristics, distinct failure modes, and distinct control families. Critically: a single AI deployment may simultaneously carry risks in all five categories.
Risk arising from the model's own properties: hallucination, drift, bias baked into architecture or weights, performance degradation on subpopulations, non-determinism making validation impossible.
CIA analogy: Closest to integrity — but behavioral integrity is not the same as data integrity. A model producing consistently wrong outputs for a specific demographic is a model risk failure, not an integrity violation in the traditional sense.
Risk arising from data used to train, fine-tune, or query the model. Data is not just an asset to protect — it is a control plane. Poisoned data changes what the system does. Biased data produces biased outputs. Regulated data in training creates legal exposure.
CIA analogy: Extends confidentiality and integrity — but the unique risk is data-as-code. The same poisoning attack that traditional infosec would classify as "data modification" in AI becomes a behavioral control attack.
Risk from deployment decisions, integration architecture, human oversight failures, process gaps, and the operational environment in which the model runs. A model that works perfectly in testing can be operationally catastrophic when deployed in the wrong context, with the wrong oversight model, or at the wrong scale.
CIA analogy: Closest to availability — but operational risk covers quality and appropriateness, not just uptime. A model that's up but being misused by its operators is an operational risk.
Risk arising when the system functions exactly as designed — but that design causes harm to individuals, groups, or society. Disparate impact, dignity violations, autonomy erosion, concentration of power, and democratic harms. This is the risk category CIA has no vocabulary for.
CIA analogy: None. Ethical risk is the category that falls entirely outside traditional infosec models — a system can be fully confidential, fully intact, and fully available while systematically harming an entire demographic group.
Risk from components, data, and services sourced from outside the organization — foundation models, training datasets, plugins, APIs, SaaS AI features. The AI supply chain has unique opacity: you often cannot fully inspect what a foundation model was trained on or what behavioral properties it has inherited.
CIA analogy: Extends traditional supply chain risk — but AI supply chain opacity is uniquely severe. A SBOM can document software dependencies; an AIBOM can document AI component provenance but cannot guarantee behavioral properties inherited from foundation model training.
Flip Cards: Five Categories — Sub-Risks and Controls
Each flip card covers a specific sub-risk within one of the five categories. Front: what it is. Back: what controls address it and which NIST AI RMF subcategory applies.
🔬 Module 3 · Risk Category Deep Dives — Expand for Full Detail
Click any risk category to expand it. Left column: what the risk means in practice and how it manifests. Right column: MVCC-specific example, real-world incident reference, and the NIST AI RMF category that addresses it.
⚡ Module 4 · Velocity, Pervasiveness & Personalization
Traditional IT governance was designed for a world where new technology adoption is relatively slow, inventoried, and controlled. AI has broken all three of those conditions simultaneously. Three forces explain why AI security gaps form faster than governance can close them.
AI adoption outpaces governance development by an order of magnitude. The typical enterprise AI governance program takes 12–18 months to build. A new generative AI capability appears every 4–6 weeks. By the time governance catches up to ChatGPT, employees are already using GPT-4o, Gemini Ultra, and Claude 3.5.
GenAI traffic grew 890% in 2024 (Menlo Security). Shadow AI surged 68% in 2025 alone. Governance policies written in 2023 are now multiple product generations stale.
AI is now embedded in tools your organization already approved. Microsoft Copilot in Office 365. AI writing in Google Workspace. AI code completion in your IDE. AI features in your CRM, your ITSM, your HR platform. Your SaaS vendors are activating AI features in your existing subscriptions — often without a notice prominent enough to trigger an AI governance review.
Organizations manage an average of 490 SaaS applications (Reco, 2025). Only 47% are authorized. Nearly every enterprise SaaS platform acquired AI capabilities in 2023–2025.
AI tools are immediately personally productive — creating strong individual incentives to bypass controls. An employee who discovers that an AI tool makes them 40% faster at their job has a rational personal incentive to keep using it even after being told not to. 48% of employees would continue using banned AI tools (Offsec, 2026). 65% consider using unvetted AI acceptable.
This makes shadow AI fundamentally different from shadow IT: IT bans were inconvenient. AI bans require giving up a personal productivity multiplier that employees experience daily.
👁️ Module 5 · Shadow AI — Definition, Scale, and Exposure Paths
Shadow AI is any AI system, tool, or model used for work purposes outside the organization's AI governance program — without approval, risk assessment, or monitoring. It includes: consumer AI chatbots used for work tasks, developer API integrations without security review, AI-powered browser extensions, open-source models run locally on company devices, and AI features embedded in authorized SaaS tools that activated without IT awareness.
The key distinction from shadow IT: With shadow IT, data went in to unapproved storage. With shadow AI, data goes out — it flows into models that can learn from it, store it, and surface it in responses to other users. The data doesn't just sit in a bucket. It becomes part of something larger and far less controllable.
The Scale of the Problem — Current Statistics
The Data Exposure Chain
Shadow AI doesn't require a sophisticated attack. The most common exposure path is mundane:
Shadow AI Patterns by Business Unit — Click to Expand
These are the most common shadow AI deployments found during enterprise audits in 2025. Click each tile to see the specific risk and typical scale.
Shadow AI vs. Shadow IT — Critical Differences
| Dimension | Shadow IT | Shadow AI |
|---|---|---|
| Data flow | Data goes in — stored in unauthorized system | Data goes out — processed by external model, potentially retained |
| Detection method | Application inventory, DNS queries, network block lists | Network DLP, API endpoint monitoring, behavioral analytics — block lists insufficient |
| Visibility | New software installation visible to endpoint management | Browser extension, web session, personal API key — invisible to traditional tools |
| Employee incentive | Convenience, preference | Productivity multiplier — strong personal incentive even after bans |
| Regulatory exposure | Data location violation, licensing issue | EU AI Act deployer obligations, GDPR data transfer, FERPA, IP exposure, model training contribution |
| Reversibility | Remove the app, data remains in control | Data already processed by external model — cannot be "un-trained" |
| EU AI Act impact | Not specifically addressed | Deployer obligations apply to ALL AI system use — shadow AI deployments may be EU AI Act non-compliant |
Shadow AI Detection Methods — The Four-Layer Approach
🛡️ Module 6 · Governance Responses — Block, Channel, Register, Monitor
Effective shadow AI governance is not a binary: block or allow. A four-response framework matches the response to the risk level of the tool and its use context. The goal is always maximum visibility with minimum friction — pushing usage into managed channels rather than underground.
| Response | When to Apply | Implementation | Risk Tradeoff |
|---|---|---|---|
| 🚫 BLOCK | Tools with no legitimate use case AND high data exposure risk. High-risk AI practices prohibited by EU AI Act Article 5. Tools actively exfiltrating data to jurisdictions with no data residency controls. | DNS block, proxy block, browser policy. Communicate block reason to users. Provide approved alternative simultaneously. | Risk: Drives usage underground to personal devices and hotspots. Only effective when combined with alternative provision. Never block without communicating why. |
| ✅ CHANNEL | Tools with legitimate use cases that can be made safe through configuration, enterprise contracts, or provisioned alternatives. Most consumer AI tools fall here — the use case is legitimate, the governance gap is the implementation. | Provide enterprise-contracted versions (ChatGPT Enterprise, Claude for Work, Copilot M365). Configure DLP policies for approved tools. When approved alternatives are provided, unauthorized use drops 89%. | Cost of enterprise licensing vs. cost of uncontrolled data exposure. Almost always favorable. Channel is the most effective shadow AI governance response. |
| 📋 REGISTER | Tools already in use that represent manageable risk and legitimate productivity value. Cannot be removed without significant disruption. Goal: make the invisible visible through a self-registration or discovery program. | AI Use Case Register self-declaration process. Amnesty period where employees declare current AI tool use without penalty. Discovery tools that surface existing usage. Use declarations to assess, then move to Channel or Block as appropriate. | Creates visibility immediately. Some registered tools will need to be subsequently blocked or channeled — the registration is not permanent approval. Treat it as a gap analysis, not an approval mechanism. |
| 👁️ MONITOR | Tools approved for general use that require ongoing surveillance for data classification violations, behavioral anomalies, or scope creep. The approved tools in your AI Use Case Register that don't need constant human review but do need automated watching. | API call pattern monitoring, DLP integration with AI tool output channels, behavioral analytics for anomalous usage patterns, automated alerts when sensitive data classifications appear in AI tool prompts. | Monitoring without action is theater. Monitor only when there is a defined alert threshold and a defined response action. Otherwise, instrument for reporting rather than real-time monitoring. |
AI Acceptable Use Policy — What Must Be in It
🧑⚖️ Module 7 · Human Factors — Automation Bias & Over-Reliance
Almost every control in this course eventually rests on one load-bearing assumption: a human is in the loop. Human oversight is the fallback for model risk, the check on operational risk, and the last line before an AI output becomes an institutional action. But we have so far treated that human as reliable by default. Human-factors research is unambiguous that they are not (Parasuraman & Riley, 1997, on the use, misuse, and disuse of automation). Oversight is a control that has to be designed — and it fails in predictable, well-studied ways when it is merely assumed.
This is not a soft-skills aside. It is the mechanism behind the Operational Risk failures in Module 3: the model can be accurate and the policy can say "a human reviews every output," and the system can still cause harm because that oversight existed only on paper.
How a Fluent AI Answer Erodes Scrutiny — Three Mechanisms
The tendency to favor an automated suggestion over your own judgment — and over contradicting evidence. A fluent, confidently-worded AI answer reads as authoritative, so reviewers weight it more heavily than they would weight a hesitant human colleague. Errors of commission (acting on a wrong AI recommendation) and omission (failing to act because the AI stayed silent) both rise.
When a system is usually right, attention decays. Parasuraman & Manzey (2010) find that complacency grows with reliability: the more trustworthy the automation, the less the monitor actually monitors. The rare error then arrives exactly when scrutiny is lowest — and is the most likely to be missed.
Skills that are not exercised atrophy. As AI handles the routine cases, the human's ability to recognize the abnormal one degrades — the very expertise the oversight role depends on. Over time the "human in the loop" is less capable of catching the model than on day one, even as the org still counts them as the safeguard.
Making Oversight Real, Not Nominal — Design Patterns
Oversight that works is engineered into the interface and the workflow — not requested in a policy sentence. Each pattern below converts a nominal check into a real one.
| Pattern | Nominal Oversight (fails) | Real Oversight (works) |
|---|---|---|
| Forcing function | "Advisors should review each flag." | The rationale panel must be opened, and a one-line justification typed, before the action button unlocks. |
| Friction on high-impact actions | One-click "Accept" for every case, low- and high-stakes alike. | Low-stakes actions stay fast; high-impact decisions (deny aid, escalate) require a second step, a confirmation, or a short cooling-off delay. |
| Show confidence & provenance | A single authoritative-looking label ("High Risk"). | The output shows its confidence/uncertainty, the drivers behind it, and its source — so a low-confidence answer visibly invites scrutiny instead of deference. |
| Independent second-person review | The person who ran the AI also approves its output. | High-impact AI outputs get a second reviewer who is not anchored by the AI's answer first. |
| Measure override rates | Nobody knows how often humans actually disagree with the AI. | Override / agreement rates are logged and reviewed. A near-100% agreement rate is treated as a rubber-stamp alarm, not a success. |
Day 14 (Incident Response): When an AI incident is live — a deepfake authorizing a wire, a prompt injection exfiltrating records — the responder is the human in the loop, under time pressure and the strongest possible pull toward automation bias. That is exactly why the AI-adapted NIST SP 800-61r3 response steps must be pre-designed and rehearsed in tabletops: so oversight survives the moment it is most likely to collapse.
Across both days the lesson is the same: oversight is something you design, instrument, and rehearse — never something you assume.
Applied Activity · Spot the Over-Reliance Failure Mode
Each item describes a human interacting with an AI output. Decide whether it is real oversight (independent judgment actually applied) or a rubber-stamp (nominal review, no real scrutiny). Click your choice to check.
🔍 Activity A · Shadow AI Risk Category Classifier
✅ Activity B · Shadow AI Detection Capability Audit
Check each detection capability your organization currently has in place. Score your maturity at the end. Work through this as if you are the CISO of MVCC assessing your current shadow AI visibility.
⚡ Activity C · Risk Category & Governance Decision Lab
Each scenario describes an AI situation at an organization. Choose the best risk category classification or governance response. Some scenarios test category identification; others test governance decision-making.
📝 Assessment Artifact
Shadow AI Risk Assessment — MVCC Scenario Analysis
Complete a two-part assessment for the following scenario: MVCC's IT department discovers that 34 faculty and staff members have been using personal ChatGPT accounts to assist with grading, creating course materials, writing recommendation letters, and analyzing student performance data. None of these uses were approved, registered, or reviewed.
- Part 1 — Risk Category Analysis (200 words): Identify which of the five AI risk categories this situation implicates and specifically how. Which is the most critical, and why? Does the EU AI Act create any additional obligations?
- Part 2 — Governance Response Plan (200 words): What governance response (Block/Channel/Register/Monitor) is most appropriate for each use case identified? Design a 90-day response plan using the four-response framework. Include: immediate action (Day 1), short-term stabilization (Days 2–30), and sustainable governance (Days 31–90).
Grading: Daily assignment category (25% of course grade). Evaluated on accuracy of risk category identification, appropriateness of governance response selection, and actionability of the 90-day plan.