Module 09: AI Security and Privacy

"Privacy is not an option, and it shouldn't be the price we accept for just getting on the internet." — Gary Kovacs

Opening scenario: An assistant pastes student names into a free chatbot. A phishing email uses deepfake audio of the "CEO." Legal asks whether GDPR applies to your new AI vendor. You are not in IT—but you must know safe deployment patterns. This module covers security risks, data privacy, regulations at a glance, and practical habits. No code required.

Learning Objectives

By the end of this module, you will be able to:

• Name common AI-related security risks (leakage, prompt injection, social engineering, model abuse).
• Apply data minimization and classification before using any assistant.
• Explain GDPR-style concepts (lawful basis, purpose limitation, rights) at a conversation level—not legal advice.
• Use vendor due diligence questions IT and legal will appreciate.
• Design safer workflows for client, patient, and student contexts at a high level.
• Run a tabletop exercise for an incident involving AI tools.

1. Threat Landscape: What Actually Goes Wrong

Data leakage: Sensitive text goes to vendor systems and may be logged or used depending on product settings and contract.

Prompt injection: Untrusted user text manipulates AI-connected automations (see Module 06).

Social engineering: Deepfakes and unusually fluent phishing messages.

Abuse: Harassment, scams, disinformation at scale.

Misconfiguration: Over-permissioned integrations between tools.

Real World Example

A manager pastes a customer list into a free chat to “clean formatting.” Lesson: even “quick” convenience can create retention and training exposure. Better path: redact to Client A/B, use an approved workspace, or ask IT for a sanitized template workflow.

Real-world examples

• Public reporting on corporate policy changes after employees pasted confidential material into consumer chat—the lesson is to use enterprise tiers or avoid pasting secrets.
• CEO voice scam: finance verifies out-of-band before wires.
• Support bots leaking internal guidance when prompts bypass guardrails.

Try This Now

• List five categories of information you must never paste into consumer chat; compare with your organization’s policy.

Table: Risk to control (plain language)

Discussion prompts

• What verification stops urgent wire fraud on your team?

2. Data Privacy Foundations (Non-Lawyer Overview)

Ideas common in GDPR and similar regimes

• Lawful basis for processing
• Purpose limitation (use data for what you said)
• Data minimization (collect only what you need)
• Storage limitation (delete when done)
• Rights (access, correction, deletion—varies by jurisdiction)
• Accountability (document decisions)

United States context: Sectoral laws (health, education, finance) plus state privacy laws. Always involve counsel for real programs.

Try This Now

• Ask your assistant: "Explain GDPR lawful basis to a small-business owner in 120 words—include a 'not legal advice' disclaimer."

Scenario: An EU customer requests deletion; your US CRM retains AI logs. Fix: map subprocessors, sign data processing agreements where required, and build an operational runbook.

Discussion prompts

• Where is your record of processing activities—even a simple spreadsheet is a start?

3. Safe Deployment Practices for Teams

Defaults

• Enterprise accounts with contractual limits on training on customer data (verify the contract)
• Single sign-on and multi-factor authentication
• Separate workspaces for sensitive programs
• Redact before paste
• Mark internal AI drafts clearly in version notes when helpful

Workflow

1. Classify data before tool selection.
2. Use approved tools only.
3. Log high-risk uses where policy requires.
4. Review access quarterly.

Real-world examples

• Hospital: no PHI in consumer chat; approved platform only.
• Law firm: client matter walls extend to AI workspaces.

Try This Now

• Write a ten-line morning habit checklist for safer AI use; post it near your monitor.

Table: Data class to rule

Discussion prompts

• Who audits compliance—and how often?

4. Vendor Due Diligence: Questions That Matter

Ask vendors:

• Subprocessors and data locations (regions)
• Whether customer content trains models—and how to opt out
• Retention of prompts and outputs
• SOC 2 or ISO reports availability
• Incident notification commitments
• Exit plan and export formats
• Indemnities and insurance (with counsel)

Try This Now

• Turn the list above into a one-page RFP attachment for your procurement file.

Discussion prompts

• Which contract clause would you refuse to sign away without escalation?

5. Regulations and Standards at a Glance

EU AI Act (orientation): Risk-based categories; higher-risk systems face additional requirements. Map your use cases with experts.

GDPR: Ground rules for personal data in EU/UK contexts with global reach in many cases.

United States: FTC expectations on deception and data practices; evolving state laws (for example, concepts similar to CPRA in California).

Standards: ISO/IEC 42001 (management system), NIST AI Risk Management Framework. Use as conversation starters with compliance partners.

Try This Now

• "Outline a 30-minute legal and compliance briefing agenda for executives on AI privacy—non-technical."

Table: When to call counsel

Discussion prompts

• How do you balance innovation speed with review gates?

Activities

1. Redaction drill: take a fake email with secrets; redact for a safe demo.
2. Vendor scorecard: grade two tools on ten questions from Section 4.
3. Phishing refresh: review the latest deepfake warnings from IT; share one tip at home.
4. Incident tabletop: script a leak; assign roles; timeline the response.
5. Policy gap scan: find three holes in your current AI norms.

Your Challenge

Create a Safe AI Use Card (laminate-friendly):

• Green, yellow, and red data examples
• Link or name for approved tools
• Where to report incidents: ___
• "If unsure, pause and ask: ___"

Distribute to your team; collect one question from each person.

Expanded challenge: step-by-step card production

1. Green column: List 5 example phrases or data types that are safe for approved tools with normal judgment (e.g., “public product description draft,” “anonymous aggregate counts you generated yourself”).
2. Yellow column: List 5 examples that need redaction or manager/IT judgment (e.g., internal strategy with no customer names but still sensitive).
3. Red column: List 5 never examples for consumer-grade tools (student IDs, patient details, credentials, unreleased financials—use your org’s real categories).
4. Approved tools: Name exact products or “use only enterprise X per IT”—not “any AI.”
5. Incident line: Write who (role/email), what to include (time, tool, what was pasted), promise (blameless for good-faith reports).
6. Pause question: One sentence, e.g., “Would I be comfortable if this screenshot appeared in the news?”
7. Pilot: Hand the card to five people; collect one question each; revise v1.1 in a week.

Sector-specific privacy triggers (conversation starters with counsel)

Not legal advice—use to brief professionals.

Comparison table: shadow IT vs. sanctioned experimentation

Discussion Corner

1. Logs: If prompts are retained 90 days, who may access them—IT only, or managers—and under what safeguards?
2. Clients: Should contracts require disclosure of subprocessors that touch AI features? What minimum language would you want?
3. International: If your team spans countries, which default data region wins—and how do you explain that to employees?
4. After a mistake: What restorative steps (notification, credit monitoring, process fix) fit your culture without scapegoating the clicker?

Try This Now (added)

1. Redaction race: Take a fake email with 8 sensitive tokens; race a colleague to redact for a demo—compare misses.
2. Ask your assistant: “List 20 types of data that should not go to consumer chat; group by regulatory theme.” Delete wrong items; add three from your org.
3. Vendor email drill: Draft a three-sentence note to IT asking whether tool X logs prompts—before you paste a yellow doc.

Key Takeaways

• Security and privacy failures are often workflow and training issues—not movie-style hacking.
• Minimize data; use enterprise controls when available.
• Vendor claims require contractual proof, not slide decks.
• Regulations are fragmented—expert help is part of the cost of serious programs.
• Tabletops build muscle memory before crisis.
• When unsure, stop and escalate—speed is not worth a breach.

Resources

• ENISA — AI and cybersecurity
• ICO UK — AI and data protection
• NIST AI RMF
• Your internal IT security page and data protection officer contact

Extended Walkthrough: Incident Response (Simplified)

Detect → contain (disable integration, revoke tokens) → assess scope → notify legal and IT → communicate with affected users if required → remediate → post-incident review.

Try This Now: Fill in your organization’s contacts on one page.

Deepfake Defense Playbook (Non-Technical)

• Call back using known phone numbers
• Shared secret phrases for finance approvals
• Slow down urgent requests
• Multi-person approval thresholds for large transfers

Comparison Table: Consumer vs. Enterprise AI Accounts

Discussion Prompts (Seminar)

• How do you train interns quickly without leaks?
• Could prompt logs become discoverable in litigation—what are the implications?
• What is your acceptable residual risk—and who signs for it?

Glossary

• PII / personal data: Information identifying or relating to a person (definitions vary).
• PHI: Protected health information (US HIPAA context).
• DPA: Data processing agreement.
• Subprocessor: A vendor’s vendor.

Reflection Journal

1. My riskiest habit to fix: ___
2. The policy I will follow even when busy: ___
3. Who I notify when unsure: ___

Facilitator Notes: Security Micro-Lesson (20 Minutes)

• Story one: paste mistake
• Story two: deepfake call
• Practice callback ritual
• Q&A

Quality Checklist: New Tool Approval

• Data flow diagram completed
• Legal review if PII is involved
• Security review if integrations touch core systems
• Training scheduled
• Offboarding plan documented

When Public AI Tools Are Often Inappropriate

Examples frequently cited by security teams: classified materials, sensitive law-enforcement operations, unreleased financial results without controls—your CISO defines the authoritative list for your organization.

Bonus: Weekly Self-Audit (5 Minutes)

• Did I use only approved tools for work content this week?
• Did any message contain names, IDs, passwords, or unreleased numbers?
• Did I verify one urgent request through a second channel?
• Do I know where to report a mistaken paste without delay?

Bonus Table: Pause Triggers Before You Paste

Sample Email to IT (Tool Request)

Subject: Request to evaluate [Tool] for [team] AI use

Include: purpose, data classification, desired integrations, pilot length, named owner, and whether customer data is involved. Ask for recommended tier and DPA status.

Closing

Privacy is respect made operational. Security is care for people who trust you with their data.