Module 05: Data-Driven Decisions with AI

"Without data you're just another person with an opinion." — W. Edwards Deming (AI helps you see data—you still choose)

Opening scenario: The leadership meeting starts in an hour. You have exports from three systems, a PDF dashboard nobody trusts, and a colleague who says, "Just ask ChatGPT what we should do." You need clarity: how to use AI to summarize, question, and stress-test numbers—without letting a chatbot invent your quarterly results. This module is for non-technical professionals who must decide with data.

Learning Objectives

By the end of this module, you will be able to:

• Explain when AI can help with analytics—and when it must not guess missing numbers.
• Use natural language workflows to explore tables you provide (CSV excerpts, sanitized summaries).
• Describe how dashboards and self-service BI tools (conceptually) differ from chat assistants.
• Draft decision memos that separate facts, interpretations, and recommendations.
• Facilitate data literacy conversations with stakeholders who fear spreadsheets.
• Apply a verification ritual before any AI-derived number reaches a client or board slide.

1. What "AI for Data" Means Without Coding

Pattern: AI can rephrase metrics, compare scenarios described in text, suggest analyses to run, and summarize tables you paste. It should not fabricate rows you did not supply.

Real World Example

A board packet almost ships with last year’s chart caption because an assistant “smoothed” the narrative. Fix: final figures only from the BI export; AI words must cite row labels you provide. Golden rule: if it isn’t in the export, it isn’t in the slide.

Real-world examples

• Principal: Pastes anonymized attendance summary; asks for plain-language trends for a board paragraph—numbers copied back from the source file manually.
• Fundraiser: Uploads sanitized donor segment counts; asks for hypotheses to test next quarter—analyst validates with SQL later.
• Retail manager: Pastes weekly sales bullets; AI drafts manager notes; POS remains system of record.

Try This Now

• Create a tiny fictional table (5 rows) of product sales. Ask: "Describe trends; if you lack data to conclude, say so."

Table: Safe vs. risky asks

Discussion prompts

• Who is accountable if a polished narrative misstates a KPI?

2. Dashboards, BI Tools, and the Role of Chat Assistants

Dashboards (Looker, Power BI, Tableau, Metabase—examples): governed metrics, refresh schedules, permissions.

Chat assistants: flexible language, risk of hallucination if not grounded in your data.

Hybrid pattern: Some products connect chat to verified datasets—admin must configure governance.

Step-by-step: board packet workflow

1. Export final KPIs from BI tool to PDF humans reviewed.
2. Ask AI: "Summarize these KPIs for trustees in 6 bullets—do not add metrics." Paste only the final numbers.
3. CFO signs off.

Real-world examples

• Hospital admin: Dashboard shows occupancy; AI drafts executive summary—COO verifies.
• Startup: Uses Notion databases + manual exports; AI helps narrative only.

Try This Now

• "What questions should I ask before trusting a new 'AI analytics' vendor?" (Use output as RFP starter.)

Discussion prompts

• When is flexibility of chat worth less than auditability of a dashboard?

3. Natural Language Queries: Asking Better Questions

Good questions name:

• Population (customers, students, SKUs)
• Time window
• Metric definition (revenue vs. cash)
• Segment (region, cohort)

Try This Now

• Write five questions about your own work metrics as if to a data analyst. Ask AI to rewrite them clearer.

Table: Question quality

Scenario: A manager asks AI for industry benchmark numbers. Risk: outdated or made-up. Fix: ask AI for search queries and source types to check; you pull primary research.

Discussion prompts

• What metric definitions does your org argue about every quarter? Document them.

4. Decision Memos: Structure AI Can Help With

Template

• Context (1 short paragraph)
• Facts (bullets with sources)
• Interpretation (labeled explicitly)
• Options (A/B/C with trade-offs)
• Recommendation (owner, date, risk)
• Next metrics (what would change our mind)

Try This Now

• Paste two factual bullets you know are true about a hypothetical project. Ask AI to draft the memo skeleton without inventing benefits.

Real-world examples

• City planner: AI organizes public comment themes from pasted anonymized quotes—staff verifies representativeness.
• Product manager: AI lists pros/cons of feature launch based on provided user research excerpts.

Table: Decision biases to watch

Mitigation: assign red team reader; require citations to internal data.

5. Data Literacy for Teams (No SQL Required)

Practices

• Metric dictionary one-pager: name, definition, owner, refresh.
• Office hours with analytics monthly.
• "Explain like I'm busy" pair sessions.

Try This Now

• Ask your assistant for a one-page curriculum to teach non-analysts about sample size and seasonality—edit to your culture.

Scenario: Teachers interpret dashboard color changes as panic when noise is normal. Fix: training on confidence and variance in plain language.

Discussion prompts

• How do you celebrate good data use without weaponizing metrics?

Activities

1. Metric dictionary draft: Pick five KPIs you touch; write definitions; peer review.
2. Hallucination drill: Ask AI for exact figures about your organization without pasting data; record every error.
3. Memo lab: Write a decision memo with and without AI; time yourself; compare clarity.
4. Dashboard tour: With permission, screenshot one chart; ask AI to explain axes—verify against tooltips.
5. Stakeholder simulation: Explain one metric to a non-expert in 60 seconds; AI critiques jargon.

Your Challenge

Create a "Data & AI Use" mini-policy for your team:

• Green uses (summaries of pasted approved tables)
• Yellow uses (brainstorming analyses to run)
• Red uses (anything involving PII or regulatory numbers without sign-off)
• Verification steps before external sharing

Present it as a one-minute verbal brief.

Expanded challenge: step-by-step

1. Inventory three places your team gets numbers (CRM export, finance report, survey tool, LMS analytics—your reality). For each, name the system of record in one phrase.
2. Write green examples: tasks where AI only rephrases or structures numbers that already appear in an approved export you paste.
3. Write yellow examples: AI suggests what to analyze next or how to phrase a question—outputs are treated as hypotheses until an analyst or owner validates.
4. Write red examples: anything involving student records, patient identifiers, unreleased financials, trade secrets, or legal discovery—match to your org’s data classes.
5. Add a verification mini-ritual (3–5 bullets) that must happen before numbers go to a board, client, or press.
6. Rehearse the one-minute brief twice: once for a non-technical peer, once for a data skeptic—adjust wording.

Industry snapshots: questions to ask with data + AI

Comparison table: Dashboard-first vs. chat-first analytics

Discussion Corner

1. Accountability: If a polished AI paragraph misstates a KPI on a public slide, who is accountable—the author, the reviewer, or the tool? What process makes that obvious beforehand?
2. Denominators: What metrics in your org get argued about because numerator and denominator are ambiguous? Could you publish a one-page metric dictionary?
3. Narrative bias: When does “storytelling” about data help decisions—and when does it smooth over uncertainty you should keep visible?
4. Equity: Could summarizing data with AI hide disparities (by neighborhood, product line, or support tier) that deserve explicit review?

Try This Now (added)

1. Paste a tiny table you invented (5 rows). Ask: “List only conclusions supported by this table; for each, quote the cells. If you cannot quote cells, do not conclude.” Grade the model: did it obey?
2. Ask: “Give me 8 search queries I should run to verify a benchmark claim about [your industry]—no numbers in your answer.” Run one query yourself.
3. Take a paragraph you wrote about “trends.” Ask the tool to highlight causal language (because, driven by)—ensure each phrase maps to evidence you control.

Key Takeaways

• AI helps communicate and explore data you already have; it is a poor oracle for facts you withheld.
• Dashboards and governed BI anchor truth; chat adds flexibility with risk.
• Better questions beat fancier tools.
• Decision hygiene separates facts from interpretation—AI can scaffold, not substitute.
• Literacy is a team sport: definitions, training, and psychological safety.
• Verify every number that could affect money, safety, or rights.

Resources

• Harvard Data Science Review — accessible articles (pick topics).
• Your organization's BI or data office internal wiki.
• OWID — public charts for practice reading trends.
• Module 09 for privacy when data is sensitive.

Extended Walkthrough: Monthly Business Review (Fictional)

Inputs: Revenue $120k, $118k, $125k last 3 months; churn 2.1%, 2.4%, 2.0%.
Ask AI: "Draft a neutral summary—no advice."
Human: Add causal notes only if known (e.g., holiday closure).
Pitfall: AI invents reasons—delete unsourced causality.

Comparison Table: Tools by Job-to-Be-Done

CFO / Principal / ED Talking Points

• "We use AI to wordsmith verified numbers, not to discover them."
• "Our system of record is [tool]; AI is not a database."
• "If it's not in the export, it's not in the slide."

Red Flags in "AI Analytics" Sales Pitches

• "No data prep needed" for messy enterprises
• Vague accuracy claims without baseline
• Promises to replace your data team entirely
• No audit logs

Discussion Prompts (Seminar)

• How do you version metrics when definitions change mid-year?
• What single chart would make your team smarter next month?
• Should AI-generated insights be signed by a human owner?

Reflection Journal

1. A metric I finally understand: ___
2. A metric I still distrust: ___
3. One question I will ask analytics next time: ___

Glossary

• KPI: Key performance indicator—should have a definition.
• ETL: Extract-transform-load—how data moves (you may not do it; you should know who owns it).
• Grounding: Tying answers to provided or indexed sources.
• System of record: The authoritative place numbers come from.

Quality Gate: Before the Board Sees It

• Numbers copied from approved export
• Axis labels understood
• Comparisons are apples-to-apples
• Causal language reviewed
• Footnotes for anomalies

When to Involve a Data Professional

Involve analytics early when decisions hinge on joining datasets, statistical significance, sampling bias, or compliance mapping. AI cannot see data it was not given.

Appendix: Sample Natural Language Prompts (Safe Patterns)

• "Given this table only [paste], which categories rose quarter-over-quarter? If unclear, say unclear."
• "List five analyses a data analyst might run next; do not claim results."
• "Rewrite this metric footnote for parents; grade-7 reading level."

Scenario: Grant Reporting

A nonprofit reports outputs to a funder. AI polishes narrative; program director supplies counts from CRM; ED signs. Never let AI inflate beneficiaries served.

Closing

Data-driven means disciplined, not fancy. AI can accelerate understanding when humans guard the facts.

Bonus Activity: "Source or Strike"

Take any AI paragraph about your work. Highlight every factual claim. For each, either paste the source line from your export or strike the sentence. If more than two sentences fail, re-prompt with tighter constraints: "Narrative only from this table; no external knowledge."

Bonus Table: Executive Questions That Keep Teams Honest