Module 09: Evaluating AI Outputs — Frameworks, RAGAS, Human Evaluation & Research Practice

Learning Objectives

By the end of this module, you will be able to:

1. Apply multi-criteria evaluation rubrics — Score model outputs on correctness, completeness, calibration, clarity, safety, and provenance; justify weights for coursework and capstone reports.

2. Implement retrieval-augmented evaluation concepts — Explain faithfulness, answer relevance, and context precision/recall in RAG systems; interpret RAGAS-style metrics with appropriate skepticism about automation bias.

3. Design human evaluation protocols — Build annotation guidelines, adjudication rules, and inter-rater reliability plans suitable for small-n student studies.

4. Separate epistemic from stylistic quality — Avoid conflating fluent prose with factual accuracy; use triangulation and adversarial test sets.

5. Operationalize continuous evaluation — Maintain golden question sets, regression checks when prompts or corpora change, and documented limitations sections aligned with ML best practices.

Deep Concept Explanation: Academic Context, Research Methodologies, and Quality Frameworks

9.1 Why evaluation is epistemic infrastructure

Generative models optimize plausible continuation under a training objective; they do not inherently optimize truth or utility for your stakeholder. In research methodology terms, evaluation is how you connect construct validity (“does this measure what we claim?”) to downstream decisions (what to ship, cite, or publish). For students, evaluation discipline transfers directly to thesis methods, HCI user studies, and startup discovery (does the MVP actually reduce pain?).

9.2 Dimensions of quality: a construct map

Correctness — Factual alignment with verifiable sources or formal derivations. In RAG, correctness splits into intrinsic (internal consistency) and extrinsic (matches ground truth or authoritative documents).

Completeness — Coverage of constraints in the prompt (population, timeframe, jurisdiction, dataset slice). Omission is a silent failure mode.

Calibration — Match between stated confidence and actual reliability. Overconfidence is common; hedging can be appropriate or evasive—context matters.

Clarity — Audience-appropriate definitions, coherent structure, and unambiguous recommendations.

Safety and ethics — Potential for harm if wrong (medical, legal-like, psychological advice); presence of stereotypes or demeaning language.

Provenance — Ability to trace claims to documents, equations, or data you can inspect.

9.3 Human evaluation: gold standard with costs

Human ratings remain the reference for subjective qualities (helpfulness, tone) and for nuanced factuality when gold answers are incomplete. Methodological requirements:

• Guidelines with labeled examples (anchor paragraphs).
• Blinding where feasible (raters unaware of model identity).
• Multiple raters and agreement metrics (Cohen’s κ, Krippendorff’s α) for formal studies.
• Adjudication process for disagreements.

For coursework, a lightweight version still specifies what each score level means.

9.4 Automated metrics: opportunities and limits

N-gram overlap metrics (BLEU, ROUGE) measure lexical overlap with references; they underreward paraphrases and overreward copying.

Embedding similarity measures semantic closeness but can be fooled by contradictions stated in different words.

LLM-as-judge is convenient but risks position bias, verbosity bias, and self-preference if the judge shares the generator family.

Principle: use automation for screening and regression detection; use humans for final judgment on high-stakes outputs.

9.5 RAG evaluation: faithfulness and grounding

Retrieval-augmented generation couples a retriever (search, embeddings, BM25) with a generator. Failure modes:

• Retrieval miss — relevant chunk not retrieved.
• Retrieval noise — irrelevant chunks distract or mislead.
• Grounding failure — answer not supported by retrieved context despite plausible text.

RAGAS (research and open-source tooling) popularized automated scores such as:

• Faithfulness — Are claims in the answer entailed by the retrieved context?
• Answer relevance — Does the answer address the question?
• Context precision — How much retrieved context is useful?
• Context recall — Did retrieval capture needed evidence?

Methodological caution: automated RAGAS scores are noisy proxies; they depend on judge models and thresholds. Report them as diagnostics, not proof of production readiness. Always pair with spot checks and human review on stratified samples.

9.6 Golden sets and adversarial testing

A golden set is a curated list of questions with reference answers or must-cite document IDs. Expand with:

• Paraphrases — same intent, different wording.
• Ambiguity — underspecified prompts to test clarification behavior.
• Typos and noise — robustness checks.
• Edge cases — rare but critical policies (FERPA-like scenarios in campus bots).

9.7 Bias and fairness in evaluation

Training data skew yields omissions and stereotypes. Evaluation should disaggregate when possible: performance by demographic groups in synthetic scenarios (ethical care: no real PII), by domain (STEM vs. humanities prompts), and by English proficiency proxies. Connect to FAccT literature: fairness is multi-dimensional; fixing one metric can harm another.

9.8 Fact-checking as scholarly practice

Workflow:

1. Extract claims — split empirical vs. normative.
2. Prioritize — high-impact claims first.
3. Source ladder — primary > secondary > tertiary.
4. Triangulate — two independent lines when stakes are high.
5. Timestamp — record check date for volatile facts.

9.9 Self-consistency and ensemble checks

For uncertain factual answers, multiple samples at temperature > 0 can reveal instability. Disagreement is a signal to verify externally, not to average blindly.

9.10 Reporting limitations (research norm)

Mirror ML reporting: dataset size, demographic coverage (if known), metric choice rationale, failure taxonomy, and ethical limitations. This aligns with Model Cards and Datasheets for Datasets thinking.

9.11 Startups: evaluation as product risk

For MVPs, tie metrics to outcomes: time saved, error rate in labeled samples, user-reported trust, support ticket volume. Avoid vanity metrics (raw generations/day) without quality gates. Map each metric to an instrument (survey item, task completion, human rubric row).

9.12 Statistical notes for small samples

Student projects often have small n. Avoid overclaiming significance. Report confidence intervals where appropriate, or use exact small-sample language (“observed failure rate 3/20 in pilot”). Pre-specify stopping rules for iterative prompt tuning to reduce p-hacking analogies (repeated peeking until a metric looks good).

9.13 Cost and latency as evaluation dimensions

Production-like coursework should track tokens, latency percentiles, and dollar estimates per query. A correct but 60s answer may be unusable in a classroom setting; document tradeoffs explicitly.

9.14 Rubric weighting and stakeholder alignment

When combining dimensions, weights encode values: a medical-adjacent tutor might weight safety above fluency; a creative-writing aid might invert that. Document who chose weights and why—this is analogous to stakeholder elicitation in requirements engineering.

9.15 Grounding checks in user-facing systems

Force citations to chunk IDs, require abstain behavior when retrieval confidence is low, and log override events when humans correct the model. These design choices are evaluation interventions: they change both user trust and measurable error profiles.

9.16 Synthesis

Treat evaluation as continuous, multi-layered (automated screen + human judgment + user feedback), and honest about what is measured.

Code and Computational Examples

CE-1 Python: toy RAGAS-like faithfulness score (LLM-free heuristic)

Illustrative only: uses token overlap between answer sentences and context. Replace with entailment models or official RAGAS in real work.

[object Object], __future__ ,[object Object], annotations

,[object Object], re
,[object Object], dataclasses ,[object Object], dataclass


,[object Object], ,[object Object],(,[object Object],) -> ,[object Object],[,[object Object],]:
    ,[object Object], ,[object Object],(re.findall(,[object Object],, text.lower()))


,[object Object], ,[object Object],(,[object Object],) -> ,[object Object],[,[object Object],]:
    ,[object Object], [s.strip() ,[object Object], s ,[object Object], re.split(,[object Object],, text.strip()) ,[object Object], s.strip()]


,[object Object],
,[object Object], ,[object Object],:
    score: ,[object Object],
    unsupported_sentences: ,[object Object],[,[object Object],]


,[object Object], ,[object Object],(,[object Object],) -> ToyFaithfulnessResult:
    ctx_tokens = tokenize(context)
    unsupported: ,[object Object],[,[object Object],] = []
    ,[object Object], sent ,[object Object], sentence_split(answer):
        st = tokenize(sent)
        ,[object Object], ,[object Object], st:
            ,[object Object],
        overlap = ,[object Object],(st & ctx_tokens) / ,[object Object],(st)
        ,[object Object], overlap < ,[object Object],:
            unsupported.append(sent)
    total = ,[object Object],(sentence_split(answer)) ,[object Object], ,[object Object],
    score = ,[object Object], - (,[object Object],(unsupported) / total)
    ,[object Object], ToyFaithfulnessResult(score=,[object Object],(score, ,[object Object],), unsupported_sentences=unsupported)


ctx = ,[object Object],
ans = ,[object Object],
,[object Object],(toy_faithfulness(ans, ctx))

CE-2 Python: golden-set runner skeleton

[object Object], json
,[object Object], dataclasses ,[object Object], dataclass
,[object Object], typing ,[object Object], ,[object Object],, ,[object Object],


,[object Object],
,[object Object], ,[object Object],:
    ,[object Object],: ,[object Object],
    question: ,[object Object],
    must_contain: ,[object Object],[,[object Object],] | ,[object Object], = ,[object Object],
    must_not_contain: ,[object Object],[,[object Object],] | ,[object Object], = ,[object Object],


,[object Object], ,[object Object],(,[object Object],) -> ,[object Object],[,[object Object],, ,[object Object],]:
    results = []
    ,[object Object], it ,[object Object], items:
        ans = answer_fn(it.question).lower()
        ok = ,[object Object],
        reasons = []
        ,[object Object], it.must_contain:
            ,[object Object], frag ,[object Object], it.must_contain:
                ,[object Object], frag.lower() ,[object Object], ,[object Object], ans:
                    ok = ,[object Object],
                    reasons.append(,[object Object],)
        ,[object Object], it.must_not_contain:
            ,[object Object], frag ,[object Object], it.must_not_contain:
                ,[object Object], frag.lower() ,[object Object], ans:
                    ok = ,[object Object],
                    reasons.append(,[object Object],)
        results.append({,[object Object],: it.,[object Object],, ,[object Object],: ok, ,[object Object],: reasons})
    passed = ,[object Object],(r[,[object Object],] ,[object Object], r ,[object Object], results)
    ,[object Object], {,[object Object],: ,[object Object],(items), ,[object Object],: passed, ,[object Object],: results}


,[object Object], ,[object Object],(,[object Object],) -> ,[object Object],:
    ,[object Object], ,[object Object],


items = [
    GoldenItem(,[object Object],, ,[object Object],, must_contain=[,[object Object],, ,[object Object],]),
    GoldenItem(,[object Object],, ,[object Object],, must_not_contain=[,[object Object],]),
]
,[object Object],(json.dumps(run_golden(items, fake_answer), indent=,[object Object],))

CE-3 Python: inter-rater agreement stub

For formal work use sklearn.metrics.cohen_kappa_score.

[object Object], typing ,[object Object], ,[object Object],


,[object Object], ,[object Object],(,[object Object],) -> ,[object Object],:
    ,[object Object], ,[object Object],(a) == ,[object Object],(b) ,[object Object], a, ,[object Object],
    ,[object Object], ,[object Object],(x == y ,[object Object], x, y ,[object Object], ,[object Object],(a, b)) / ,[object Object],(a)


r1 = [,[object Object],, ,[object Object],, ,[object Object],, ,[object Object],, ,[object Object],]
r2 = [,[object Object],, ,[object Object],, ,[object Object],, ,[object Object],, ,[object Object],]
,[object Object],(,[object Object],, percent_agreement(r1, r2))

CE-4 YAML: human eval rubric item bank

[object Object], ,[object Object],
,[object Object], ,[object Object],[object Object],
,[object Object],
  ,[object Object], ,[object Object],
  ,[object Object], ,[object Object],
  ,[object Object], ,[object Object],
,[object Object],
  ,[object Object], ,[object Object], ,[object Object],
    ,[object Object], ,[object Object],
  ,[object Object], ,[object Object], ,[object Object],
    ,[object Object], ,[object Object],
  ,[object Object], ,[object Object], ,[object Object],
    ,[object Object], ,[object Object],
,[object Object], ,[object Object],

CE-5 Shell: optional RAGAS install note

[object Object],
python -m venv .venv
,[object Object],
pip install ,[object Object], datasets langchain-core

CE-6 Python: pairing outputs for blind preference (A/B)

[object Object], random
,[object Object], dataclasses ,[object Object], dataclass


,[object Object],
,[object Object], ,[object Object],:
    question: ,[object Object],
    text_a: ,[object Object],
    text_b: ,[object Object],
    correct_preference: ,[object Object], | ,[object Object],


,[object Object], ,[object Object],(,[object Object],) -> ,[object Object],[,[object Object],, ,[object Object],, ,[object Object],]:
    swap = random.random() < ,[object Object],
    first, second = (p.text_b, p.text_a) ,[object Object], swap ,[object Object], (p.text_a, p.text_b)
    ,[object Object], (,[object Object],, first, second)


sess = PairwiseSession(,[object Object],, ,[object Object],, ,[object Object],, ,[object Object],)
,[object Object],(present_blind(sess))

CE-7 Markdown: failure taxonomy template for reports

[object Object],

| Category | Count | Example | Mitigation |
|----------|-------|---------|------------|
| Retrieval miss | | | expand corpus / hybrid search |
| Grounding error | | | cite forcing / post-check |
| Overconfidence | | | calibrated prompting |
| Safety | | | policy layer |
| Format | | | JSON schema tool |

CE-8 Python: weighted rubric score (toy)

[object Object], dataclasses ,[object Object], dataclass


,[object Object],
,[object Object], ,[object Object],:
    name: ,[object Object],
    weight: ,[object Object],
    score_1_to_5: ,[object Object],


,[object Object], ,[object Object],(,[object Object],) -> ,[object Object],:
    ,[object Object], ,[object Object],(d.weight * (d.score_1_to_5 / ,[object Object],) ,[object Object], d ,[object Object], dims) / ,[object Object],(d.weight ,[object Object], d ,[object Object], dims)


dims = [
    DimScore(,[object Object],, ,[object Object],, ,[object Object],),
    DimScore(,[object Object],, ,[object Object],, ,[object Object],),
    DimScore(,[object Object],, ,[object Object],, ,[object Object],),
]
,[object Object],(,[object Object],, ,[object Object],(weighted_total(dims), ,[object Object],))

CE-9 Python: bootstrap confidence interval for pass rate (small-n friendly)

Use when reporting passed/total from golden runs without overclaiming precision.

[object Object], __future__ ,[object Object], annotations

,[object Object], random


,[object Object], ,[object Object],(,[object Object],) -> ,[object Object],[,[object Object],, ,[object Object],, ,[object Object],]:
    ,[object Object], ,[object Object], trials:
        ,[object Object], ,[object Object],, ,[object Object],, ,[object Object],
    rng = random.Random(seed)
    p_hat = ,[object Object],(trials) / ,[object Object],(trials)
    stats: ,[object Object],[,[object Object],] = []
    n = ,[object Object],(trials)
    ,[object Object], _ ,[object Object], ,[object Object],(n_boot):
        sample = [trials[rng.randrange(n)] ,[object Object], _ ,[object Object], ,[object Object],(n)]
        stats.append(,[object Object],(sample) / n)
    stats.sort()
    lo = stats[,[object Object],((alpha / ,[object Object],) * n_boot)]
    hi = stats[,[object Object],((,[object Object], - alpha / ,[object Object],) * n_boot)] - ,[object Object],
    ,[object Object], p_hat, lo, hi


flags = [,[object Object],, ,[object Object],, ,[object Object],, ,[object Object],, ,[object Object],, ,[object Object],, ,[object Object],, ,[object Object],]
p, lo, hi = bootstrap_pass_rate_ci(flags)
,[object Object],(,[object Object],)

CE-10 Python: stratified sample indices for human scoring

Given per-item difficulty tags, sample evenly so human raters see hard items, not only easy ones.

[object Object], __future__ ,[object Object], annotations

,[object Object], collections ,[object Object], defaultdict
,[object Object], random


,[object Object], ,[object Object],(,[object Object],) -> ,[object Object],[,[object Object],]:
    buckets: ,[object Object],[,[object Object],, ,[object Object],[,[object Object],]] = defaultdict(,[object Object],)
    ,[object Object], i, t ,[object Object], ,[object Object],(tags):
        buckets[t].append(i)
    rng = random.Random(seed)
    out: ,[object Object],[,[object Object],] = []
    ,[object Object], t, idxs ,[object Object], buckets.items():
        rng.shuffle(idxs)
        out.extend(idxs[:k_per_stratum])
    ,[object Object], ,[object Object],(out)


tags = [,[object Object],] * ,[object Object], + [,[object Object],] * ,[object Object], + [,[object Object],] * ,[object Object],
picked = stratified_indices(tags, k_per_stratum=,[object Object],)
,[object Object],(,[object Object],, ,[object Object],(picked), ,[object Object],, picked[:,[object Object],])

Practice Exercises

1. Claim extraction — Take a 400-word AI essay; highlight empirical claims; classify verified / unverified / wrong with sources.

2. Source ladder — One controversial topic; compare Wikipedia lead, news article, and primary study; document what each establishes.

3. Rubric calibration — Two raters independently score five answers on six dimensions; discuss disagreements using anchor examples from your YAML rubric.

4. Harm domain policy — Pick high-stakes advice domain; write mandatory human review SOP (1 page) with escalation triggers.

5. RAGAS vs. human — On a small RAG project, compute automated scores if tooling allows; human-score the same items; analyze correlation informally and note judge-model limitations.

6. Adversarial set authoring — Write ten “nasty but fair” questions your campus bot should handle; include two unanswerable-with-current-corpus items and expected behavior.

7. Metric critique essay — 600 words: strengths/weaknesses of LLM-as-judge for your discipline’s values (cite at least two secondary sources you opened).

8. Startup metric map — For a hypothetical MVP, define north-star, guardrails, and quality metrics; link each to an evaluation instrument (rubric row, golden check, or survey item).

Mini-Project: Evaluation Report for a Student-Built or Public AI Assistant

Context — Evaluate your own RAG mini-system, course chatbot, or a public assistant with Terms of Use respected.

Deliverables

1. Test protocol — 25–40 queries spanning easy/medium/hard; include adversarial and policy-edge cases; redact PII; pre-specify success criteria per item where possible.

2. Scoring — Combine automated checks (golden must_contain / must_not_contain) with human rubric on a stratified sample (≥15 responses); document rater training time.

3. Report (4 pages) — Executive summary; methods (instruments, sample, blinding); aggregate results with failure counts by category; recommended mitigations; limitations and threats to validity.

4. Appendix — Raw outputs or hashes; rater guidelines version; model and corpus date pins; optional RAGAS configuration snippet.

5. Ethics — If human subjects (student raters beyond normal class activity), follow institutional guidance; otherwise state classroom evaluation scope.

Rubric (summative fragment)

Key Takeaways

Academic integrity and scientific norms

• Automated metrics are assistive, not authoritative, unless validated for your setting.
• Report what you measured, who rated, blinding if any, and what you did not test.
• When you report a single accuracy or faithfulness figure, ask what would invalidate it: judge change, corpus drift, or sampling bias are all common nullifiers.
• Hallucination-free is not a binary claim; use rates, confidence intervals, and failure taxonomies instead of slogans.

Practical workflow

• Maintain a golden set as living regression suite; rerun after prompt, retriever, or corpus changes.
• Pair RAGAS-style checks with human judgment on nuance and safety.
• Version-control rubric text alongside code; eval is software + social process.
• For capstone demos, precompute offline fallbacks when APIs fail—evaluation includes reliability under outage.

Conceptual anchors

• Fluency is cheap; verification is expensive—budget accordingly.
• Bias is both data and process—who annotates, with what guidelines, under what time pressure?
• Construct validity first: ensure your rubric dimensions measure what stakeholders care about, not what is easy to score.
• Entailment ≠ understanding; chain-of-thought judges can still miss subtle policy contradictions.

Startup link

• Tie evaluation to user outcomes and risk; ship small with explicit limitations and guardrail metrics.
• Investors and customers increasingly ask how you know the system works—your golden set and human sample are due diligence artifacts.

Integrity in classroom studies

• If classmates rate each other’s outputs, declare potential conflicts and consider blinding or external raters for high-stakes grades.
• Do not p-hack prompts: log each template change and avoid selective reporting of cherry-picked queries.

Further resources

• RAGAS documentation and associated papers (verify current version notes).
• Model Cards (Mitchell et al.); Datasheets for Datasets (Gebru et al.).
• NIST AI RMF; surveys on fairness and human–AI evaluation in HCI.
• ACL / NeurIPS tutorials on evaluation of generated text (update annually; field moves quickly).
• Module 03 (writing integrity), 11 (team eval of software), 12 (capstone evaluation).

Closing reminder

Evaluation is continuous. If you change the model, corpus, or prompt, re-run your harness and update the limitations section—just as you would for any empirical instrument.

References and Cross-Modules

• Pair with Module 08 (RAG/memory) for technical depth.
• Export golden items to your capstone repo as eval/golden.yaml.