Foundations · Module 4

Responsible AI basics and limitations

AI systems can cause harm even when everybody is trying to do the right thing.

1h 4 outcomes AI Foundations

Previously

Supervised and unsupervised learning

When we say a model learns, we mean it changes its internal settings so it can make better guesses.

This module

Responsible AI basics and limitations

AI systems can cause harm even when everybody is trying to do the right thing.

Next

AI Foundations practice test

Test recall and judgement against the governed stage question bank before you move on.

Progress

Mark this module complete when you can explain it without rereading every paragraph.

Why this matters

Glossary Tip.

What you will be able to do

  • 1 Explain responsible ai basics and limitations in your own words and apply it to a realistic scenario.
  • 2 Responsibility is not a document. It is an operating loop with measurement and correction.
  • 3 Check the assumption "Someone owns the decision" and explain what changes if it is false.
  • 4 Check the assumption "The system can pause" and explain what changes if it is false.

Before you begin

  • No previous technical background required
  • Read the section explanation before using tools

Common ways people get this wrong

  • No monitoring. Without monitoring, you only learn when a user complains or a regulator asks. That is too late.
  • Policy lives only in words. A policy that is not enforced by the system is a wish. Guardrails must be in the request path.

Main idea at a glance

The AI lifecycle and where risks appear

Risk shows up at every step. Oversight wraps the loop.

Stage 1

Collect data

I gather examples, labels, and context. The data I choose shapes what the model can learn.

I think data collection is not technical work. It is a values choice about who is represented and what is measured.

AI systems can cause harm even when everybody is trying to do the right thing. The harm is not always dramatic. Sometimes it is quiet and personal. Someone is incorrectly flagged as suspicious. Someone does not get offered an opportunity. Someone is pushed into a bubble of content that makes them angrier and more certain.

The core reason is simple. Models learn patterns, not truth. They learn what tends to follow what in the data they were given. They do not know what is fair. They do not know what is lawful. They do not know what is kind. They only know what was rewarded during training.

Interactive lab

Glossary Tip

This module includes an interactive practice component. Open the deeper tool or workspace step when you want to test the idea rather than only read it.

Bias can enter through the data, through the labels, and through the choices we make. If your training data under represents some faces, facial recognition errors show up first in those groups. If your labels reflect past human decisions, you teach the model those decisions as if they were objective truth. If you optimise only for speed, you often trade away care.

In a real system, a “small” bias can become a big harm because automation runs every day. Imagine an AI triage tool that consistently underestimates risk for one group. Even if the average performance looks fine, the harm concentrates on real people.

In practice, the fix is not only better metrics. The fix is process: clear accountability, human review for high impact cases, and monitoring that triggers action when outcomes drift or complaints rise.

Interactive lab

Glossary Tip

This module includes an interactive practice component. Open the deeper tool or workspace step when you want to test the idea rather than only read it.

Fairness is not a single magic number. It is a set of decisions. What harm do we want to prevent. Which groups matter in this context. What trade offs are acceptable. It belongs with humans, not only with metrics.

Interactive lab

Glossary Tip

This module includes an interactive practice component. Open the deeper tool or workspace step when you want to test the idea rather than only read it.

Explainability matters most when decisions affect real lives. An automated decision system that cannot explain itself becomes hard to challenge. People then learn to distrust the whole process, or worse, they stop trying to challenge it at all.

Bigger models do not automatically mean better or safer. They can be more capable and still be more confusing. They can also be more expensive to run, which encourages shortcuts. If it costs too much to monitor and evaluate, teams quietly stop doing it.

What this optimises for

What this approach optimises for

  1. Automation speed and consistency for low-risk tasks

    Use models where quick pattern matching reduces routine workload safely.

  2. Scaled assistance with human context

    Use model outputs to support, not replace, human judgement in nuanced decisions.

What this makes harder

What this makes harder

  1. Accountability when oversight is vague

    Without named owners, harm appears but responsibility is delayed.

  2. Safe rollback when drift or failure appears

    If rollback is not pre-planned, incidents expand before containment starts.

AI confidence is not the same as correctness. A model can be very confident and still be wrong. This shows up most painfully when people over trust AI answers. A model that sounds fluent can still invent details. It is good at sounding plausible. It is not automatically good at being true.

Interactive lab

Glossary Tip

This module includes an interactive practice component. Open the deeper tool or workspace step when you want to test the idea rather than only read it.

This is why human judgement must stay in control. Use AI to help you think, draft, or explore. Do not let it quietly become the decision maker for high impact outcomes. If a system can deny a benefit, flag a person, or change a life, it needs clear oversight and a way to appeal.

Interactive lab

Glossary Tip

This module includes an interactive practice component. Open the deeper tool or workspace step when you want to test the idea rather than only read it.

Drift is not rare. It is normal. People change behaviour. Fraud patterns adapt. Language evolves. Even a simple recommendation system can drift into a bubble because the system is changing the data it then learns from. You cannot evaluate once and declare victory.

Worked example. Hallucination meets a real policy decision

Worked example. Hallucination meets a real policy decision

Imagine a team uses a chat assistant to draft a customer policy response. The model writes something confident, polite, and wrong. It invents a rule that does not exist. A tired human skims it, copies it into an email, and the customer now has written confirmation of a policy the company does not have.

This is not a “prompt engineering” problem first. This is a governance problem. Who owns the output. What gets checked. What is the escalation path when the model produces something risky.

Common mistakes in responsible AI

Responsible AI mistakes that cause incidents

  1. Declaring fairness from one metric snapshot

    Fairness requires ongoing checks across groups, scenarios, and changing conditions.

  2. Treating confidence as correctness

    Confidence signals model belief, not factual truth or policy safety.

  3. Shipping without monitoring

    Unobserved systems drift silently until users or regulators surface harm.

  4. No appeal path in high-impact decisions

    Without challenge and review routes, automation errors become entrenched.

Verification. A lightweight governance checklist you can actually run

Lightweight governance checklist

  1. Define the use case and target harm

    State what risk you are reducing and what new risk the model could introduce.

  2. Assign approval and accountability

    Name who can release changes and who owns outcomes after release.

  3. Set the evidence trail

    Keep evaluation, monitoring, and user-feedback evidence ready for review.

  4. Write a practical rollback plan

    Document exact steps to pause, revert, and recover if harms appear.

After this section you should be able to

Section outcomes

  1. Explain why models do not understand intent or truth

    Describe why fluent output can still be wrong and operationally risky.

  2. Explain why evaluation and monitoring are continuous

    Show how drift and behaviour shifts make one-off validation insufficient.

  3. Explain the trade-off between automation and oversight

    Design a clear human-control path for high-impact decisions.

You now have the foundations to talk clearly about data, learning, and limits. Intermediate will build on this with evaluation, deployment, and system thinking. Foundations are about how you think, not which tool you use.

Mental model

Responsible use is a loop

Responsibility is not a document. It is an operating loop with measurement and correction.

  1. 1

    Measure outcomes

  2. 2

    Decide thresholds

  3. 3

    Deploy with guardrails

  4. 4

    Review and improve

Assumptions to keep in mind

  • Someone owns the decision. If nobody owns the trade-off, the system becomes a default that nobody can explain. Ownership is part of accountability.
  • The system can pause. When quality drops or misuse appears, the safe move is to degrade gracefully. Pausing is a feature, not a failure.

Failure modes to notice

  • No monitoring. Without monitoring, you only learn when a user complains or a regulator asks. That is too late.
  • Policy lives only in words. A policy that is not enforced by the system is a wish. Guardrails must be in the request path.

Check yourself

Quick check. Responsible AI basics and limitations

0 of 12 opened

Scenario. A well-meaning automation quietly denies opportunities to a subset of people. Why can AI cause harm even when well intentioned

Models learn patterns from data and can scale mistakes, bias, and bad assumptions.

Scenario. A model repeats what historically happened, even if it was unfair. What does it mean that models learn patterns, not truth

They learn statistical regularities, not what is correct, fair, or lawful.

Scenario. Name two places bias can enter an AI system

Through data coverage, labels, or the assumptions and objectives chosen.

Scenario. Facial recognition fails more often for some groups. Why is that a fairness issue

Error rates can be higher for under-represented groups, causing unequal harm.

Scenario. You upgrade to a bigger model and stop doing monitoring because it is expensive. Why is bigger not automatically safer

More capability can come with more opacity, cost, and operational shortcuts.

Scenario. The model sounds certain but is wrong. Why is AI confidence not the same as correctness

A model can be confident and still be wrong because confidence is not a truth check.

Scenario. The system invents a policy that does not exist and presents it as fact. What is that called

A hallucination. A confident output that is not grounded in the input or reality.

Scenario. Performance drops over three months as users change behaviour. What is model drift

When data or behaviour changes over time so the model performs worse than before.

Scenario. When must humans stay in control

When decisions are high impact and affect rights, safety, or access to opportunities.

Scenario. You want a safe default for high impact decisions. What should it be

Human review with clear accountability, appeal paths, and monitoring.

Scenario. An incident happens and you need to explain what the model was for and where it fails. What is one governance artefact you should keep

A model card, decision log, or documented risk and monitoring plan.

Scenario. Why is it important to define who is accountable when an AI system causes harm

Automation can obscure responsibility. Clear accountability ensures someone owns outcomes and can be held responsible for decisions and failures.

Artefact and reflection

Artefact

A short module note with one key definition and one practical example

Reflection

Where in your work would explain responsible ai basics and limitations in your own words and apply it to a realistic scenario. change a decision, and what evidence would make you trust that change?

Optional practice

A small visual practice tool to show how models learn by small steps, not sudden understanding. Useful for intuition about optimisation and failure modes.

Also in this module

Spot AI risks in everyday scenarios

Read short AI stories and practise identifying bias, safety risks and over trust in model outputs.