Practice and strategy · Stage test

AI Advanced stage test

No governed timed route exists for this stage yet, so this page gives you an honest untimed stage-end check built from the published bank.

Format Untimed self-check
Questions 12
Best time to use it After the stage modules and practice

Question 1

Scenario: An agent can call tools. What control limits blast radius fastest?

  1. Scoped permissions + logging + stop conditions
  2. Give it admin access
  3. Disable logs
  4. Let it learn in production
Reveal answer

Correct answer: Scoped permissions + logging + stop conditions

Question 2

Why must LLM systems log tool use?

  1. Tool misuse is a primary failure mode
  2. It makes the UI prettier
  3. It reduces latency automatically
  4. It replaces governance
Reveal answer

Correct answer: Tool misuse is a primary failure mode

Question 3

Governance is most defensibly framed as:

  1. Paperwork to satisfy compliance
  2. Decision quality under uncertainty with evidence and review triggers
  3. A way to slow teams down
  4. Something you do after an incident only
Reveal answer

Correct answer: Decision quality under uncertainty with evidence and review triggers

Question 4

Scenario: You must store evidence but protect privacy. What is the defensible approach?

  1. Store the minimum needed with retention, access controls, and clear purpose
  2. Store everything forever just in case
  3. Store nothing at all
  4. Store only screenshots
Reveal answer

Correct answer: Store the minimum needed with retention, access controls, and clear purpose

Question 5

A system passes a benchmark but fails in production. What was tested incorrectly?

  1. The model alone, not the socio-technical system (pipelines, UI, humans, incentives)
  2. Only the UI, not the model
  3. Only the GPU, not the code
  4. Only the database, not the network
Reveal answer

Correct answer: The model alone, not the socio-technical system (pipelines, UI, humans, incentives)

Question 6

Scenario: An agent starts looping tool calls. What is the safest immediate control?

  1. A kill switch / stop condition and scoped permissions
  2. Increase temperature
  3. Give it more tools
  4. Turn off logging
Reveal answer

Correct answer: A kill switch / stop condition and scoped permissions

Question 7

What makes an AI incident response plan credible?

  1. Clear triggers, owners, evidence to collect, and rollback steps
  2. A vague statement that safety matters
  3. Only model retraining
  4. Only a policy document
Reveal answer

Correct answer: Clear triggers, owners, evidence to collect, and rollback steps

Question 8

Which signal is most useful for detecting retrieval problems early?

  1. Evidence mismatch rate (answers not supported by cited chunks)
  2. GPU utilisation
  3. Number of UI clicks
  4. Number of features in the index
Reveal answer

Correct answer: Evidence mismatch rate (answers not supported by cited chunks)

Question 9

Scenario: Leadership wants 'one model for everything'. What is the first system-level risk to raise?

  1. Different domains have different harms/constraints; one-size-fits-all increases systemic risk
  2. It will always reduce incidents
  3. It guarantees faster delivery
  4. It eliminates the need for evaluation
Reveal answer

Correct answer: Different domains have different harms/constraints; one-size-fits-all increases systemic risk

Question 10

Scenario: You need to prove safe operation to an auditor. What evidence is most defensible?

  1. Versioned policies, monitoring dashboards, incident logs, and review triggers
  2. A promise that the model is safe
  3. Only a benchmark score
  4. Only screenshots of the UI
Reveal answer

Correct answer: Versioned policies, monitoring dashboards, incident logs, and review triggers

Question 11

Scenario: Product wants to disable refusal behaviour because it 'hurts conversion'. What is the governance response?

  1. Treat it as a risk acceptance decision: document harms, evidence, owner, and review trigger
  2. Let sales decide without documentation
  3. Disable monitoring to avoid scrutiny
  4. Change nothing and hope for the best
Reveal answer

Correct answer: Treat it as a risk acceptance decision: document harms, evidence, owner, and review trigger

Question 12

Scenario: An agent suggests a plan. What is the most defensible way to execute it?

  1. Ask the agent to propose steps, but require explicit confirmation for each high-impact tool action
  2. Let the agent run end-to-end without review
  3. Disable logs to reduce cost
  4. Give the agent admin permissions to avoid blockers
Reveal answer

Correct answer: Ask the agent to propose steps, but require explicit confirmation for each high-impact tool action

Back to AI