Applied · Stage test

AI Intermediate stage test

No governed timed route exists for this stage yet, so this page gives you an honest untimed stage-end check built from the published bank.

Format Untimed self-check
Questions 12
Best time to use it After the stage modules and practice

Question 1

Scenario: Your RAG bot answers confidently but cites the wrong paragraph. What do you fix first?

  1. Retrieval quality and chunking/indexing
  2. Model size
  3. Font size
  4. GPU driver
Reveal answer

Correct answer: Retrieval quality and chunking/indexing

Question 2

Scenario: A prompt change breaks a workflow. What engineering practice should exist?

  1. Treat prompts like interfaces: version, test, review
  2. Never change prompts
  3. Only use longer prompts
  4. Disable monitoring
Reveal answer

Correct answer: Treat prompts like interfaces: version, test, review

Question 3

Scenario: The model performs well overall but fails for one user segment. What catches this?

  1. Slice testing by segment and scenario
  2. Only aggregate accuracy
  3. Add more emojis
  4. Use a bigger context window
Reveal answer

Correct answer: Slice testing by segment and scenario

Question 4

Scenario: You must choose a threshold. What should it be based on?

  1. Cost of false positives vs false negatives and review capacity
  2. The highest possible number
  3. What looks good in a demo
  4. The model name
Reveal answer

Correct answer: Cost of false positives vs false negatives and review capacity

Question 5

Scenario: You add lots of context and answer quality drops. What is the most likely reason?

  1. Too much context dilutes key facts and increases distraction
  2. More context always improves accuracy
  3. The GPU is out of memory
  4. The prompt became encrypted
Reveal answer

Correct answer: Too much context dilutes key facts and increases distraction

Question 6

Scenario: Users try to trick the system by changing wording until it misbehaves. What is the right framing?

  1. Adversarial behaviour / distribution shift that needs monitoring and guardrails
  2. A harmless UX issue only
  3. A database indexing problem
  4. A compiler bug
Reveal answer

Correct answer: Adversarial behaviour / distribution shift that needs monitoring and guardrails

Question 7

Scenario: A team wants the assistant to answer policy questions using current governed documents. What should you try before fine-tuning?

  1. Retrieval augmented generation with permissions, traceability, and cited sources
  2. Full model retraining immediately
  3. Random prompt changes with no retrieval layer
  4. Disable citations so the answer sounds smoother
Reveal answer

Correct answer: Retrieval augmented generation with permissions, traceability, and cited sources

Question 8

Scenario: Retrieval returns the right chunk, but the model still answers wrongly. What do you add?

  1. Citations and answer-grounding checks (and refuse when evidence is weak)
  2. More temperature
  3. A bigger logo
  4. Disable tests
Reveal answer

Correct answer: Citations and answer-grounding checks (and refuse when evidence is weak)

Question 9

Which evaluation approach is most defensible for a user-facing assistant?

  1. A scenario set with slice tests and acceptance criteria linked to harms
  2. One overall benchmark score
  3. Only speed tests
  4. Only subjective demo feedback
Reveal answer

Correct answer: A scenario set with slice tests and acceptance criteria linked to harms

Question 10

Scenario: Users report 'it was fine yesterday'. What do you check first?

  1. Versioned changes (prompt, retrieval index, tools) and correlated failure spikes
  2. Only model temperature
  3. Only the marketing page
  4. Only the GPU type
Reveal answer

Correct answer: Versioned changes (prompt, retrieval index, tools) and correlated failure spikes

Question 11

Scenario: Facts change every week and the answer must stay current. Which deployment pattern is the best starting point?

  1. A retrieval-augmented system grounded in current documents
  2. A frozen model with no retrieval layer
  3. Batch predictions with no source refresh
  4. Manual copy and paste into the prompt every day
Reveal answer

Correct answer: A retrieval-augmented system grounded in current documents

Question 12

Scenario: Your RAG system retrieves contradictory policies. What should the assistant do?

  1. Surface the conflict, cite both sources, and ask a clarifying question or escalate
  2. Pick one at random to keep flowing
  3. Hide citations and answer confidently
  4. Ignore retrieval and answer from memory
Reveal answer

Correct answer: Surface the conflict, cite both sources, and ask a clarifying question or escalate

Back to AI