# Handbook answer test: completed key and blank matrix

Use with the [fictional handbook corpus](/learn/resources/handbook-test-corpus.md). This is an illustrative answer key, not a report of a working search feature. All actual app outcomes below are **Not tested**.

## Set the run boundary

- Choose the staff actor and load R1 exactly. Keep the manager seed restricted in your app, even though this downloadable exercise contains only fictional text.
- Choose one route before testing: **search** (the exact card question; inspect at most the first five displayed results) or **browse** (a recorded path with no more than three navigation steps). Record the route for the run. Browse success does not establish search success; record a separate run to claim both.
- Open the candidate source. Record its document ID, version, exact section and supporting words. A title, result count or snippet alone does not pass.
- Pass a supported-answer case only when the chosen route reaches the current permitted passage within its boundary. Fail if it leads to an archive as the current answer, a forbidden page or a passage that does not support the answer. A clear version/revision indication or another verifiable current-source marker is needed for the evidence.
- For Q4/Q5, verify no permitted current source supports the requested answer and use the D04 question-owner route. Do not invent hours or expose a private title, snippet or body. Opening D04 is a fallback, not evidence of the missing answer. Use the full private-access worksheet for broader permission testing.
- If you have not performed the action, record **Not tested**. A fixture expectation is not a pass. A blocked or unavailable test stays untested with its reason; a performed test that breaks the contract fails.

## Six scenarios

| Case | Actor | Exact question | What this checks |
| --- | --- | --- | --- |
| Q1 | staff | Where do I collect print samples? | Direct wording |
| Q2 | staff | I need to pick up samples. Where do I go? | Alternate wording |
| Q3 | staff | Is back-door pickup still used? | Old wording |
| Q4 | staff | Can we collect samples on Sunday? | Unsupported fact |
| Q5 | staff | What are the supplier exception rules? | No permitted supporting source |
| Q6 | staff | Where do I collect print samples? | Repeat Q1 after the approved location change |

Q6 is the update repetition of Q1, not a sixth unique search question. Run Q1–Q5 on R1; then load R2 and run Q6, Q2, Q3, Q4 and Q5. That is ten run rows across five distinct question texts. Keep both revisions' results.

## Completed expected-result key

The source locations are exact fixture identifiers. Replace them with actual app links in your observed run, preserving the expected document/version/section.

| Run | Expected source section or fallback | Expected supported answer | Actual app outcome |
| --- | --- | --- | --- |
| R1-Q1 | D01 v1 / collection-location | Front desk | Not tested |
| R1-Q2 | D01 v1 / collection-location | Front desk | Not tested |
| R1-Q3 | D01 v1 / replaces-old-instructions | No. Front desk | Not tested |
| R1-Q4 | No permitted supporting source; fallback D04 v1 / question-owner | No supported answer. Ask the studio coordinator. | Not tested |
| R1-Q5 | No permitted supporting source; fallback D04 v1 / question-owner | No supported answer. Ask the studio coordinator. | Not tested |
| R2-Q6 | D01 v2 / collection-location | Dispatch counter | Not tested |
| R2-Q2 | D01 v2 / collection-location | Dispatch counter | Not tested |
| R2-Q3 | D01 v2 / replaces-old-instructions | No. Dispatch counter | Not tested |
| R2-Q4 | No permitted supporting source; fallback D04 v1 / question-owner | No supported answer. Ask the studio coordinator. | Not tested |
| R2-Q5 | No permitted supporting source; fallback D04 v1 / question-owner | No supported answer. Ask the studio coordinator. | Not tested |

Q3 must explicitly reject back-door pickup using the current D01 passage. For Q5, the staff-facing result must not disclose the manager-only seed's title, snippet, body or existence. A manager seeing that seed is outside this staff-only question run and does not satisfy it.

## Blank run matrix

Copy this block for each corpus revision and chosen route. If you use your own documents, freeze their IDs, versions and answer key before observing results; record your assumptions instead of borrowing this fictional policy.

Corpus revision: ______  Actor/account role: ______  Date: ______

Route: search / browse (choose one). Exact search text or browse path: ______

| Run/case | Expected permitted source and section, or no supported answer | Actual opened source/version/section | Actual supporting words or visible fallback | Pass / Fail / Not tested | Evidence / repair owner |
| --- | --- | --- | --- | --- | --- |
| ______ | ______ | ______ | ______ | Not tested | ______ |
| ______ | ______ | ______ | ______ | Not tested | ______ |
| ______ | ______ | ______ | ______ | Not tested | ______ |
| ______ | ______ | ______ | ______ | Not tested | ______ |
| ______ | ______ | ______ | ______ | Not tested | ______ |

## Update and decide

After an approved D01 update, R2 uses Dispatch counter. Q6 (the rerun of Q1), Q2 and Q3 need the new current passage. Rerun Q4 and Q5 too: their lack of a permitted answer stays unchanged. Keep the original R1 observations and separate repair/retest rows.

If any performed case fails, repair its source, route or access behavior and rerun the affected cases before relying on that workflow. If any case remains untested, retain that gap. Passing this small set supports only this corpus, actor and route; it is not a search-quality percentage, security certification or proof of behavior with all staff documents.

Turn the failures into a focused brief at [Overskill's brief builder](https://www.overskill.com/learn#brief-builder). Review the generated brief and edit its input fields, then choose Start build for a new prototype; for an existing app, use the same failed cases to request a focused repair and retest it.