Phases and Pages¶

Consent, instructions, training, surveys, and the order they run in. It ships with the potato-tasks skill:

/plugin marketplace add davidjurgens/potato-skill
/plugin install potato-tasks@potato

Everything a researcher describes as "they should see X first" and "at the end I want to ask them Y" lives here. None of it is in the generated key reference: phases and surveyflow are documented as object with no sub-keys, and potato validate --strict does not check inside either of them. Every fact below was established by running the server and reading the startup log.

The six phase types¶

consent | instructions | training | annotation | poststudy | prestudy

That is the complete list. Anything else fails at boot with Failed to load phase 'x': Unknown phase: x — and potato validate --strict passes it. A config naming fourteen invented phase types validates clean and then loses fourteen phases at startup.

There is no survey phase, no demographics, no debrief, no end, no done. A post-study questionnaire is a poststudy phase whose page happens to contain survey questions. Screening before the task is prestudy.

order is honoured as written, including where you put prestudy. Walked in a browser, this order

phases:
  order: [consent, prestudy, instructions, training, annotation, poststudy]

serves consent, then the screening page, then the instructions page, then the training questions, in exactly that sequence. A phase with required questions gates the next one — the annotator cannot advance past it unanswered — which is what "only people who answer the screening questions get through" means in practice. Phase pages record; they do not route. There is no mechanism for screening someone out on the basis of what they answered, and no documented way to feed a survey answer into assignment. The nearest thing is category_assignment, which routes by item category and can gate on a qualification earned in training, not on a survey response. If a researcher says "only screen-reader users get through", tell them the phase collects the answer and someone has to act on it — do not imply the phase enforces it.

Note also that the top-level prestudy: key is dead code — the key documentation says so outright ("the only code that reads it is an unreferenced method"). Screening is a prestudy phase, not the prestudy block.

Two ways to write it¶

Mapping form — an order list plus one entry per phase, keyed by name:

phases:
  order: [consent, instructions, training, annotation, poststudy]
  consent:
    type: consent
    title: Consent to take part
    file: surveyflow/consent.jsonl
  instructions:
    type: instructions
    title: About this study
    file: surveyflow/instructions.jsonl
  training:
    type: training
    title: Practice
  annotation:
    type: annotation
  poststudy:
    type: poststudy
    title: Your own news habits
    file: surveyflow/news_habits.jsonl

List form — the same thing with name inside each entry, no order:

phases:
  - {name: consent, type: consent, title: Consent, file: surveyflow/consent.jsonl}
  - {name: annotation, type: annotation}
  - {name: poststudy, type: poststudy, title: Wrap up, file: surveyflow/end.jsonl}

Both work. Mapping form reads better once any phase has more than three keys.

Naming a phase in order without defining it takes the study down, if the annotator would reach it before annotation. The log says otherwise:

WARNING: Phase 'consent' in order but not defined in phases config, skipping

It does not skip. The phase is dropped from the page map and left in the annotator's sequence, so the first request for it raises KeyError and every page — including / — is a 500. validate --strict passes the config and the boot is clean, so the first sign of it is a browser.

An undefined phase after annotation is harmless: a study with order: [annotation, poststudy] and no poststudy block annotates all its items and reaches the completion page. It is only fatal in front of the annotation phase, which in practice means consent, instructions and prestudy.

Define every phase you name. If you want a phase gone, take it out of order.

The annotator never sees it and the config still validates. If a phase you configured is not appearing, this is the first line to grep for.

Every non-annotation phase needs a page¶

ERROR: Phase consent requires 'instrument', 'instruments', or 'file' to specify its annotation schemes

file: — a path to a .jsonl page (below). Use it for anything you wrote yourself.
instrument: / instruments: — take a string (or list of strings) naming a built-in instrument from the library below. Handing instrument a list of scheme dicts fails with unhashable type: 'list', which is the error you get if you assume it is an inline scheme list.

The two combine: instrument questions are laid out first, then the file's. Note that instrument and instruments do not — the singular wins and the list is ignored, silently. Use instruments: for one or many and forget the singular exists.

The instrument library¶

Potato ships 55 validated questionnaires, so a post-study survey that asks for a standard measure should almost never be hand-written into a page file. Naming one writes the whole questionnaire, wording and response options included, into the phase.

phases:
  order: [annotation, poststudy]
  poststudy:
    type: poststudy
    instruments: [tipi, who-5]

That boots to a page of fifteen questions — ten from tipi, five from who-5 — with no page file anywhere. Eight categories:

Category	Examples
`personality`	`bfi-2` (60 items), `tipi` (10), `nfc`, `gse`
`mental_health`	`phq-9`, `gad-7`, `k6`, `ces-d`, `pss-10`, `who-5`
`affect`	`panas`, `iri`
`social`	`rse`, `ucla-loneliness`, `mos-ss`, `macarthur-ladder`
`attitudes`	`mfq`, `sdo-7`, `rwa`, `trust-ess`, `political-efficacy`
`response_style`	`mc-sds` — social desirability, for detecting it in the rest
`short_forms`	`bfi-10`, `mini-ipip`, `phq-2`, `gad-2`, `pss-4`
`demographics`	batteries lifted from ANES, GSS, ESS, WVS, MIDUS, IPUMS and the ACS

The registry is the authority, and it carries the item count and a source URL for each one:

from potato.survey_instruments import list_instruments, get_categories
get_categories()                      # category -> instrument ids
list_instruments("mental_health")     # id, name, items, description, url

Two things to tell a researcher before they pick one. Long instruments are long: bfi-2 is 60 questions and ztpi is 56, a bigger ask than the annotation task in most studies. The short_forms category exists for that reason. And a fifteen-question page runs about two and a half screens, so the submit button ends up below the fold. check_ui.py --config config.yaml --phase poststudy reports the exact position — use --phase rather than the ordinary walk, which starts at the landing page and never gets past annotation on a corpus of any size.

psychometrics is a different thing entirely despite the neighbouring name: it fits an item-response model over the annotation labels to estimate annotator ability and item difficulty. It has nothing to do with questionnaires.

training and annotation need no page: training reads training.data_file and annotation reads annotation_schemes.

The page file format¶

A phase page is JSONL, one annotation scheme per line, using exactly the scheme syntax from annotation-types.md:

{"annotation_type":"pure_display","name":"consent_text","allow_html":true,"description":"<h3>Consent</h3><p>…</p>"}
{"annotation_type":"radio","name":"consent_agree","description":"Do you agree to take part?","labels":["I agree","I do not agree"],"humanize_labels":false,"label_requirement":{"required":true}}

Notes that cost time to find:

Prose goes in a pure_display scheme, in its description, with allow_html: true. There is no text: key on a phase that renders anything — title renders, text is ignored.
pure_display logs WARNING: No labels provided for pure display content on every render. It is noise; the content renders.
Phase-page prose lands inside a <legend>, which the theme sets to font-weight: 600. A page of it reads as entirely bold. Fix with base_css:

form.pure-display legend { font-weight: 400; }
form.pure-display legend b, form.pure-display legend h3 { font-weight: 650; }

base_css is inlined into a <style> tag rather than served as a file, so the path only has to be readable at boot. - Anything with label_requirement: {required: true} is enforced on the phase page the same way it is on the annotation page. - Keys inside these lines are not validated at all. A misspelled annotation_type here fails at boot; a misspelled option key is ignored.

Phase-level schemes replace the top-level list, and you cannot mix¶

phases:
  poststudy:
    type: poststudy
    annotation_schemes: [...]     # with top-level annotation_schemes present

fails validation:

Configuration has both top-level 'annotation_schemes' and phase-level
'annotation_schemes' in: poststudy. Use only one location to avoid confusion.

This is all-or-nothing across the config. Defining a survey inline on one phase forces every phase, including annotation, to carry its own scheme list. For a normal study — one question set for the items, different questions on the survey pages — use file: for the phase pages and keep the top-level annotation_schemes for annotation. Reach for phase-level schemes only when you genuinely want different questions per phase and are willing to write them all out.

`annotation_instructions` is not the instructions phase¶

annotation_instructions is the collapsible banner at the top of every page, including consent, training and post-study. It is a standing reminder, not a page.

Two consequences:

Long label definitions in it push the item and the questions below the fold on every single item. Put the definitions on an instructions phase and keep the banner to a few lines.
It appears above the consent text, which looks like a mistake to an annotator reading it for the first time. Word it so it survives that context.

Rendering a phase page¶

potato preview --phase X --screenshot out.png renders the page for phase X. Two things about it:

It accepts only five names. --phase prestudy is refused:

Error: phase must be one of ('consent', 'instructions', 'training', 'annotation', 'poststudy'), got 'nonsense'

prestudy is a valid phase for the server and not for the preview CLI, so a screening page cannot be rendered this way. Walk it in a browser instead.

It exits 1 on a healthy phase page. Every phase page fires Potato's own /api/current_instance 404 and two [SpanManager] console errors, and --screenshot counts those as things to report:

$ potato preview config.yaml --phase consent --screenshot consent.png ; echo $?
console.error: [SpanManager] Failed to get server instance ID during initialization
HTTP 404: http://127.0.0.1:9080/api/current_instance
1

$ potato preview config.yaml --screenshot annotation.png ; echo $?
Rendered cleanly — no browser errors.
0

Both configs are fine. Do not gate on the exit code for a phase page, and do not go looking for the bug — look at the PNG and check the error list contains nothing beyond those three lines.

Only pass a phase the task actually has, or every route redirects toward a page that does not exist and the render fails with a redirect loop.

`surveyflow` versus `phases`¶

surveyflow is the older way to attach pages before and after annotation:

surveyflow:
  "on": true
  order: [pre_annotation, post_annotation]
  pre_annotation: [surveyflow/consent.jsonl, surveyflow/instructions.jsonl]
  post_annotation: [surveyflow/news_habits.jsonl]
  sample_status: []

Quote the "on" key. Bare on: is a YAML boolean, and the loader warns:

`on:` at surveyflow was read as the boolean True … quote the key ("on":) to be explicit.

Prefer phases. It names the phase types explicitly, it controls order in one place, and it is what the training phase and --debug-phase key off. A directory called surveyflow/ holding the page files is a convention worth keeping even when the surveyflow: block is absent — that is where the pages in the example above live.

Checking the workflow actually assembled¶

Validation cannot see any of this. Boot the server and read the log:

grep -E "phase|Loaded [0-9]+ (training|attention|gold)" server.log

A healthy boot is silent about phases. Any of these means a phase is missing from the annotator's walk:

Line	Meaning
`Phase 'x' in order but not defined`	You listed it and never described it
`Failed to load phase 'x': Unknown phase: y`	`type:` is not one of the six
`Phase x requires 'instrument', 'instruments', or 'file'`	No page attached
`Failed to load phase 'x': unhashable type: 'list'`	`instrument:` got a list of schemes

Then walk it in a browser. A phase can load cleanly and still be unreachable because of what precedes it. See running-a-task.md.