# When a Task Misbehaves

Symptoms that validate clean and then do something else. It ships with the `potato-skill` skill:

```
/plugin marketplace add davidjurgens/potato-skill
/plugin install potato-skill@potato
```

---

Ordered by where you are when it happens. Everything here was hit for real on
potato-annotation 2.7.0.

## The command

| Symptom | Cause | Fix |
|---|---|---|
| `potato --help` seems not to list `validate` or `preview` | They are under **other commands**, after the long `start` flag list | `potato validate --help`, `potato preview --help` |
| A command never returns | `potato start` holds the foreground until killed | `nohup potato start … &` and poll for a 200 |
| `validate --strict` prints "Re-run with --strict to fail on unknown keys" while you are running with `--strict` | Wrong message; the exit code is correct | Trust `$?`, not the sentence |
| `--strict` reports nothing about unknown keys | Warnings are suppressed while there are hard errors | Fix the errors and run again |
| Killing the server leaves the port held | It spawns children | `pkill -f "potato start <config> -p <port>"`, or `killpg` on the process group |
| `timeout` not found (macOS) | Not installed | Poll in a loop, or use a Python wrapper |
| `preview` prints `ERROR: Missing required field 'task_dir'` on a config `validate --strict` just passed | `preview` reads the raw file; the loader defaults `task_dir` to the config's own directory, so the server runs fine | Write `task_dir: .` and stop thinking about it. Every other relative path resolves against it anyway |

## Boot

| Symptom | Cause | Fix |
|---|---|---|
| `ConfigSecurityError: Path … outside the project directory` | Running from the wrong directory, or a path escaping `task_dir` | `cd` to the directory `task_dir: .` refers to |
| `Training data must be a JSON object` | The training file is an array | Wrap it: `{"training_instances": [...]}` |
| `Training instance missing required fields: {…}` and it names none | Missing one of `id`, `text`, `correct_answers` | See `quality-control.md` |
| `WARNING: Attention check item missing required fields` → `Loaded 0 attention check items` | Items lack `id` or `expected_answer`. **Boot still succeeds** | Fix the file; re-check the count |
| `Gold standard item skipped, missing gold_label` | Exactly what it says | Add `gold_label` |
| `Phase 'x' in order but not defined in phases config, skipping` | Listed in `order`, never described | Add the phase entry |
| `Failed to load phase 'x': Unknown phase: y` | `type:` is not one of the six | `consent`, `instructions`, `training`, `annotation`, `poststudy`, `prestudy` |
| `Phase x requires 'instrument', 'instruments', or 'file'` | Non-annotation phase with no page | `file: pages/x.jsonl` |
| `Failed to load phase 'x': unhashable type: 'list'` | `instrument:` got a list of scheme dicts | It takes a **string**; use `file:` instead |
| `` `on:` at X was read as the boolean True `` | YAML reads bare `on` as a boolean | Quote it: `"on": true` |
| `Invalid label format: True` from a label list | Same trap in `labels:`. YAML reads bare `Yes`, `No`, `On`, `Off`, `Y`, `N` as booleans, so `labels: [Yes, No]` becomes `[True, False]` | Quote them: `labels: ["Yes", "No"]`. The scheme renders as a heading with no inputs until you do |
| `ai_support.endpoint_type is required when ai_support is enabled` | Enabled without a model | Set `endpoint_type` |
| `adjudication.adjudicator_users must be a non-empty list` | Enabled with no adjudicator | Name one |
| `gold_standards.mode must be one of: training, mixed, separate` | Invented mode | Use one of the three |
| `training.feedback must be a dictionary` / `passing_criteria` / `accuracy` | Passed a bare `true` or a number | `{show_explanations: true}`, `{min_correct: 3}` |
| `attention_checks: specify either 'frequency' or 'probability', not both` | Both set | Pick one |
| `Configuration has both top-level 'annotation_schemes' and phase-level 'annotation_schemes'` | Mixing the two | All-or-nothing; use `file:` for phase pages |

## The rendered page

| Symptom | Cause | Fix |
|---|---|---|
| A scheme is missing from the screenshot | Behind `display_logic`; the initial render does not satisfy it | Comment out the condition, render, put it back — or drive the gate in a browser |
| The item shows a file path instead of an image | No `instance_display` | Add one with the right display type |
| A span scheme with nothing to highlight | No field has `span_target: true` | Set it, and `target_field` on the scheme |
| `span_target is set but type X does not support span annotation` | Wrong display type | The error lists the twelve that accept it |
| Labels read "Information Or Advice" | `humanize_labels` is on by default and title-cases prose labels | `humanize_labels: false` |
| A keyboard shortcut does nothing | The key is `key_value`, not `key_binding`, and nothing validates it | `potato preview --format json` — an empty keybinding list means the label key is misspelled |
| `preview` reports keybinding conflicts | Real for explicit `key_value`, spurious for `sequential_key_binding` (which the server reallocates) | Press the key on a live page. Nothing warns at startup |
| A shortcut does nothing and nothing warns | Two schemes claimed the same `key_value`; the second one loses silently | `potato preview` is the only reporter |
| A canvas scheme renders an empty drawing area | No `instance_display` field of that media type. `source_field` names the data key, it does not fetch the media | Add the display field; expect the media to render twice |
| The image appears twice on an image task | Correct behaviour: display field plus canvas copy | Hide the display copy with `base_css` if it bothers you |
| `No point cloud for this item: the "X" field is empty` on a field that is not empty | `spatial_annotation` reads the path off the rendered page, and a display `label:` ends up in the same text node, which disqualifies it | Display the field with no `label:`, or make the cloud path the item's `text_key` |
| A `region_caption` panel that never lists a region, or a `grounding_eval` whose only answer is "not present" | Neither draws; both need an `image_annotation` scheme beside them | Add the canvas scheme — `modalities.md` has it |
| `+ New event` on `multi_document_event` does nothing, forever | No top-level `event_template:` block, so `/corpus/api/event` 404s. Nothing appears on the page | Add `event_template: {enabled: true, slots: [...]}` |
| The waveform zoom buttons do nothing | `view.getZoom` does not exist in the bundled Peaks build; both audio and video widgets call it | Nothing to configure. *Fit* works; the rest is broken on 2.8.2-11 |
| A free-text answer arrives as one cramped line | `text` defaults to `<input>` | `multiline: true` |
| A labelled Likert renders as radio buttons | `Complex labels detected … using radio layout` | Cosmetic. The metric keys off `annotation_type`; check `kind` in `/admin/iaa` |
| Whole phase pages render bold | Phase prose sits in a `<legend>`, weight 600 | `base_css` with `form.pure-display legend { font-weight: 400 }` |
| Item cards are enormous for one line of text | `instance_display.resizable` defaults on | `resizable: false` |
| The questions are below the fold on every item | `annotation_instructions` is long and expanded by default | Move definitions to an `instructions` phase; keep the banner short |
| Instructions banner appears on the consent page | It is page-level, on every page | Word it to survive that, or shorten it |
| A grid scheme is squeezed into a narrow column | Auto layout | `layout` on the scheme, or `instance_display.layout` |

## Console and network

| Symptom | Cause |
|---|---|
| `GET /api/current_instance 404` + `[SpanManager] Failed to get server instance ID` on consent/instructions/poststudy | Normal. Phase pages have no instance; the span layer asks anyway |
| `GET /api/spans/null 404`, `POST /api/track_annotation_change 400` on phase pages | Same cause. Also benign |
| `potato preview --phase consent --screenshot` exits 1 on a task that is fine | Same cause: those three lines are counted as things to report. The annotation page exits 0. Judge the PNG, not `$?` |
| `--phase prestudy` refused: `phase must be one of (…)` | The preview CLI knows five phases; the server knows six. Walk a prestudy page in a browser |
| `ERROR potato.flask_server: Error getting instance text: 'null'` | Same cause, server side |
| `[NAV] Navigation failed: 400` | The server refused the save; body is JSON with `unsatisfied_schemas` |
| `400 {"status":"validation_error","unsatisfied_schemas":["x"]}` after answering everything visible | A required **span** scheme with no span drawn. The client cannot validate spans, so the only feedback is a corner toast naming the internal scheme name |
| A scheme renders as a heading with no inputs | Its generator failed; the startup log names it |

## Assignment and data

| Symptom | Cause | Fix |
|---|---|---|
| Annotators never see the last items | `max_annotations_per_user` is set below the item count, or `per_annotator_quota` is capping them lower still | Raise whichever applies; `per_annotator_quota` wins, so raising the global alone changes nothing |
| `instance_per_annotator` has no effect | It is not read; `automatic_assignment` sub-keys are unvalidated | Use `max_annotations_per_user`, or `per_annotator_quota` for per-annotator caps |
| Agreement pages are empty | No overlapping annotations yet | Not a config problem |
| Every admin JSON route returns `403 {"error":"Admin access required"}` | They need `X-API-Key` | `cat admin_api_key.txt` in `task_dir`; `/admin` HTML is open, the APIs are not |
| One annotator got fewer items than the others | Usually a quota set below the item count | Count annotators per item across **all** `user_state.json` files, not one |
| Wiping `annotation_output/` changed nothing, and new annotators get "Thank You!" immediately | State is in memory; the wipe happened while the server was running | Stop, wipe, start |
| A new annotator gets "Thank You!" on a server that was never wiped | `items x num_annotators_per_item` is used up, so there is no work to hand out | Raise `num_annotators_per_item`, or add items. No log line marks this |
| A re-run of a browser driver lands on the sign-in page | Duplicate registration navigates to `/register`, which has no `switchTab()` | Use a fresh username per run |
| `gold_standards.auto_promote: false` / `accuracy: 0.7` rejected | Both must be dictionaries despite being typed `boolean\|object` and `number\|object` | `{enabled: false}`, `{min_threshold: 0.7}` |
| Annotations look wrong after a restart | Something hand-edited `output_annotation_dir` | Never edit it; the server rewrites those files whole |
| Edited schemes but the page looks the same | `layouts/task_layout_*.html` is cached by hash | `rm -rf layouts` and restart |
| A codebook holds labels you already changed | The first boot seeded `project.sqlite` from the config | `potato codebook config.yaml` |
| Test annotators show up in the researcher's data | They are real annotators holding real assignments | Wipe `annotation_output/` before handover |

## The form

| Symptom | Cause |
|---|---|
| A `layout` group renders with fewer questions than it lists | Group membership used to match on `data-schema-name`, which 16 generators (`slider`, `span`, the canvas and media schemes, `multi_document_event`) never render. The missing ones sat loose above the group. Fixed; on an older build, put those schemes outside groups |
| An item is "Labeled" before anyone answered | `slider`, `vas` and `soft_label` render at a starting position and the sync used to store it. Fixed; on an older build, treat a midpoint answer as possibly untouched, and check `annotation_telemetry` for whether anyone interacted |
| A required `slider` refuses to let anyone past | The requiredness check wanted `data-modified`, which nothing set on a range input, so dragging it never satisfied the requirement. Fixed; on an older build, do not mark a slider scheme required |
| A slider snaps to its minimum when the instance changes | `clearAllFormInputs` reset to the `min` attribute, and `starting_value` is not an attribute, so it never won. Fixed with the above |
| Slider tick labels overlap and the value bubble covers them | The scheme is in a narrow grid column. Give it `layout: {columns: 2}` or leave it full width |
| Two schemes both claim key `1` and only one fires | Explicit `key_value` is never reallocated. Only `preview` reports it |

## Verification traps

| Trap | Why it lies |
|---|---|
| Testing persistence with a page refresh | Browsers restore form state across refresh; the check passes when the server stored nothing. Navigate away and back instead |
| Checking a conditional input with width/height | Hidden inputs still have a bounding box. Walk the ancestor chain for `display:none` |
| Treating `--screenshot` exit 0 as "the UI is fine" | It renders one page, viewport only, and phase-page console noise is filtered out of your attention |
| Trusting `validate` on phases, surveyflow or scheme entries | It does not look inside them |
| Trusting `validate` on the paths a config names | It checks them now, but at two severities. A missing `base_css`, `header_logo` or data file is an error; a missing `keyword_highlights_file` is a warning, because the server starts anyway with the feature off. `--strict` makes both fatal, so run it |
| Driving a span drag after `scroll_into_view_if_needed` | Puts the text under the sticky navbar; the drag selects the header. Scroll up ~160px first |