When a Task Misbehaves

Symptoms that validate clean and then do something else. It ships with the potato-tasks skill:

/plugin marketplace add davidjurgens/potato-skill
/plugin install potato-tasks@potato

---

Ordered by where you are when it happens. Everything here was hit for real on potato-annotation 2.7.0.

The command

Symptom	Cause	Fix
potato --help seems not to list validate or preview	They are under other commands, after the long start flag list	potato validate --help, potato preview --help
A command never returns	potato start holds the foreground until killed	nohup potato start … & and poll for a 200
validate --strict prints "Re-run with --strict to fail on unknown keys" while you are running with --strict	Wrong message; the exit code is correct	Trust $?, not the sentence
--strict reports nothing about unknown keys	Warnings are suppressed while there are hard errors	Fix the errors and run again
Killing the server leaves the port held	It spawns children	pkill -f "potato start <config> -p <port>", or killpg on the process group
timeout not found (macOS)	Not installed	Poll in a loop, or use a Python wrapper
preview prints ERROR: Missing required field 'task_dir' on a config validate --strict just passed	preview reads the raw file; the loader defaults task_dir to the config's own directory, so the server runs fine	Write task_dir: . and stop thinking about it. Every other relative path resolves against it anyway

Boot

Symptom	Cause	Fix
ConfigSecurityError: Path … outside the project directory	Running from the wrong directory, or a path escaping task_dir	cd to the directory task_dir: . refers to
Training data must be a JSON object	The training file is an array	Wrap it: {"training_instances": [...]}
Training instance missing required fields: {…} and it names none	Missing one of id, text, correct_answers	See quality-control.md
WARNING: Attention check item missing required fields → Loaded 0 attention check items	Items lack id or expected_answer. Boot still succeeds	Fix the file; re-check the count
Gold standard item skipped, missing gold_label	Exactly what it says	Add gold_label
Phase 'x' in order but not defined in phases config, skipping	Listed in order, never described	Add the phase entry
Failed to load phase 'x': Unknown phase: y	type: is not one of the six	consent, instructions, training, annotation, poststudy, prestudy
Phase x requires 'instrument', 'instruments', or 'file'	Non-annotation phase with no page	file: pages/x.jsonl
Failed to load phase 'x': unhashable type: 'list'	instrument: got a list of scheme dicts	It takes a string; use file: instead
`on:` at X was read as the boolean True	YAML reads bare on as a boolean	Quote it: "on": true
Invalid label format: True from a label list	Same trap in labels:. YAML reads bare Yes, No, On, Off, Y, N as booleans, so labels: [Yes, No] becomes [True, False]	Quote them: labels: ["Yes", "No"]. The scheme renders as a heading with no inputs until you do
ai_support.endpoint_type is required when ai_support is enabled	Enabled without a model	Set endpoint_type
adjudication.adjudicator_users must be a non-empty list	Enabled with no adjudicator	Name one
gold_standards.mode must be one of: training, mixed, separate	Invented mode	Use one of the three
training.feedback must be a dictionary / passing_criteria / accuracy	Passed a bare true or a number	{enabled: true}, {min_accuracy: 0.6}
attention_checks: specify either 'frequency' or 'probability', not both	Both set	Pick one
Configuration has both top-level 'annotation_schemes' and phase-level 'annotation_schemes'	Mixing the two	All-or-nothing; use file: for phase pages

The rendered page

Symptom	Cause	Fix
A scheme is missing from the screenshot	Behind display_logic; the initial render does not satisfy it	Comment out the condition, render, put it back — or drive the gate in a browser
The item shows a file path instead of an image	No instance_display	Add one with the right display type
A span scheme with nothing to highlight	No field has span_target: true	Set it, and target_field on the scheme
span_target is set but type X does not support span annotation	Wrong display type	The error lists the twelve that accept it
Labels read "Information Or Advice"	humanize_labels is on by default and title-cases prose labels	humanize_labels: false
A keyboard shortcut does nothing	The key is key_value, not key_binding, and nothing validates it	potato preview --format json — an empty keybinding list means the label key is misspelled
preview reports keybinding conflicts	Real for explicit key_value, spurious for sequential_key_binding (which the server reallocates)	Press the key on a live page. Nothing warns at startup
A shortcut does nothing and nothing warns	Two schemes claimed the same key_value; the second one loses silently	potato preview is the only reporter
A canvas scheme renders an empty drawing area	No instance_display field of that media type. source_field names the data key, it does not fetch the media	Add the display field; expect the media to render twice
The image appears twice on an image task	Correct behaviour: display field plus canvas copy	Hide the display copy with base_css if it bothers you
No point cloud for this item: the "X" field is empty on a field that is not empty	spatial_annotation reads the path off the rendered page, and a display label: ends up in the same text node, which disqualifies it	Display the field with no label:, or make the cloud path the item's text_key
A region_caption panel that never lists a region, or a grounding_eval whose only answer is "not present"	Neither draws; both need an image_annotation scheme beside them	Add the canvas scheme — modalities.md has it
+ New event on multi_document_event does nothing, forever	No top-level event_template: block, so /corpus/api/event 404s. Nothing appears on the page	Add event_template: {enabled: true, slots: [...]}
The waveform zoom buttons do nothing	view.getZoom does not exist in the bundled Peaks build; both audio and video widgets call it	Nothing to configure. Fit works; the rest is broken on 2.8.2-11
A free-text answer arrives as one cramped line	text defaults to <input>	multiline: true
A labelled Likert renders as radio buttons	Complex labels detected … using radio layout	Cosmetic. The metric keys off annotation_type; check kind in /admin/iaa
Whole phase pages render bold	Phase prose sits in a <legend>, weight 600	base_css with form.pure-display legend { font-weight: 400 }
Item cards are enormous for one line of text	instance_display.resizable defaults on	resizable: false
The questions are below the fold on every item	annotation_instructions is long and expanded by default	Move definitions to an instructions phase; keep the banner short
Instructions banner appears on the consent page	It is page-level, on every page	Word it to survive that, or shorten it
A grid scheme is squeezed into a narrow column	Auto layout	layout on the scheme, or instance_display.layout

Console and network

Symptom	Cause
GET /api/current_instance 404 + [SpanManager] Failed to get server instance ID on consent/instructions/poststudy	Normal. Phase pages have no instance; the span layer asks anyway
GET /api/spans/null 404, POST /api/track_annotation_change 400 on phase pages	Same cause. Also benign
potato preview --phase consent --screenshot exits 1 on a task that is fine	Same cause: those three lines are counted as things to report. The annotation page exits 0. Judge the PNG, not $?
--phase prestudy refused: phase must be one of (…)	The preview CLI knows five phases; the server knows six. Walk a prestudy page in a browser
ERROR potato.flask_server: Error getting instance text: 'null'	Same cause, server side
[NAV] Navigation failed: 400	The server refused the save; body is JSON with unsatisfied_schemas
400 {"status":"validation_error","unsatisfied_schemas":["x"]} after answering everything visible	A required span scheme with no span drawn. The client cannot validate spans, so the only feedback is a corner toast naming the internal scheme name
A scheme renders as a heading with no inputs	Its generator failed; the startup log names it

Assignment and data

Symptom	Cause	Fix
Annotators never see the last items	max_annotations_per_user is set below the item count	Raise it, or leave it unset so the cap is the item count
instance_per_annotator has no effect	It is not read; automatic_assignment sub-keys are unvalidated	Use max_annotations_per_user
Agreement pages are empty	No overlapping annotations yet	Not a config problem
Every admin JSON route returns 403 {"error":"Admin access required"}	They need X-API-Key	cat admin_api_key.txt in task_dir; /admin HTML is open, the APIs are not
/admin/api/agreement returns experiment_col TypeErrors for every scheme	Build bug, seen on a geometry task	Use /admin/iaa
One annotator got fewer items than the others	Usually a quota set below the item count	Count annotators per item across all user_state.json files, not one
Wiping annotation_output/ changed nothing, and new annotators get "Thank You!" immediately	State is in memory; the wipe happened while the server was running	Stop, wipe, start
A new annotator gets "Thank You!" on a server that was never wiped	items x num_annotators_per_item is used up, so there is no work to hand out	Raise num_annotators_per_item, or add items. No log line marks this
A re-run of a browser driver lands on the sign-in page	Duplicate registration navigates to /register, which has no switchTab()	Use a fresh username per run
gold_standards.auto_promote: false / accuracy: 0.7 rejected	Both must be dictionaries despite being typed boolean\|object and number\|object	{enabled: false}, {min_threshold: 0.7}
Annotations look wrong after a restart	Something hand-edited output_annotation_dir	Never edit it; the server rewrites those files whole
Edited schemes but the page looks the same	layouts/task_layout_*.html is cached by hash	rm -rf layouts and restart
A codebook holds labels you already changed	The first boot seeded project.sqlite from the config	potato codebook config.yaml
Test annotators show up in the researcher's data	They are real annotators holding real assignments	Wipe annotation_output/ before handover

The form

Symptom	Cause
A layout group renders with fewer questions than it lists	Group membership used to match on data-schema-name, which 16 generators (slider, span, the canvas and media schemes, multi_document_event) never render. The missing ones sat loose above the group. Fixed; on an older build, put those schemes outside groups
An item is "Labeled" before anyone answered	slider, vas and soft_label render at a starting position and the sync used to store it. Fixed; on an older build, treat a midpoint answer as possibly untouched, and check annotation_telemetry for whether anyone interacted
A required slider refuses to let anyone past	The requiredness check wanted data-modified, which nothing set on a range input, so dragging it never satisfied the requirement. Fixed; on an older build, do not mark a slider scheme required
A slider snaps to its minimum when the instance changes	clearAllFormInputs reset to the min attribute, and starting_value is not an attribute, so it never won. Fixed with the above
Slider tick labels overlap and the value bubble covers them	The scheme is in a narrow grid column. Give it layout: {columns: 2} or leave it full width
Two schemes both claim key 1 and only one fires	Explicit key_value is never reallocated. Only preview reports it

Verification traps

Trap	Why it lies
Testing persistence with a page refresh	Browsers restore form state across refresh; the check passes when the server stored nothing. Navigate away and back instead
Checking a conditional input with width/height	Hidden inputs still have a bounding box. Walk the ancestor chain for display:none
Treating --screenshot exit 0 as "the UI is fine"	It renders one page, viewport only, and phase-page console noise is filtered out of your attention
Trusting validate on phases, surveyflow or scheme entries	It does not look inside them
Driving a span drag after scroll_into_view_if_needed	Puts the text under the sticky navbar; the drag selects the header. Scroll up ~160px first