Evaluating and Iterating on the Interface

How to judge a rendered annotation page and what to change. It ships with the potato-tasks skill:

/plugin marketplace add davidjurgens/potato-skill
/plugin install potato-tasks@potato

---

A config that validates tells you the server will start. It tells you nothing about whether a person can do the work. That question only has one answer: render the page, look at it, and change something.

The loop

potato preview config.yaml --screenshot shot-01.png

Before you start: the PNG is 1280x900 of the top of one page, not the whole page. Half the checklist below is about things further down. Get a full-height render by driving a browser against a running server (running-a-task.md), and treat --screenshot as the fast smoke test rather than the visual check.

Then, in order:

1. Read the error lines first. Uncaught exceptions and console.error mean part of the interface never initialized. Nothing else you notice matters until those are gone — except the three Potato logs on every phase page (/api/current_instance 404 and two [SpanManager] lines), which are present on a healthy task.
2. Look at the PNG. Not the exit code. A clean exit means nothing threw.
3. Make one change.
4. Re-render to a new filename — shot-02.png, not the same name. You need the pair to see what your change did.

Keep going until the checklist below passes. Three or four rounds is normal for anything with a custom layout.

What to look for

Work down the list. Each line is a failure that has shipped in a real task.

The item

• Is it there at all? A raw file path where an image should be means no instance_display.
• Is it readable — not clipped, not scrolled off, not one line of a long document with no way to see the rest?
• If it is media: does the player have controls, and are they inside the frame?

The questions

• Is every scheme present? Count them against the config. A missing one is usually behind display_logic (see below) or failed to generate.
• Is any of them empty — a heading with no inputs under it?
• Are the labels readable, or truncated with an ellipsis?
• Do radio buttons and checkboxes actually appear? An invisible input control is the classic CSS collision, and the label text still renders, so the question looks fine at a glance.
• Do tick marks, scale points and their labels line up, or do the labels overlap each other at the ends?
• For a grid or matrix scheme: does it use the full width, or is it squeezed into a narrow column with horizontal scroll?

The page

• Is anything shifted right, overlapping, or overflowing its container?
• Is the Next button visible without scrolling past the fold? An annotator hits it several hundred times.
• Does the whole page scroll horizontally? It should not.

The work itself

• Count the interactions to complete one item: clicks, scrolls, keystrokes. Multiply by the number of items. That number is the study's cost.
• Do the schemes have keyboard shortcuts where they could?
• Is anything ordered so that the annotator has to scroll up and back down?

The states a screenshot will not show you

The render captures the page as it loads. Four things live outside that, and each has to be checked another way.

Conditional schemes. Anything behind display_logic is absent from the initial render, and its absence proves nothing. Comment out the display_logic block, render, look, put it back. Do this every time — the widget behind the condition is often the whole point of the task.

Anything that appears after an interaction. A span highlight, a drawn box, a playing video, an expanded section, minting a code, saving a note. Render to confirm the control is there, then drive it if you have a browser tool.

Without one there is still more evidence than the PNG. Run the server once in the background and read what it did:

• the startup log says which subsystems initialized and what they loaded — codebook labels, detected cases, how many instances got indexed
• project.sqlite in the task directory holds the codebook, cases, memos and the search index, and can be queried directly
• annotation_output/<user>/user_state.json shows what a scripted pass actually stored

That distinguishes "the feature is configured and initialized" from "the button is on the page", which is most of the distance. Say in your handover which of the two you established.

Other phases. --phase consent, --phase instructions, --phase training, --phase poststudy. Only those four plus annotation: the preview CLI refuses prestudy even though the server supports it as a phase.

A phase-page render exits 1 on a healthy task. Phase pages have no instance, so the span layer's /api/current_instance 404 and two [SpanManager] console errors are always reported, and --screenshot counts them. Judge the PNG and the error list, not $?. See phases-and-pages.md. Each is a page an annotator sees, and the training phase renders real schemes, so it can break in ways the annotation page does not.

Only render a phase the task actually has. --phase X parks an annotator in that phase, so if the task never configures one, every route redirects toward a page that does not exist and the render fails with a redirect loop. annotation_instructions is the banner on the annotation page, not an instructions phase — a task can have the banner and no phase.

State across navigation. Answers are saved and restored on a full page reload. If you have a browser tool, annotate, click Next, click Previous, and check the answer is still shown — visually, not just in a hidden input. Do not test this with a page refresh: browsers restore form state across refresh on their own, so a refresh-based check passes even when the server never stored anything.

Changing the interface

Reach for these in order. Stop at the first one that fixes it.

Want	Use
Fields side by side rather than stacked	instance_display.layout.direction: horizontal
More or less space between fields	instance_display.layout.gap
Choices in a row rather than a column	layout on the scheme
Shortcut hints along one line	horizontal_key_bindings: true
Standing instructions at the top	annotation_instructions
Something above every item	header_file
A logo	header_logo
Colours, spacing, fonts	base_css
More room — no navbar, no jump control	hide_navbar, jumping_to_id_disabled
An arrangement none of the above reaches	task_layout

task_layout is last for a reason: hand-written form HTML stops tracking changes to the schema generators, so a scheme that gains a feature later will not gain it here. Most requests that sound like they need it are instance_display.layout plus scheme order.

With base_css, change what you meant to change. A rule that targets a bare element selector will reach the navbar and the admin pages too.

Iterating without going in circles

One change per render. Two changes and a difference you cannot attribute is how a session turns into twenty screenshots and no conclusions.

Keep the screenshots numbered. They are the record of what you tried.

Write down what you were fixing. "shot-03: moved confidence above the span so the page stops reflowing" beats three unlabelled PNGs.

Know when to stop. The interface is done when: no console errors, every scheme visible and usable, nothing overlapping or clipped, the Next button reachable without scrolling, and shortcuts on anything that can take them. Past that you are decorating, and the researcher has opinions you do not have access to. Hand it over and say what you would change next.

Running the task writes to the project directory. The first server start creates project.sqlite and seeds it — for a codebook task, the labels in your config become the project codebook. Change the labels afterwards and the codebook still holds the old ones until potato codebook config.yaml re-syncs. Worth knowing before you verify a config you are still editing.

When a change does nothing, suspect the key before the CSS. Unrecognized config keys only warn, so a typo is silently ignored and the feature you thought you enabled is off. Run potato validate --strict.

Reporting what you saw

A handover that says "renders correctly" is not evidence. Say what you checked:

Rendered at 1280×900. Three schemes present, one behind display_logic (verified separately by disabling the condition — screenshot shot-04.png). No console errors. Keyboard shortcuts on the yes/no question. The consent and instructions phases render (shot-05, shot-06).

Not checked: span highlighting behaviour after a drag, and whether answers survive navigating away and back — neither is visible in a static render.

The second paragraph is the part that makes the first one worth reading.