# Agent Instructions Codex and Cursor read `AGENTS.md` from the project root, so it needs no install step: copy it into the project you are working in. Claude Code reads `CLAUDE.md` instead, and a `CLAUDE.md` holding the line `@AGENTS.md` imports it. It is the short version. The references and helper scripts it points to come with the skill, installed as below. The text after the rule is the file itself. ## Installing the skill ``` /plugin marketplace add davidjurgens/potato-skill /plugin install potato-skill@potato ``` For Codex and Cursor, the `skills` CLI copies the whole skill -- `SKILL.md`, the references and the helpers -- into `.agents/skills/potato-skill/`: ```bash npx skills add davidjurgens/potato-skill --agent codex cursor ``` The skill is a plain directory, so it also travels: ```bash python scripts/package_skill.py ``` builds `dist/potato-skill/` -- `SKILL.md`, the references, and the 9 executable helpers -- and `dist/potato-skill.zip`. One build, three destinations: | Where | How | |---|---| | Claude Code, every project on the machine | `cp -r dist/potato-skill ~/.claude/skills/`, or `--install-personal` | | Claude Agent SDK | Point the SDK's skill directory at `dist/`; it loads the skill the same way Claude Code does | | Claude API | Upload `dist/potato-skill.zip` as a skill, then name it on the request. The endpoint is in beta -- check Anthropic's Agent Skills documentation for the current call | The scripts in the bundle drive the `potato` command and import Potato's registries, so wherever the skill runs, Potato has to be installed in the environment running the commands. `SKILL.md` opens by checking exactly that. --- Potato is a web annotation platform. A task is one YAML config plus a data file. This file tells you how to build one correctly. ## The loop Do not write a config and hand it over. Run it through these steps first. ```bash potato validate config.yaml --strict # structural check potato preview config.yaml # what form did I declare? potato preview config.yaml --screenshot out.png # what does it look like, did it break? ``` All three return promptly. `potato start config.yaml -p 8000` runs the task for a human to click through, but **it does not return** -- it holds the server in the foreground until killed, and will hang a non-interactive session indefinitely. The screenshot step already boots a real server and shuts it down, so nothing in this loop needs it. Background it if you genuinely want one running. `potato validate` runs the same checks `potato start` runs at boot, so passing it means the server will accept the config. Its errors name the alternatives: an unknown `annotation_type` comes back with the list of valid ones. `--strict` turns unrecognized keys from a warning into a failure, which is how you catch a typo that silently disabled a feature. `--screenshot` is the step that catches what validation cannot. It boots the task, opens it in a headless browser, and reports every uncaught exception, `console.error` and failed request. A config can validate cleanly and still render a broken interface, because most of the annotation UI is built by JavaScript after the HTML arrives. Exit code is 0 only if the page rendered with nothing to report. ## Deciding what to build A config is the last step, and the decisions before it are the ones that are expensive to reverse: what one annotator sees at a time, what they are asked, how they answer, how many people answer each item, and how you find out whether the answers are any good. When a description leaves those open, decide the cheap ones yourself and confirm the expensive ones -- the unit of annotation, the annotator budget, consent wording, and anything involving identifiable people. Build with stated assumptions rather than waiting on an answer. Two that are easy to get wrong and expensive to discover late: - **The response format decides the agreement metric.** Potato picks the metric from the schema kind, so a 1--5 rating stored as a `radio` is scored as unordered and "1 vs 2" counts the same as "1 vs 5". - **`num_annotators_per_item` sets the count.** Unset, there is no limit. `min_annotators_per_instance` and `max_annotations_per_item` are older spellings of the same setting, and two of them with different numbers fail validation. The potato-skill skill has the full versions in its `references/` directory: `designing-a-task.md`, `asking-the-experimenter.md` and `building-the-ui.md`. Without the skill, read them at and the pages beside it. ## Start from a working example There are 218 example projects. Copying one beats assembling a config from field lists, and every one of them is checked in CI. ```bash # Search the catalog python -c "from potato.server_utils.examples_manifest import search_examples; \ print([e['dir'] for e in search_examples(annotation_type='span')])" ``` The catalog is also published as JSON at `potato/schemas/potato-examples.manifest.json`. Each entry records the annotation types and display types a config uses, the features it switches on, and the command that runs it. ## The minimum config Four keys are always required, plus a data source: ```yaml # yaml-language-server: $schema=https://potatoannotator.readthedocs.io/en/latest/schemas/potato-config.schema.json annotation_task_name: Sentiment Annotation task_dir: . output_annotation_dir: annotation_output/ data_files: - data/items.json item_properties: id_key: id text_key: text annotation_schemes: - annotation_type: radio name: sentiment description: How does this text feel? labels: [positive, neutral, negative] ``` Keep the first line. It switches on live validation in VS Code, JetBrains, Zed and Helix, and it tells the next agent where the contract is. The data file is JSON, JSONL, CSV or TSV. JSON may be an array of objects or one object per line. Every object needs the fields named by `id_key` and `text_key`. ## Rules that will bite you **Paths resolve against `task_dir`, and anything resolving outside it is rejected.** This is a security check, not a convenience. If you see `ConfigSecurityError: Path ... is outside the project directory`, you are running from the wrong directory or pointing outside the project. **`annotation_type` must come from the registry.** There are 61 of them. Do not guess a plausible-sounding name. `sentiment` and `classification` are not types. ```bash python -c "from potato.server_utils.schemas.registry import schema_registry; \ print(schema_registry.get_supported_types())" ``` **Unknown config keys only produce a warning.** A typo'd key is silently ignored and the feature you thought you enabled is off. Run `potato validate --strict`, which turns those warnings into a failure. **Phase-level `annotation_schemes` replace the top-level list.** They do not add to it. Having both is an error. **Never hand-edit anything in `output_annotation_dir`.** The server rewrites those files wholesale. ## Reference Every key, with its type, default and description, is in `docs/configuration/config_reference.md`, and the same information machine-readable is in `potato/schemas/potato-config.schema.json`. From Python: ```python from potato.server_utils.config_key_docs import get_key_doc get_key_doc("attention_checks.frequency") from potato.server_utils.schema_examples import example_scheme_for example_scheme_for("bws") # a scheme from a config that really runs ``` ## MCP `potato mcp` is two servers, both listed under **other commands** in `potato --help`. `potato mcp serve --root .` answers authoring questions out of the same registries as everything above, through tools such as `list_annotation_types`, `describe_annotation_type`, `list_examples`, `validate_config`, `preview_config` and `render_task_screenshot`. `potato mcp tools` prints all of them. Same answers as the shell, and `render_task_screenshot` hands you the rendered page as an image. `potato mcp connect --url … --token …` bridges a *running* task, if its config carries an `mcp` block listing tools in `mcp.tools` and an agent holds a token from `potato mcp issue-token`. The bridge adds 17 live tools named `live_get_status`, `live_get_progress`, `live_list_items`, `live_get_item`, `live_list_annotators`, `live_get_config`, `live_get_agreement`, `live_assign_items`, `live_export_data`, `live_submit_annotation`, `live_add_items` and `live_delete_annotations`, plus five that drive the labeling engine: `live_engine_status`, `live_engine_requests`, `live_engine_run`, `live_engine_certify` and `live_engine_certificate`. `live_delete_annotations` destroys an annotator's work on an item, so it also has to be listed in `mcp.destructive` and called with `confirm: true`; Potato refuses it otherwise. Each publishes a real parameter schema, so `tools/list` tells you what to pass. ```bash potato mcp config --root . # prints a client config block potato mcp tools # the tool list, without starting anything ``` ## Reporting a bug or asking for a feature Sort it before reporting it. `potato validate config.yaml --strict` failing, or a boot log line reading `Loaded 0 `, means a config mistake -- a key misspelled inside a block Potato treats as opaque is accepted, ignored, and never mentioned again. That is the usual cause of "the config is right and nothing happens", and it is not a defect. If Potato does something other than what it documents, that is a bug. If it does exactly what it documents and that is not enough, that is a feature request -- and before filing one, name the closest existing annotation type or config key and say why it does not fit, because it exists more often than people expect. Issues go to . A useful report carries the version (ask `potato --version`; releases up to 2.7 have no such flag, and neither `importlib.metadata.version("potato-annotation")` nor `potato.__version__` answers the same way from every directory, so say which you used and name the commit if you run from a clone), the `--strict` result, the shortest config that shows it, the shape of one data record rather than the records, and what you measured against what you expected. Strip `secret_key`, `ai_support.ai_config.api_key`, crowd credentials and annotator email addresses first. The potato-skill skill has a helper that assembles all of that, `scripts/report_issue.py` in the skill's directory. `npx skills add davidjurgens/potato-skill --agent codex cursor` puts that directory at `.agents/skills/potato-skill/`. Filing is public and cannot be withdrawn. Show the report to whoever owns the study and get an answer before you file it. ## Where to read more - Configuration reference: `docs/configuration/config_reference.md` in Potato's repository - Every annotation type: `docs/annotation-types/schemas_and_templates.md` in Potato's repository - Full docs in one file: - Curated index: