# Data Files and Admin Access Input formats, side files, and reading results back out. It ships with the `potato-skill` skill: ``` /plugin marketplace add davidjurgens/potato-skill /plugin install potato-skill@potato ``` --- Where items come from, and who is allowed to annotate them. Nothing else in this pack covers either. ## Items ```yaml data_files: - data/articles.json item_properties: id_key: id text_key: body ``` `data_files` accepts JSON, JSONL, CSV and TSV. JSON may be an array of objects or one object per line. Every object needs the fields named by `id_key` and `text_key`. Several files merge in the order listed, and they do not have to carry the same fields — I loaded a three-item file and a two-item file whose objects had different extra keys and got five instances, the first file's items first. But **an id repeated across files stops the server dead**, and `validate` does not see it coming: ``` $ potato validate config.yaml --strict -> OK — no issues found. $ potato start config.yaml -> ValueError: Duplicate instance ID 'SHARED' found at item 2 ``` Failing rather than silently keeping one of the two is the right call, and the message names the id. Worth knowing before you split a corpus across files, because it is the one data problem that passes validation and then refuses to boot. | Key | Effect | |---|---| | `item_properties.id_key` | Field holding the unique identifier | | `item_properties.text_key` | Field shown to the annotator when there is no `instance_display` | | `item_properties.category_key` | Field holding a category, used by `category_based` assignment | | `item_properties.kwargs` | Extra per-item fields carried through to the display layer | **`text_key` still matters when you define `instance_display`.** It is what plain-text rendering falls back to, what span schemes anchor to by default, and what the training and search surfaces read. Point it at the field with the actual content, not at a title. **Every other field in the row is available to the display layer** by name. A row with `outlet`, `headline` and `body` can render all three as separate `instance_display.fields` without any of them being `text_key`. Fields you do not display are still stored, which is how you keep a condition label out of the annotator's view without dropping it from the data — but only `parquet` exports them. csv, tsv and jsonl carry `instance_id` and the answers and no item fields at all, so plan to hand the data file over with the export, or to use `parquet`. `getting-the-data-out.md` has the comparison. **Every path in the config has to resolve inside the project directory.** A `data_files` entry that climbs out with `../` is refused at boot with `ConfigSecurityError: Path '...' resolves to '...' which is outside the project directory`, and `potato validate` reports the same thing without booting. Copy shared data into the project, or move the config up to a directory that contains both. ### A directory instead of a list ```yaml data_directory: ./data/incoming data_directory_encoding: utf-8 watch_data_directory: true # rescan while running and pick up new files watch_poll_interval: 5.0 ``` It reads `.json`, `.jsonl`, `.csv` and `.tsv`, and skips everything else without comment. A directory of `.md` or `.txt` files loads nothing, and the only sign is `Loaded 0 instances from data_directory: ./incoming` at INFO in the boot log — the config validates, the server starts, and the first annotator gets an empty study. Count the instances in the log before handing out the URL. This is also not the way to get documents into a `document` field; see `building-the-ui.md`. ### Remote and live sources ```yaml # type: url, s3, huggingface, google_sheets, database, google_drive, # dropbox or file data_sources: - type: url url: https://example.org/items.json data_cache: enabled: true ttl_seconds: 3600 max_size_mb: 200 ``` Sources can be polled for new rows, which is how a task keeps ingesting while annotators work. Pair with `automation` (rules over incoming items) and `triage` (rank what arrives). Both are in `modes-and-subsystems.md`. ### Media ```yaml media_directory: media # default; served at /media/ ``` Put images, audio and video under it, and write the path **including the directory** in the data field: `media/photo.png`, not `photo.png`. The displays emit the field value as the URL; only `depth_map` prepends the directory itself, so a bare filename resolves against the site root and 404s. Measured across one page on 2.8.2-26: `image`, `gallery`, `video`, `audio`, `pdf`, `audio_dialogue` and `web_agent_trace` all need the prefix in the data, and all of them except `pdf` fail without a word on the page. Without an `instance_display` field of the right type, that path renders as **text** — the classic "the item shows a file path instead of an image". See `building-the-ui.md` for the display types. ### Big corpora ```yaml partial_loading: {...} # load lazily rather than reading everything at boot item_store: backend: paged # memory (default) or paged cache_size: 1000 path: .item_cache.sqlite # under the output directory when unset ``` An unknown `item_store.backend` warns and silently falls back to memory rather than refusing to start, so check the boot log if paging was the point. ## Output ```yaml output_annotation_dir: annotation_output/ export_annotation_format: [csv] # periodic auto-export; empty means none auto_export_interval: 60 # seconds export_include_phase_data: false # consent and survey answers; usually where the PII is export_include_annotation_changes: false ``` **There is one storage format and it is not configurable.** Annotations live at `annotation_output//user_state.json`, one file per annotator, whatever else the config says. Any other shape is an export, and there are two ways to ask for one: `export_annotation_format`, which writes it on a timer, and the admin export route, which writes it on demand. `output_annotation_format` is deprecated — the loader reads it as `export_annotation_format` and warns. Write the live key in anything you author. So what a researcher gets at the end of a study is a directory of per-annotator JSON, not a table. If they want one row per item, produce it: either set `export_annotation_format: [csv]`, or read the `user_state.json` files and write the table yourself. Decide which before handover and say so, because "where is my CSV" is the first question after the annotators finish. `annotation_output/` is rewritten wholesale by the server. Never hand-edit anything in it — `potato repair-annotations` exists for when something already did. Per-annotator state lives at `annotation_output//user_state.json` and is the fastest way to check what the server actually stored. Keys worth knowing: `instance_id_ordering`, `instance_id_to_label_to_value`, `instance_id_to_span_to_value`, `phase_to_page_to_label_to_value`, `training_state`, `max_assignments`. ## Who may log in ```yaml login: type: password # password | url_direct | none user_config: allow_all_users: true # anyone may register ``` | Key | Effect | |---|---| | `login.type` | `password` (register and sign in), `url_direct` (identity in a query parameter), `none` | | `login.url_argument` | The query parameter carrying the user id under `url_direct` | | `user_config.allow_all_users` | Default `true`. Anyone can register | | `user_config.users` | Allowlist, used when `allow_all_users: false` | | `require_password` / `require_no_password` | Force the choice regardless of `login.type` | | `authentication` | SSO and database-backed accounts | | `rbac` | Role assignments and SSO role mapping | `url_direct` is what crowdsourcing platforms want: the platform appends the worker id and the annotator never sees a login screen. ### Running on a crowd platform ```yaml login: {type: url_direct, url_argument: PROLIFIC_PID} crowdsourcing: {provider: prolific} completion_code: CODE-ABC123 auto_redirect_on_completion: true auto_redirect_delay: 3000 # milliseconds on the completion page first ``` `crowdsourcing.provider` takes `prolific`, `mturk`, `connect`, `sona`, `microworkers`, `clickworker`, `expert`, `generic` or `url_direct`, and the boot log confirms it — `Initialized crowdsourcing provider: prolific`. Set `login.url_argument` to whatever the platform appends: Prolific sends `PROLIFIC_PID`, so `?PROLIFIC_PID=worker777` seats that worker with no login screen and their id in the navbar. Naming a provider changes the completion page, which is the part worth seeing before you launch. With the redirect off it reads Thank You! You have completed the annotation task and your responses are saved. CODE-ABC123 Click to copy your completion code Return to Prolific — the code, a copy control, and a return link named after the provider. With `auto_redirect_on_completion: true` the worker is sent to the platform's completion URL instead, after `auto_redirect_delay`. I drove both: the redirect really does leave for the platform, so test with it off first unless you enjoy losing your browser session mid-check. The login page has two tabs, `login` and `register`, in one document — if you are driving it in a browser, switch with `switchTab('register')` before filling the register form. Both forms post `action` (`login` or `register`), `email` and `pass`; the fields are `#login-email`/`#login-pass` and `#register-email`/`#register-pass`. **The default authenticator keeps accounts in memory.** The boot log names the one you have: `Initialized UserAuthenticator with method: in_memory`. That method has no store behind it, so a username is only as durable as the process. `authentication` is the block that gives database-backed or SSO accounts; `secret_key` and `persist_sessions` cover the Flask session, which is a different thing. Decide this before you recruit rather than after the first restart. ## Serving it | Key | Effect | Default | |---|---|---| | `port` | Port; `-p` overrides | 8000 | | `host` | Interface to bind; `0.0.0.0` exposes beyond localhost | localhost | | `secret_key` | Flask session signing key, and the key that decides whether a login survives a restart: unset, every boot signs with a fresh random key and everyone is logged out | — | | `session_lifetime_days` | How long a session lasts. The login response carries it as an explicit cookie `Expires` | 2 | | `persist_sessions` | Asks for sessions to be kept across a restart; refuses to start without `secret_key` | false | | `admin_api_key` | Admin API key, sent as `X-API-Key`. Generated into `{task_dir}/admin_api_key.txt` when unset | — | **`debug: true` is for a server on your own machine, never for one an annotator can reach.** The same goes for `--debug` on the command line. On its own it still shows the login page. With `--debug-phase annotation` it signs every visitor in as one shared `debug_user` and drops them into the task, which is what makes it useful for checking the interface. It also opens the admin pages without a key to requests from the server's own machine, so an admin-auth check you run locally under debug proves nothing. On 2.11.1, with `--debug` on the default `0.0.0.0` bind, `/admin/api/overview` answered 200 on `localhost` and 403 on the machine's network address. 2.11.0 refused that bind outright ("Refusing to start: debug is enabled and the server is bound to 0.0.0.0 rather than loopback"), so on 2.11.0 pass `--host 127.0.0.1`. Binding `0.0.0.0` is how a researcher on the same network opens the task. Say so explicitly in a handover, along with the port, rather than leaving them to guess that localhost is not the only address.