Skip to content

Tester server: quarantine without a serial (server brief)

Decided with Angel 2026-10-21. Written for the session that works on the tester API and the ERP (~/assistant, Plex). Station-side changes are a separate piece of work on Angel's Mac (Universal_Tester, Cow Robot station) and are listed at the end only so the contract is clear.

The decision

A permanent serial (MMYY####) is issued only when a board first PASSES, never for a failed board and never before a test. A failed board is identified by its chip ID (when readable) and the run ID. Once the operator chooses Quarantine, the case is identified by a Quarantine ID (QID). The Citizen receipt and Zebra bag label carry the QID, chip ID and failure, with no serial. The Brother serial sticker prints on PASS only. A quarantined board that is repaired and later passes gets its serial at that pass, and the serial attaches to its existing quarantine case.

Why (measured on the station logs, 2026-09-26 to 2026-10-21)

62 attempts, 27 failures; all 27 were given a serial and 13 had a serial sticker printed on a failing board. Of the 16 boards that failed, 12 later passed on a retest, so most failures are seating or flaky contacts and the serial was spent too early. Example: board 10260016 failed with "DFU did not enumerate" and still burned a serial and a sticker. Gaps are allowed by the existing decision, but stickers on boards that never passed, and a serial per retest attempt, are noise.

What the server does today (from the station client, server_client.py)

  • GET /tester/v1/serial?product=&chip_id= returns the chip's serial; serial_assigned events issue and link serials; the station draws from a reserved pool.
  • Event quarantine with action quarantine|work|keep|release|scrap. For quarantine the station currently sends serial, chip_id (optional), stage, run_id, related_event_id, failure_summary, record. The server mints the QID and creates a case; the station then calls GET /quarantine and refuses to print or move the board until the case has erp_status == "done".
  • GET /quarantine by quarantine_id, or product + serial, or product + chip_id; the case carries quarantine_id, product, serial, chip_id, disposition, original_stage, events, dispositions, erp_status, erp_error.
  • test_run events are keyed by chip_id; a run with no identity is sent as UNIDENTIFIED-<run_id> with chip_id_synthetic. Per the decision note, runs sent before a serial exists attach to the serial once it exists.
  • GET /next-stage takes serial or chip_id.
  • The ERP creates the unit "as one unfinished build output" at a pre-coating PASS or an explicit Quarantine disposition, and that output is serialized. This is the part that needs a decision below.

Required server changes

  1. Open a quarantine case with no serial. Accept quarantine events that have run_id, stage and optionally chip_id, and no serial. Mint the QID. Store serial = null. Idempotent on event_id (retries reuse it) and on run_id. At most one active case per chip ID when a chip ID is present. Keep accepting events that still carry a serial (old clients, already-queued outbox, and every case that exists today).
  2. Lookups with a null serial. GET /quarantine must work by quarantine_id and by chip_id for serial-less cases, and return serial: null. By serial, a case that later received one must be found. Keep the response shape otherwise unchanged; the client checks product, the identifier it asked for, and quarantine_id.
  3. ERP representation of a serial-less quarantined board (needs your design, prove it on the sandbox ERP first). Recommended: create an unserialized stock item in the Quarantine location, quantity 1, batch code = the QID, for the pretest assembly part, consuming only the materials physically installed at pretest (same staging rule as today). When the board later gets a serial, serialize that existing stock item (InvenTree has a stock serialize action; confirm the exact endpoint and that it keeps the build-output link, the test results and the location history). SERIAL_NUMBER_GLOBALLY_UNIQUE stays on. If the sandbox shows serializing a build output is not supported, the fallback is to not create the ERP unit at quarantine and record the case only in the tester database, with the consumed materials reported as scrap by QID on scrap; say which you chose and why. Either way erp_status must reach "done" for a serial-less case, otherwise the station will never print.
  4. Attach the serial when it arrives. A serial_assigned event for a chip that has an open serial-less case (or a link by quarantine_id/run_id for an unreadable-chip board) sets the case's serial, serializes the ERP item (point 3) and re-attaches the earlier chip-keyed test_run history to the serial. Scrapped boards that never pass simply never get one; a scrapped board does not consume a serial.
  5. Unreadable chip. The failed run arrives as UNIDENTIFIED-<run_id> and the case is identified by its QID and run_id alone. Stop depending on the "label-only serial" path for new boards (the station will stop sending reserve_label_serial); keep reading the old ones. When a chip becomes readable during rework the station links chip_id to the case via the QID.
  6. /next-stage for a board with no serial. Accept chip_id of a board with no serial yet and return the right stage (first test = pre-coating) or a "blocked: open quarantine case " answer. It must not create a serial as a side effect.
  7. Reporting. The Grafana/reports view should show quarantine cases with a null serial, keyed by QID, and include them in the work-order reconciliation (boards physically assembled = passed + quarantined + scrapped).

Acceptance checks (run on the sandbox, never on production)

  1. Failed run, chip readable, Quarantine chosen: case created with QID, serial null, erp_status done, no serial consumed from the pool.
  2. Same board repaired, retested, PASS: serial_assigned arrives, the case now shows the serial, the ERP item is serialized and still in/after the quarantine history, earlier failed runs are on it.
  3. Failed run, chip unreadable: case by QID and run_id only; later chip linked; later PASS gets a serial.
  4. Quarantine then scrap: reason and confirmation required, materials stay consumed, no serial ever issued, serial pool count unchanged.
  5. Old-style events (with serial) and the 6 boards already quarantined still read and work.
  6. Retry of the same event_id returns the same QID; two different failed runs of one chip do not create two active cases.
  7. Server unreachable: the station still queues and refuses to print (unchanged); no behaviour depends on a serial existing for a failed board.

Station side (Angel's Mac, for contract awareness only)

  • Assign the serial only after a PASS (after shutdown, where the Brother label prints today); no assignment before flashing or in the fail cleanup; Brother sticker on PASS only.
  • Quarantine event sent without serial; Citizen receipt and Zebra label print QID, chip ID and failure after the server confirms (erp_status == "done").
  • Rework lookup by QID or chip ID (already supported); a rework PASS assigns the serial and links it.
  • Remove the reserve_label_serial path for new boards.

Rules for this work

Sandbox first, against the sandbox ERP (erp-sandbox up); see ~/Notes/decisions/sandbox-workflow.md. Nothing is tested on production. When done, update ~/Notes/decisions/tester-serial-service.md (the "Two stages, one serial" bullet is amended 2026-10-21) and the tester API card in the map, run notes-sync, and tell Angel what the station side must now send.

Handoff 2026-10-21 (chat 1 → chat 2)

Start the next chat with: "read the handoff in ~/Notes/projects/tester-quarantine-no-serial-server-plan.md and continue".

State

  • Nothing implemented yet. Orientation, representation proof and reporting branch are done.
  • Code worktree: ~/assistant-tester-quarantine, branch sandbox-tester-quarantine (from main 4dd8f8c, clean). The old sandbox branch was stale, so this replaces it; tell Angel. Files: tester_api.py (1576 lines), tester_inventory.py (unchanged), tests/test_tester_api.py (2064 lines). Baseline: 127 passed (24 s) with /home/angel/.local/share/assistant-venv/bin/python -m pytest -q tests/test_tester_api.py.
  • Reporting branch: ~/erp-config-qid-reports, branch tester-quarantine-reports, commit 0e3d871 (worker-made, not pushed). Needs a diff review (git -C ~/erp-config-qid-reports show 0e3d871) before relaying; do not push or deploy.

Decision made (proven on the sandbox ERP)

Representation A: an unserialized build output (qty 1, batch_code = QID, no serial_numbers) in the Quarantine location (status 75). The serial is attached in place with PATCH /api/stock/{pk}/ {"serial": ...}: same pk, build link, test results and tracking history preserved; global uniqueness enforced; scrap and complete still work. Fallback B (no ERP unit until first PASS) was not needed.

Design to implement (tests first, fail on unmodified code, then pass)

  1. Migration: quarantine_cases.serial becomes nullable (SQLite cannot ALTER NOT NULL: rebuild the table; drop the indexes first; run the migration before executescript(_SCHEMA) on an old table). NULLs are distinct in unique indexes, so keep idx_quarantine_chip_active (product,chip_id) and add an active index on (product, run_id) or QID. serial != ? and serial=? are NULL-unsafe: use IS.
  2. _apply_quarantine: accept no serial; mint QID; serial=NULL; idempotent on event_id and run_id; one active case per chip; still accept serial-bearing events.
  3. Serial attach: serial_assigned for a chip with an open serial-less case sets the case serial and re-attaches earlier chip-keyed test_run history. New link event by quarantine_id/run_id plus chip_id/serial for unreadable-chip boards. A scrapped case never consumes a serial.
  4. ERP mirror: serial-less _create_build_output (no serial_numbers, batch_code=QID); on attach PATCH the serial. Test that a rework PASS creates no second output (same pk, one output) and that failed-run test results land on the QID item.
  5. GET /quarantine: by quarantine_id, chip_id, serial, run_id; return serial: null when unset. /next-stage: accept chip_id with no serial, return the first stage or "blocked: open quarantine case ", never create a serial.
  6. _close_finished_builds: include serial-less cases (union).
  7. Tests: extend FakeERP (create-output without serial_numbers, serial PATCH reflected in /api/stock/?serial= with uniqueness, build+batch lookup, serial/batch in stock detail); replace the legacy-schema slice in test_quarantine_migration_preserves_legacy_database with a frozen string; assert serials row count unchanged in checks 1 and 4; update permanent_serial_required (tester_api.py:509).

Sandbox acceptance run (checks 1-6; 7 reported as "server side: nothing requires a serial; station side not exercised")

  • The sandbox has two Production builds for part 181 but _find_single_open_build needs exactly one: create a dedicated sandbox test part copying part 181's flags (trackable, testable, assembly, same required test templates) with one Production build, plus a test products.yml pointing at it.
  • Separate tester-api instance on another port with a scratch DB. Check 5 uses a read-only backup of the production DB: sqlite3 'file:/home/angel/.local/state/tester-api/tester.sqlite3?mode=ro' ".backup <scratch>". Never touch the production DB. Sandbox ERP is already up; never erp-sandbox down.

After the code

Commit (trailer Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>), call advisor, update ~/Notes/decisions/tester-serial-service.md (representation and why, new contract, acceptance results), find and update the tester API card in ~/Notes/map/ (check processes/ and effects/; objects grep only found tester in inventree, grafana, torrentvpn-stack, jobs, schedules), run map/_meta/build-index.sh if a card changes, append a Log line here, notes-sync.

Tell Angel what the station must now send

Quarantine event without serial (with run_id, stage, optional chip_id, failure_summary, record); poll GET /quarantine until erp_status == "done" before printing; serial_assigned only after PASS (or a link event for an unreadable chip); rework lookup by QID or chip; stop reserve_label_serial for new boards.

Faster next time

Write the failing tests and the server changes in two parallel worker helpers only if file ownership is split (tests vs tester_api.py); otherwise one chat, targeted pytest (-k quarantine) while iterating and the full suite once at the end.

Handoff 2026-10-21 (chat 2 → chat 3)

Start the next chat with: "read the handoff in ~/Notes/projects/tester-quarantine-no-serial-server-plan.md and continue".

State

  • Code done and committed, not pushed: ~/assistant-tester-quarantine, branch sandbox-tester-quarantine, commits 414651f and 92bf587 on main 4dd8f8c. 143 tests pass (/home/angel/.local/share/assistant-venv/bin/python -m pytest -q tests/test_tester_api.py). The old sandbox branch was stale and is replaced by this one; tell Angel.
  • Sandbox ERP is up (erp-sandbox up took ~3 min; status was "down" at the start of this chat, restored from stamp 2026-10-21_0315). Never erp-sandbox down.
  • Acceptance script written, not yet run to completion: /tmp/claude-1000/-home-angel-claude-remote/74baa72d-5f2f-53b6-8a8f-46c7704d87e4/scratchpad/accept.py (checks 1-4, 6, plus chip_replaced by QID and next-stage; ~40 assertions). Alongside it: env.sh (scratch env), stations.yml (station sbx, key sandbox-accept-key), products.yml (copy of the real one). The scratch server was stopped; start it with . env.sh; cd ~/assistant-tester-quarantine; python -m uvicorn tester_api:app --port 8211 (use the assistant venv python). Run the script directly (not through | tail, it takes minutes); delete scratch.sqlite3* first.
  • Sandbox set up so far: part 181 had no Production build after the restore (BO-0002 pk 3 was Pending, BO-0001 complete). I issued BO-0002 (POST /api/build/3/issue/), so it is the single Production build for part 181.

Finding that blocked the first run (not a code bug in the new feature)

Every serial-less quarantine event stayed erp_status=pending with part 8: missing explicitly available build allocation (1.0); allocate stock explicitly. That is the existing pretest material consumption (tester_inventory.py:145): the build needs explicit stock allocations on its lines before a pretest can consume them. BO-0002 has none. A first partial run left two QID outputs in build 3 (chips CHIP-A-c7e63d / CHIP-C-c7e63d, one got serial 10260001/10260002 in the scratch DB), so reset the sandbox first for a clean run: erp-sandbox reset (about 3 min), then POST /api/build/3/issue/, then allocate stock to the build lines (try POST /api/build/3/auto-allocate/ {"interchangeable":true,"substitutes":true,"optional_items":true}; the pretest parts are the BOM lines not in final_ipns/coating_ipn of cow-robot in products.yml), delete the scratch DB, restart the scratch server, run accept.py. - The allocation need is the same on production, so it says nothing against the feature; but note it for Angel: a serial-less quarantine waits (pending) until the build has allocated pretest stock, and the station will not print until then, same as before for serial cases.

Still to do

  1. Clean acceptance run as above; fix anything real it finds (the unproven spots: batch list filter, pagination in _erp_rows for many outputs, PATCH-serial flow end to end, material consumption landing on the QID output and staying consumed after scrap).
  2. Check 5: migration on a read-only copy of the production DB: sqlite3 'file:/home/angel/.local/state/tester-api/tester.sqlite3?mode=ro' ".backup <scratch>", then open it with TESTER_DB=<scratch> via tester_api._db(); compare row counts of the quarantine tables before/after, PRAGMA integrity_check, PRAGMA foreign_key_check, and read the 6 existing cases through GET /quarantine. Never touch the production DB.
  3. Call the advisor on the sandbox results.
  4. Review git -C ~/erp-config-qid-reports show 0e3d871 (branch tester-quarantine-reports); do not push or deploy.
  5. Notes: update ~/Notes/decisions/tester-serial-service.md (Representation A and why, new contract, acceptance results, known gaps); update the tester API card in ~/Notes/map/ (look in processes/ and effects/), run map/_meta/build-index.sh if a card changes, add a Log line to this note, notes-sync.
  6. Report check 7 as "server side: nothing requires a serial; station side not exercised".

Tell Angel (final message)

Station must now send: quarantine without serial (with run_id, stage, optional chip_id, failure_summary, record; omit chip_id when synthetic, the server also ignores UNIDENTIFIED-*); poll GET /quarantine (by quarantine_id, chip_id or run_id) until erp_status=="done" before printing; serial_assigned only after PASS; a quarantine event with action:"link" (quarantine_id or run_id plus chip_id and/or serial) for an unreadable chip; chip_replaced by quarantine_id for a serial-less case; rework lookup by QID or chip; stop reserve_label_serial for new boards; do not require the returned run_id to equal the one asked for (a lookup by a later linked run returns the case that run opened); release needs a serial (serial_required_for_release). Design choices to flag: link is a quarantine event with action:"link"; a repeat quarantine for an open chip or run is an accepted no-op returning the same case; a serial-less create requires run_id; serial_assigned for a scrapped board is rejected (board_scrapped); a late PASS test_run on a scrapped board is still recorded and posted on the scrapped item (judged harmless).