Sandbox workflow — branch the code, snapshot the data¶
Decided 2026-09-12 with Angel: "a different branch… for the sandbox, so we can experiment and then we merge."
The rule¶
Code is branched. Data is snapshotted. Nothing is tested on production.
| Thing | Where it lives | Sandbox | Promote to production |
|---|---|---|---|
| Assistant | ~/assistant (branch main) |
worktree ~/assistant-sandbox on branch sandbox, service assistant-sandbox.service on 127.0.0.1:8201, talks to the sandbox ERP |
git merge sandbox on main, systemctl --user restart assistant.service |
| ERP print templates and plugins | ~/erp-config (templates/, plugins/) |
branch sandbox, install with INVENTREE_URL=http://127.0.0.1:8300 … erp-templates |
merge, then erp-templates against production |
| Home page | ~/embeddedera-home |
home-deploy sandbox → gated preview sandbox.embeddedera-home.pages.dev |
home-deploy |
| SOP site | ~/Notes/sops (the vault) |
sop-deploy preview publishes the working tree to a gated preview before it is committed |
commit, sop-deploy |
| KiCad library | EmbeddedEra/KiCad |
a branch in that repo; the sync tool only ever writes to a checkout you point it at | merge to master |
| ERP data | production InvenTree | erp-sandbox up --fresh — a copy restored from a backup taken right now, on 127.0.0.1:8300 and http://sandbox.lan on the shop network (NPM, password login — the admin password is printed by erp-sandbox up), named SANDBOX, never exposed through the tunnel |
never — data does not flow back; you redo the real change on production by following the SOP |
Why data does not merge¶
A database is not a document. Two people editing production while a sandbox drifts produces conflicts nobody can reason about, and a "merge" of stock levels is meaningless. The sandbox exists to prove that a change to code or configuration behaves, on real-shaped data. Once it does, the code merges and the data change, if any, is done once on production through the ordinary procedure.
Freshness¶
erp-sandbox reset takes a new backup first, so the sandbox is minutes old, not a night old. The
backup it makes is a real backup set and lands off-site like any other — resetting the sandbox is
also a free extra backup.
What the red-team suite runs against¶
The sandbox assistant on 8201, backed by the sandbox ERP on 8300. Production is never a test target.
Related: roles-and-skills, inventree.
Two modes, from 2026-09-12¶
| Mode | Command | Data | Who may see it |
|---|---|---|---|
| rehearsal | erp-sandbox reset (or --fresh) |
a real production copy, minutes old | EmbeddedEra only |
| demo | erp-sandbox reset --demo |
the same shape, fictional contents | anyone invited, including outside evaluators |
--demo renames every company from a fixed pseudonym list, deletes contacts, addresses and every
attachment (rows and files), blanks notes and non-distributor links, multiplies every price by a
different random factor per part so costs cannot be recovered, renumbers orders to SO-9001…, and
replaces the users with one demo@example.com superuser. Categories, part numbers, descriptions,
BOMs, stock, templates and plugins survive — a visitor must see a working shop.
erp-sandbox status reports the mode; the instance name reads DEMO — fictional data, resets often;
the sandbox assistant shows a matching banner. erp-sandbox verify-demo is the gate before any
invitation: it cross-checks company names and prices against production (read-only), and fails if
an attachment row or a real email address survives. Never invite anyone without it passing.
Guard by construction: the scrub refuses to run unless the target is port 8300 and the compose project is the sandbox one, so it can never point at production.
Task board (decided 2026-09-15, task 84)¶
The sandbox does not get its own Kanboard. There is one register, tasks.embeddedera.com, for production and sandbox work alike; sandbox code that needs a throwaway task creates it in the Unassigned project and removes it after the test (the pattern every test this week used).
One board or two — the comparison behind that decision¶
Written out on 2026-09-18 because the decision was recorded without it. No dollar figures and no invented measurements: the shop has not metered a Kanboard container, so this compares the work and the surface area, which is what the choice actually turns on. Facts about the live instance come from kanboard.
| One shared board (what we do) | A second sandbox board | |
|---|---|---|
| Infrastructure | Two containers, kanboard (SQLite, no database service) and kanboard-proxy, on the private kanboard-net bridge, publishing 127.0.0.1:8400 only. |
Two more containers, a second private bridge and a second loopback port. On a 16 GB box where Postiz's six containers were already turned off once for memory, another always-on pair is not free even though its own footprint is small. |
| Storage | One SQLite file plus data/files/, already inside the nightly erp-backup set (taken with VACUUM INTO from inside the container). |
A second SQLite file and attachment tree. Either it joins the backup set — protecting data whose whole purpose is to be thrown away — or it is deliberately excluded, which is one more exception somebody has to remember. |
| Reachability | One Cloudflare Access app ("EmbeddedEra Tasks"), one tunnel ingress, one DNS name, the tasks token in departments.yml. |
A second hostname to gate, or a LAN-only NPM host like the sandbox ERP's. Either way it is another door to keep shut and another entry in the access map. |
| Setup | Done. | A kanboard-sandbox analogue of erp-sandbox: bring up, restore db.sqlite, and then re-apply everything that does not live in the database file — the API token (the card already warns it may not survive a restore), the custom stylesheet, which is stored in the application's own settings rather than in a file, the patched Smtp.php, and the FPM env.conf that routes the mail key. |
| Updates | One image to bump, and one re-check of the upstream SMTP patch, which has to be redone after every Kanboard upgrade. | Two of each, and a standing risk that the two drift — the worse failure, because a test then passes against a board that is not the board. |
| Backups and reset | Nightly, and no reset story is needed. | A reset path has to exist and be trusted, or the sandbox board slowly becomes a second production board nobody dares clear. |
| Tooling | Every tool reads one ~/.config/kanboard/env: _kanboard.py, onboard kanboard-sync, tasks-migrate, tasks-export, tasks-notify, tasks-digest, tasks-metrics, task-verify, sop-tasks, part-watch. |
Each of those needs a target to point at, and every future one inherits the obligation. That is the cost that keeps growing. |
| Isolation of task data | None between "real" and "test" rows beyond naming and the Unassigned project: a test task is a real row on the real board until it is deleted, and it is visible to anyone looking, and to the nightly tasks.md export if it is still there at 23:55. |
Complete. Test rows never touch the register. |
| Permissions | None. Kanboard's projects, groups and roles are instance-wide; onboard kanboard-sync drives them from departments.yml and people.yml. A permission experiment changes what real people can see. |
Complete, and this is the strongest argument for a second board. |
| Notifications | None. Mail goes out through Resend to real inboxes; a test that triggers a notification has already delivered it. | Complete. |
| What it avoids | A second instance to install, gate, monitor, patch, upgrade, back up and reset — and the drift between them. | The four risks above. |
Why one board is the right answer for phase 1¶
Phase 1 of task-board is one list of tasks per department, provisioned from the files we already
keep. Almost everything built on it — sop-tasks, task-verify, part-watch, the exports and the
digests — is exercised the way the house tests everything else: offline, against a fake kb.rpc,
with fixtures. Those suites never need a board at all, which is why the second instance would sit
idle most weeks while still demanding patching and upgrades. The register is also the one place
where a stale copy is actively harmful: a second board invites someone to read the wrong one, and
tasks.md in the vault is generated from exactly one.
It is also cheap to reverse. Kanboard is two containers and a SQLite file; the exit path back to the
assistant's tasks table is already documented and tasks-export --sqlite still works. If the day
comes that a permission or notification change genuinely has to be rehearsed live, that is the
trigger to revisit this — not a reason to pre-build the instance now.
What we give up, stated plainly: there is no isolation for permission changes, for notification changes, or for anything instance-wide. Those are tested offline with fixtures, or they are done once on production, carefully, by a person, with the change written down first.
The exception, and its edges¶
The house rule says nothing is tested on production. The recorded exception — a throwaway task in the Unassigned project, removed after the test — sits inside it rather than against it, because the rule's subject is data: code is branched, data is snapshotted. A task row is disposable, is addressed by its own id, belongs to nobody, and its deletion is complete. Creating and deleting one is authorised and stays authorised.
It authorises nothing else. In particular it does not cover:
- changing projects, columns, swimlanes, groups, roles or any user's access — these are instance-wide and land on real people;
- anything that sends mail, because Resend delivers it and deleting the task afterwards does not unsend it;
- installing a plugin, editing
config.php, the Caddyfile or the mounted patches, or upgrading the image; - writing to, reopening, moving or bulk-editing tasks that somebody actually owns.
Deleting the test task afterwards is cleanup, not isolation. Isolation is a property of the blast radius before the write, not of the tidying after it; a permission change is already in effect and a notification is already delivered by the time the row is removed. Anything in that list is tested with fixtures, or it is a deliberate production change made by a person under the relevant SOP.
Task 84 was closed by the register on 2026-09-18 once this comparison was published and attached to the card. The decision itself is unchanged from 2026-09-15.
On-demand ERP lifecycle — 2026-09-22¶
Owner decision: the ERP sandbox stays stopped except during testing. Bring it up only when a test needs ERP, verify health, and stop after the final test, including failure/cancellation cleanup. Coordinate concurrent test owners before stopping. Offline tests need no ERP. Never fall back to production.
Routine lifecycle preserves containers and volumes:
docker compose --project-name inventree-sandbox --project-directory /home/angel/torrentvpn/inventree-sandbox -f /home/angel/torrentvpn/inventree-sandbox/docker-compose.yml start
# Wait for all five sandbox containers to be healthy before testing.
# For development-store integration tests only:
systemctl --user start ee-store-order-sync.timer ee-store-catalog-observe.timer
# Run the authorized tests; always perform cleanup, even on failure.
systemctl --user stop ee-store-order-sync.timer ee-store-catalog-observe.timer
systemctl --user stop ee-store-order-sync.service ee-store-catalog-observe.service
docker compose --project-name inventree-sandbox --project-directory /home/angel/torrentvpn/inventree-sandbox -f /home/angel/torrentvpn/inventree-sandbox/docker-compose.yml stop --timeout 30
The two development-store ERP timers are disabled at boot; start them temporarily only for integration testing. Store browsing and the unrelated Cloudflare purge timer remain available. The assistant sandbox service is separate and remains available for tests that do not need ERP.
Do not use erp-sandbox down for routine cleanup: it removes volumes. erp-sandbox up is a restore workflow, not the resume command for a retained stopped instance. Use intentional fresh restore/reset only when the test requires it, following the existing snapshot rules above.
Inspection also found stale red-team documentation: the installed assistant-test currently starts a separate assistant on 8202 and sources the main assistant environment, rather than using 8201. It was not run or changed as part of this shutdown. Audit its isolation separately before treating it as a sandbox test.
erp-sandbox up on a stopped sandbox — 2026-10-01¶
up writes a new random DB password into the sandbox .env, but a retained (stopped) db volume keeps the old one, so invoke update failed with pg_dump "password authentication failed for user inventree". up now refuses when inventree-sandbox_db exists and prints the resume command (docker compose ... start); use erp-sandbox down first only if you want to discard the data. Routine stop/resume is unchanged.