Engineering workflow audit — 2026-09-16¶
The foundation exists, but push-to-portal discovery and release approval are not dependable yet. The highest-priority fixes are check/release gating, board identity, and publication failure handling. The title block needs one validated metadata contract across the schematic, PCB, and release.
Scope and evidence¶
- Fresh clone of
EmbeddedEra/Project_Templateat67a287f4773d777c168eab58ee0333f068bbae56; local KiCad library atc85a79d514fd79b22218d07391620eb4bbbea8c8; compared Cow_Robot_Board, Cow_Remote, and Dashcam_Drive workflows. - Inspected
/home/angel/.local/bin/engineering-build,engineering-deploy, generated site, caches, cron, and deployment log; reviewed SOP-001, SOP-021, SOP-022 and template instructions. - Sonnet CLI performed a read-only template audit; the orchestrator checked consequential claims independently. Raw worker findings are supplementary and include errors corrected below.
- Exported starter schematic and PCB PDFs using the installed pinned
kicad/kicad:10.0.5image. Visually inspected both PDFs; extracted text from both PDFs. Reproduced checker failure behavior using an isolated mocked failing KiCad subprocess. - Inspected actual GitHub run 35106909797:
erc-drcfailed andfab-outputssucceeded. - Live domain returns a Cloudflare Access login redirect. Deployment log reports a completed deployment, but authenticated browser content was not independently inspected. This audit does not certify electrical correctness, legal wording, or manufacturing readiness. No production code, workflows, or hardware files changed during this audit.
Findings requiring correction¶
| Priority | Finding and evidence | Required outcome |
|---|---|---|
| P1 | Normal branch pushes do not trigger KiCad checks. Project_Template/.github/workflows/kicad-checks.yml:20 defines push.tags without push.branches. The README promises every hardware push. Recent Cow runs confirm KiCad runs on hardware tags while ordinary pushes run firmware CI. |
Explicit branch and tag triggers; include .kicad_pro, .kicad_dru, worksheets, manifest, fab settings, and checker scripts in relevant paths. |
| P1 | Cow workflows call an absent checker. Both Cow_Robot_Board and Cow_Remote lack .github/scripts/kicad_checks.py and manifest validation files. GitHub run 35106909797 failed with python3: cannot open .../.github/scripts/kicad_checks.py, exit 2. Dashcam_Drive has all three files. |
Distribute workflow dependencies with the workflow; verify on a clean checkout and require their success before release. |
| P1 | Failed KiCad execution can pass CI. .github/scripts/kicad_checks.py:44 converts missing/malformed reports to an empty list; lines 73 and 84 ignore subprocess return codes. Isolated reproduction returned exit 0 and a green row after both commands returned 1 with no reports. |
Fail on tool errors, missing/invalid reports, and absent expected inputs. Keep diagnostics; distinguish execution failure from valid design violations. |
| P1 | Fab releases bypass checks. Workflow fab-outputs:84 has no needs: erc-drc, and no manifest/metadata gate. Actual run above published outputs despite failed checks. |
Release job requires successful checks and release metadata validation for the exact tagged commit. Existing faulty releases need review, not silent retagging. |
| P1 | 004B remains misclassified despite the corrected manifest. engineering-build:104 adds a filename suffix when two .kicad_pro files share a folder; :287 matches the resulting label against manifest name/path basename. 004B has both Robot-Cow-004B.kicad_pro and leftover Robot Cow.kicad_pro; neither discovered label matches. Generated index places the valid board in Archive with product revision A. |
Use normalized manifest path plus an explicit project file as identity; keep display labels separate. Diagnose duplicate/stale project files. Re-test from discovery through generated index, not just the manifest helper. |
| P1 | Deploy failure is reported as success. engineering-deploy:30 ends the Wrangler pipeline with || true; line 31 saves the new hash regardless. Preview and main also share one hash file (:20). |
Propagate deploy exit status; save hash only after success; separate deployment state by branch and use isolated staging/locking. |
| P1 | Assembly PDFs omit the title block. Workflow lines 294–297 omit --include-border-title. An actual starter PCB export with those flags contains only board/silkscreen text. Installed KiCad help confirms the opt-in flag. |
Include and visually inspect border/title in top and bottom assembly drawings; verify bottom mirroring does not compromise the document frame. |
| P1 | Production schematic and PCB revisions disagree. Cow_Robot_Board/hardware/Robot-Cow-004B/Robot-Cow-004B.kicad_sch:7 says revision 004/date 2025-05-24; PCB :10 says 004B/date 2026-08-11; manifest says 004B. |
Resolve the intended revision with engineering evidence and enforce consistency before release. Preserve historical release identity. |
| P2 | Starter metadata looks like finished metadata. hardware/board-1/board-1.kicad_pro:785 defaults to Designer AGD, Part_Number 123, Revision 001; schematic/PCB dates are fixed at 2026-04-24, schematic title is only Schematic, PCB native title is absent. Export confirmed these values are printed. |
Explicit draft placeholders; require real product/board title, author, board IPN, revision and issue date before release. No automatic attribution to AGD. |
| P2 | Worksheet lookup differs across projects. 004B .kicad_pro:566,753 and Remote 006 :560,747 use ${KICAD_USER_TEMPLATE_DIR}; CI sets only ${EMBEDDEDERA_DIR} for the shared library. Finger 003 embeds a worksheet; current template uses library worksheet 003. |
Resolve one versioned worksheet in clean CI, or embed an explicitly versioned copy. Fail on missing worksheets; verify actual released PDFs. Exact fallback appearance for existing Cow releases was not tested. |
| P2 | Revision meanings are not reconciled. SOP-001 requires letter revision suffixes for PCB IPNs, SOP-021 release tags use A/B/etc., while manifest docs accept 4.0.1 and live boards use 004B/006. No check joins tag, manifest, project variables, native title blocks and silkscreen. | Document legacy design labels separately from the controlled release revision. For new boards use one revision consistently; retain an explicit mapping for legacy hardware instead of renaming history. |
| P2 | Manifest validation is partial and rollout incomplete. Template manifest_check.py:26 checks board paths/IPNs/status and product shipping evidence, but not required top-level identity, revision, firmware references, duplicate paths, or exact project triplets. It permits multiple project files. Product scaffold has no board status and reverses A/P examples in comments. |
Validate the documented contract; use explicit prototype status for a new board; copy or reuse the validator in each product repo. Do not infer a board status from missing data. |
| P2 | Current production mappings remain incomplete. Cow manifest :14–18 has shipping 004B with null IPN; :33 still maps firmware to retired 4.0.1. 004B has no fab.yml or matching release; existing 4.0.1 pack is historical. |
Confirm ERP identity and firmware compatibility; populate verified 004B manufacturing settings and generate its own gated release. |
| P2 | Portal results can look cleaner than they are. engineering-build:634–638 turns absent/null checks into zero and substitutes whole-repo totals if no board matches. :232 selects the latest CI run without matching branch/SHA. :355 can borrow another board's IPN/revision. |
Exact path and SHA matching; show unknown/failed/stale explicitly; never borrow another board's metadata or counts. |
| P2 | Portal discovery errors can publish an incomplete index. engineering-build:57 treats GitHub failures as empty lists, caps repos at 100, does not inspect recursive-tree truncation; :83 keeps stale checkout after failed pull; :653–656 still returns success. |
Fail publication on incomplete discovery or freshness failure; preserve the last good site, paginate, and expose build health. |
| P2 | Discovery and manifest contract disagree. Builder scans all .kicad_pro files rather than declared boards, requires same-stem PCB/schematic, and maps missing status to Archive. Multiple same-named folders can collide. find_sheet_files:269 copies only immediate schematic files. |
Manifest-led stable board identity and explicit discovery policy; diagnose missing pairs and support referenced hierarchical sheets with preserved relative paths. |
| P2 | Reproducibility depends on mutable shared inputs. CI KICAD_LIB_REF:15 is master, git-lfs download is latest, and YY1 jobs use the library's current feeder setup. Portal cache keys include product SHA but not library, worksheet, renderer or helper revisions. |
Pin and record tool/library/worksheet/setup versions; include them in cache keys and release provenance. Preserve the generated feeder-state snapshot. |
| P2 | Check evidence upload points to the wrong directory. Workflow passes --repo .../hardware at line 74; script writes reports under that root at line 60, but upload at workflow line 81 targets _kicad_reports/ at repository root. |
Upload hardware/_kicad_reports/ or standardize the output argument; missing expected evidence must be visible. |
| P2 | Library errors are suppressed globally. Checker ENVIRONMENT_NOISE:19 removes library-link and footprint-mismatch violations even after CI configures library tables. board-1 remains a fragment exemption by directory name. |
Separate infrastructure diagnostics from design waivers; reject placeholder/fragment projects for manufacturing releases and report library conformance explicitly. |
| P3 | Cache repair and cleanup are incomplete. engineering-build:574–588 retries failed IBOM only if renders also need rebuilding. Existing board directories and copied source files are not pruned when boards disappear. |
Retry each failed artifact independently; build the output tree cleanly, retaining only deliberately published history. |
Workflow files are currently byte-identical in the four audited repositories. Copy drift is a future risk, not an observed mismatch. Pinning a reusable workflow or adding a synchronization check is preferable to maintaining four independent implementations.
GitHub confirms that tags-only filters exclude branch pushes and path filters are not evaluated for tag pushes: official workflow syntax. The preliminary worker's contrary tag/path claim is rejected. Its claim that Cow already runs a manifest-check workflow is false: both Cow repos lack it; Dashcam_Drive has it. The Haiku follow-up also inspected some workspace-relative paths incorrectly; the orchestrator corrected this using absolute paths and GitHub failure logs. A directory containing a revision is not itself a tag-parser collision; the real issue is unvalidated revision semantics.
Title-block contract to adopt¶
Use the existing EmbeddedEra visual identity and one versioned worksheet. Native title/revision/company fields should remain meaningful even when another viewer uses a default worksheet. Project text variables supply shared values; validation checks the manifest against them. KiCad supports these mechanisms directly: KiCad 10 drawing-sheet documentation.
| Field | Rule |
|---|---|
| Company | EmbeddedEra logo and confirmed company name; preserve the current ownership notice until its owner approves wording changes. |
| Title | Product — board name; distinguish schematic from assembly drawing. |
| Drawing/part identity | Board IPN from ERP, not the product assembly IPN; visible on both document types. |
| Revision | Same controlled board revision in manifest, schematic, PCB, relevant silkscreen and release tag; record legacy labels separately. |
| Issue date | Explicit document/release date, not a template date or an automatically changing print date. |
| Designer | Actual author identity; no default AGD or invented credentials. |
| Checked/approved | Actual reviewer and approval record for release; blank or Draft before review. |
| Customer | Correct spelling, preserving a temporary Costumer alias only during migration. |
| Classification | Visible confidentiality/classification field and approved ownership notice. No passwords/tokens or unsupported compliance claims. |
| Sheet | Native sheet name and page/total; consistent on all hierarchy pages. |
| Provenance | Source commit and library/worksheet versions in release manifest; compact commit reference on drawings if useful. |
The current schematic PDF already renders the logo, notice, designer, part number, revision, date and sheet numbering correctly when the library variable resolves. Reuse that foundation. Add required metadata and validation rather than redesigning it from scratch. Provide a generic empty-board template alongside the STM32 reference design so a new circuit need not inherit that MCU.
Intended workflow and acceptance checks¶
- Create a private product repo from Project_Template. Provision read access to the pinned shared library and register explicit board path/project, name, prototype status, IPN when assigned, revision and firmware compatibility. Keep one canonical project triplet per board.
- A normal default-branch push triggers metadata validation and ERC/DRC. Publish the working design with its exact commit and honest check status, even if still in development; publication is not manufacturing approval.
- Retain hourly portal polling as a recovery mechanism. Add an authenticated post-push/build trigger if faster visibility is required; do not expose server credentials or add a second deployment stack unnecessarily.
- A hardware release tag must match approved board metadata and pass checks, worksheet/PDF verification, manufacturing-spec validation and required review. Generate immutable fab outputs with provenance; never relabel another board's pack.
- Build the portal in isolation, validate completeness, deploy, and only then update branch-specific success state. Preserve the previous site on discovery/build/deploy failure.
- Acceptance: a new prototype board appears without manual page editing; 004B stays Production even with a rejected stale project file; all drawings agree on identity; invalid/missing checks cannot pass; a failed check blocks fab output; a failed deployment retries; a library-only change invalidates affected cached artifacts; removing a board removes stale unpublished output.
Suggested implementation order¶
- Correct push triggers, checker failure handling, report upload, and release dependencies together; verify with isolated positive and negative cases.
- Repair manifest-based portal identity and 004B classification, unknown-check handling, and fail-safe publication.
- Standardize title-block fields/worksheet resolution; render schematic plus both assembly PDFs and review them before rolling out to real boards.
- Strengthen and distribute manifest validation, resolve legacy revision/IPN/firmware mappings, and pin dependencies and provenance.
Audit artifacts in /home/angel/Documents/Codex/2026-09-16/home-angel-notes-projects-engineering-portal/audit/: starter-schematic.pdf, starter-schematic.png, starter-pcb.pdf, extracted PDF text, checker-failure-reproduction.json, and worker reports in this directory. No full ERC/DRC sweep or new cloud release was run. The KiCad library's pre-existing scripting/partlookup.py modification was left untouched.
Delegation: one Sonnet audit followed by one Haiku verification; combined CLI-reported estimated cost $0.80, not a billing statement. Persistent preferences saved in /home/angel/.codex/AGENTS.md using the documented global instructions location.