Hardening plan — 21 ranked items, what is done, what is next¶
The catalog of the "Plex Storage Hardening" artifact (v9, 2026-09-08). Read this card for status; open the artifact only for the prose. Design system there is settled (IBM Plex, tokens) — edit content, don't redesign. Rebuild the local copy from the artifact's saved HTML; do not keep a divergent draft.
Status¶
| # | Item | Status |
|---|---|---|
| 01 | Scrutiny → ntfy alerts | done 09-02 |
| 02 | SnapRAID sync/scrub failures → ntfy (-l logs, OnFailure=snapraid-alert@) |
done 09-02 |
| 03 | Precheck mass-delete guard (DEL_LIMIT=500 UPD_LIMIT=2000) |
done 09-02 |
| 04 | content file + guard scripts into the rescue kit | done 09-02 |
| 05 | off-box copy of immich + EmbeddedEra | half done 09-08: a versioned restic repo now lives on the pool inside parity (history-repo) — history survives any one disk. The off-box half is a config change away: set B2_* + HISTORY_OFFBOX_REPO in ~/.config/restic/env and the nightly job restic copys to B2 (native, no rclone) |
| 06 | 1.2M stale files out of parity (recovered-* excluded; 45k files remain) | done 09-03; rm -rf of the 292 GiB deliberately deferred — verify the USB archive extracts first |
| 07 | ext4 reserve → parity 0 %, data 1 % (633 GiB freed) | done 09-03 |
| 08 | errors=remount-ro,noatime on all four (live + fstab + superblock) |
done 09-03 |
| 09 | WD idle3 head-parking disabled | done — written 09-03, latched at the 09-05 15:55 power cycle, 0 parks in 62 h |
| 10 | hd-idle on parity by serial (rests 31–34 °C, was 55) | done 09-02 |
| 11 | smartd: 5 devices, staggered long tests Tue/Thu/Sat 03:00, -M exec smartd-ntfy.sh |
done 09-03; tune2fs -c 30 on disk1 deliberately skipped (3.66M inodes) |
| 12 | autosave 100 + 4th content file /var/snapraid/ |
done 09-03 |
| 13 | mergerfs remount with corrected options | done 09-03 14:09 — see mergerfs-pool card for what was changed from the plan |
| 14 | cool the drive stack (add a 92 mm intake) | downgraded to maintenance by 10; no longer paired with 09 |
| 15 | parity reconstruction drill | done 09-09 — maint-drill: sync (no-op) → audit d3 0 errors in 173 min → file moved out, fix reconstructed it in 4 s, sha256 identical. Parity provably rebuilds. Report: map/reports/drill-d3-2026-09-09.md |
| 16 | re-link duplicate downloads↔library | partial — tranche A 8 pairs / 15.5 GiB done; tranche B: the 09-03 incident deleted most of its download copies — census 09-08 finds 20 same-disk size-matched candidates; scheduled: maint-dedupe 2026-09-11 08:00, auto-links on a clean scan |
| 17 | badblocks read-only sweep of disk1's defect band (fsck range 699,924,574–704,653,769) |
done 09-09 23:00 — 4.8M blocks read in 268 s, 0 unreadable, 0 kernel errors, Reallocated 0. The February band reads clean; nothing to reallocate. Report: map/reports/badblocks-disk1-2026-09-09.md |
| 18 | qBittorrent share limits | reverted — caused the 09-03 15:22 incident (791 GiB deleted from downloads, library untouched). Before retrying: removeCompletedDownloads=false in every *arr (now set); never a seeding-minutes limit; pin private torrents |
| 19 | replace parity HGST, then add 2-parity | open (hardware) |
| 20 | UPS | open (hardware); 54 of 73 NVMe power cycles unsafe |
| 21 | Scrutiny monitors /dev/nvme0 |
done 09-03 |
Not in the plan but noticed: the 09-03 sudoers grant — revoked by the user 2026-09-08. /var/log/docker-backup.log (832 MB, unrotated) — /etc/logrotate.d/docker-backup ships with ~/maintenance/install.sh.
Connected to¶
- joins: every storage card; processes/root-maintenance (how items were executed)
- See also: memory
plex-storage-hardening-report.md(session-level pointer; this card is the home)
See¶
- Source: the artifact URL in frontmatter · incident remediation script
~/fix-arr-remove-completed.sh