Continues the per-cluster Description-clarity work after #382 (dpi),
#383 (dns+threats), #385 (identity+monitoring). Fixes the remaining
cluster-member placeholder headlines across the AI, DPI, mesh, and
SOC clusters:
AI cluster:
- secubox-ai-gateway: "Ai-gateway Module" → "data-sovereignty
router for AI requests"
- secubox-mcp-server: "Mcp-server Module" → "Model Context
Protocol server for AI tools"
DPI cluster (completes #382's work):
- secubox-ndpid: "nDPId Module" → "DPI dashboard with JA3/JA4
TLS fingerprinting"
Mesh cluster (completes #384's work):
- secubox-meshname: "Meshname Module" → "Yggdrasil-overlay DNS
resolver"
SOC cluster:
- secubox-soc: "Soc Module" → "Security Operations Center
aggregator"
Each Description now cross-references the sibling packages in its
cluster so an operator running `apt-cache show <pkg>` can pick the
right one — same pattern as #382, #383, #385.
Side fixes:
- Maintainer corrected from "SecuBox <dev@secubox.local>" to
"Gerald KERMA <devel@cybermind.fr>" on ai-gateway, mcp-server,
ndpid (three of five had the placeholder).
- Distribution corrected from "stable" → "bookworm" on ndpid
(initial-release stanza had the wrong distribution).
Patch-version bump on each so apt sees the metadata change:
- secubox-ai-gateway: 1.0.0 → 1.0.1
- secubox-mcp-server: 1.0.0 → 1.0.1
- secubox-ndpid: 1.0.0-1 → 1.0.1-1~bookworm1 (also adopts the
project-standard ~bookworm1 suffix)
- secubox-meshname: 1.0.0 → 1.0.1
- secubox-soc: 1.0.1 → 1.0.2
10 files, +104/-15. No code/endpoints/services/menu changes. All
changelogs parse cleanly.
8 placeholder Descriptions still remain on master: backup,
config-advisor, gitea, iot-guard, localrecall, nextcloud, tor,
vortex-firewall. These are standalone or LXC-service packages
(don't-merge list members), file separately if pursued.
Continues the per-cluster Description-clarity work started in #382
(dpi) and #383 (dns+threats). Fixes 4 more "X Module" placeholder
headlines in the identity and monitoring clusters:
Identity stack:
- secubox-identity: "Identity Module" → "decentralized identity
management"
- secubox-users: "Users Module" → "local account manager
(Argon2id + TOTP + RBAC)"
Monitoring cluster:
- secubox-watchdog: "Watchdog Module" → "LXC + service + HTTP
health monitor"
- secubox-device-intel: "Device-intel Module" → "network asset
discovery + fingerprinting"
Each Description now cross-references the sibling packages in its
cluster so an operator running `apt-cache show <pkg>` can tell at a
glance what to install for what — same pattern as #382 and #383.
Also corrected the Maintainer on secubox-identity from the
placeholder "SecuBox <dev@secubox.local>" → "Gerald KERMA
<devel@cybermind.fr>".
Patch-version bump on each so the metadata change ships in apt:
- secubox-identity: 1.0.0 → 1.0.1
- secubox-users: 1.4.0 → 1.4.1
- secubox-watchdog: 1.0.0 → 1.0.1
- secubox-device-intel: 1.2.1 → 1.2.2
8 files, +85/-9. No code/endpoints/services/menu changes. All four
changelogs parse cleanly via dpkg-parsechangelog.
Remaining placeholder Descriptions (file separately if pursued):
ai-gateway, backup, config-advisor, gitea, iot-guard, localrecall,
mcp-server, meshname, ndpid, nextcloud, soc, tor, vortex-firewall.
Tier 3 mesh-cluster consolidation. secubox-master-link's 851-LOC API
was effectively dead on production: the package shipped no nginx
config, so its 21 endpoints on /run/secubox/master-link.sock were
unreachable from the web. Meanwhile secubox-p2p has been serving the
visible /master-link/ UI all along — its nginx config aliases
/master-link/ to /var/www/secubox/master-link/ (frontend installed
by p2p), and its API has its own /master-link/* endpoints (status,
token, invite, invite-local, join, approve, peers, tree, config,
token/validate, token/revoke, token/cleanup, cleanup-local,
join-with-token, join-script — 16 in total).
Same situation that mail-lxc/webmail (#380) and mmpm (#381) were
in: a parallel implementation never wired up against the canonical
one. Apply the same transitional-shell pattern:
- secubox-master-link 1.1.0: Section: oldlibs, payload removed,
Depends: secubox-p2p (>= 1.7), postinst stops + disables +
removes the orphaned secubox-master-link.service plus the
/usr/share/secubox/www/master-link/, the menu entry, and any
leftover nginx snippet. The pre-1.1 state directory
/var/lib/secubox/master-link/ is intentionally preserved so
direct-socket clients (very rare) can retrieve their data
manually.
- secubox-p2p 1.7.0: Breaks/Replaces secubox-master-link (<< 1.1),
Description updated to call out master-link ownership. No code
changes — p2p's existing master-link surface stays as-is.
Operator-scoping decision encoded here:
- secubox-master-link assumed unused (literally unreachable via
nginx — no operator could be using it).
- secubox-p2p assumed canonical (it's what /master-link/ has
been serving in production).
- secubox-master-link's 21 endpoints NOT ported into p2p.
Reasoning: they were never reachable, so they were never
relied on. The 5 endpoints p2p doesn't currently expose
(heartbeat, stats, update-online-status, peers/bulk-delete,
peers/{id}/wireguard, wireguard/peers, wireguard/config) are
dropped along with the rest. If any of these turn out to be
needed, file a follow-up enhancement issue against secubox-p2p
rather than reviving the dead package.
Diff: 12 files changed, +90 / -1374. Net -1 effective package.
Verified: audit re-run shows secubox-master-link as is_empty=True
oldlibs depending on secubox-p2p; secubox-p2p unchanged on routes
(/api/v1/p2p/, /master-link/, /p2p/). Both changelogs parse cleanly.
Not verified: end-to-end dpkg-buildpackage; recommend rebuilding
both on a build host and a canary apt full-upgrade rehearsal from
1.6 → 1.7 before merging.
Maintainer of secubox-master-link corrected from
"SecuBox <dev@secubox.local>" to "Gerald KERMA <devel@cybermind.fr>".
Per-package inspection of the mesh cluster found:
- secubox-mesh (Yggdrasil daemon control), secubox-meshname
(Meshname DNS resolver), secubox-daemon (Go mesh binary) — all
distinct, keep.
- secubox-mirror — miscategorized: it's an APT/CDN cache, not
mesh-related at all. Audit grouped it by prefix matching.
- secubox-master-link — effectively dead on a running system:
ships an 851-LOC API and frontend but no nginx config; its 21
API endpoints (/run/secubox/master-link.sock) are unreachable
from the web. The "master-link" UI operators actually see is
served by secubox-p2p (which has its own /master-link/status +
/master-link/token endpoints + ships the frontend to
/var/www/secubox/master-link/).
- secubox-p2p is the canonical master-link owner.
Real consolidation candidate: master-link → p2p fold. Bigger than
the mmpm/magicmirror fold (issue #381) because of asymmetric API
surface — master-link's 21 endpoints vs p2p's 2 existing
/master-link/* endpoints means the merge isn't mechanical. Defer
to a separate per-cluster issue with its own scope decisions.
Mesh cluster net reduction potential: -1 (after master-link → p2p
fold), not the -4 the original audit projected. Total realistic
floor unchanged from previous revision (~130-136 packages).
Six packages in the dns + threats clusters shipped "X Module"
placeholder Description headlines that didn't help operators choose
between siblings. Phase 1 consolidation audit + #382 (dpi clarity)
established the pattern; this completes the dns and threats halves.
Each Description now:
- States the actual subsystem in the headline (no "X Module"
placeholders)
- Names the backend it talks to (BIND, dnsmasq, unbound, CrowdSec)
- Cross-references the sibling packages so operators reading
apt-cache show know what to install for what
DNS cluster:
- secubox-dns: "Dns Module" → "authoritative BIND zone manager"
- secubox-dns-guard: "Dns-guard Module" → "anomaly detection +
dnsmasq blocking"
- secubox-vortex-dns: "Vortex-dns Module" → "recursive firewall
with RPZ + threat feeds"
- (dns-provider, ad-guard already had informative headlines —
not touched)
Threats cluster:
- secubox-threat-analyst: "Threat-analyst Module" → "AI agent
that writes CrowdSec scenarios"
- secubox-cve-triage: "Cve-triage Module" → "vulnerability
assessment workflow"
- secubox-network-anomaly: "Network-anomaly Module" → "DNS-log
anomaly detection"
- (threats, cyberfeed, ipblock, openclaw already had informative
headlines — not touched)
Also corrected the Maintainer from the placeholder
"SecuBox <dev@secubox.local>" to "Gerald KERMA <devel@cybermind.fr>"
on four of six packages, and added explicit Recommends: for the
backend daemons each uses (dnsmasq for dns-guard + network-anomaly,
unbound + dnsmasq for vortex-dns, crowdsec for threat-analyst).
Patch-version bump on each (1.0.0 → 1.0.1; 1.4.0 → 1.4.1 for
threat-analyst) so the metadata change ships in apt.
No code, endpoints, services, nginx routes, menu entries, or
inter-package dependencies changed. dpkg-parsechangelog parses
cleanly on all six.
Per-package code inspection of the streamlit (already revised), dpi
(revised in 382), dns, and threats clusters all returned the same
finding: the audit's initial cluster groupings were based on naming
similarity, not architectural redundancy.
DNS cluster (5 → 5, was 5 → 1):
Five distinct DNS-layer subsystems, no shared config files. The
audit's "all overlap on /etc/resolv.conf and/or unbound" claim was
wrong — grep for /etc/resolv.conf in any of them returns zero hits.
Each touches its own config + data directory.
Threats cluster (7 → 7, was 7 → 1):
Seven distinct security capabilities with distinct backends
(crowdsec scenarios, nftables, dnsmasq logs, remote feeds, own
data dirs). Only the umbrella secubox-threats actually subscribes
to the CrowdSec/Suricata event bus the audit claimed all seven
did.
Tier 4 table revised to "candidates for inspection" — each cluster
needs the same per-package code check before any merge work.
Realistic-reduction floor updated from ~100 packages to ~130-136
(a 4-8% reduction from 141 rather than the originally projected 28%).
Where consolidation actually paid off: Tier 0 packaging bug (#378),
Tier 1 mail dead-code (#380), Tier 2 mmpm-magicmirror scaffolding
duplicate (#381), Tier 2 dpi description clarity (#382).
The pattern observation section explicitly flags this for future
work: most "fuzzy clusters" in out/clusters-fuzzy.txt are naming
overlaps, not merge targets. Treat the audit's cluster lists as
candidates for Description-clarity passes rather than packaging
merges.
After investigation in #382, the dpi cluster's four packages have
distinct operator roles, not duplicated ones:
- secubox-netifyd: daemon lifecycle dashboard
- secubox-dpi: analytics layer on top of netifyd
- secubox-ndpid: dedicated dashboard for the different nDPId engine
- secubox-mediaflow: streaming/VoIP consumer of DPI
The audit's original "dual-stream netifyd/nDPId" framing was
aspirational; the code is netifyd-only and nDPId analysis already
belongs to secubox-ndpid.
Real bug was a misleading Description on secubox-dpi (fixed in #382),
not a redundant-package situation. Keep all four packages.
Updates the realistic-reduction table: Tier 2 net is -1
(magicmirror only) rather than -3.
secubox-dpi and secubox-netifyd both ship a dashboard backed by the
netifyd daemon — that overlap had readers thinking one was redundant.
On closer inspection, they're a clean two-layer split:
- secubox-netifyd manages the daemon lifecycle (start/stop/config/
alerts/restart/interfaces) and is what an operator installs to
control netifyd itself.
- secubox-dpi is the analytics layer on top: top apps, top
protocols, bandwidth-by-app, bandwidth-by-device, talkers, risks,
plus the tc mirred ifb0 setup endpoint.
Update secubox-dpi's metadata to make that split explicit:
- Description headline: "netifyd Dashboard" → "Analytics —
netifyd-backed app/protocol classification" so the apt-cache
description no longer collides with secubox-netifyd.
- Description body: explicitly point at secubox-netifyd (and
secubox-ndpid for the different-engine case) so operators know
what to install for what.
- Recommends: secubox-netifyd, in addition to the existing netifyd
daemon recommendation. Layered model: control + analytics
together is the intended setup.
- Drop the "dual-stream netifyd/nDPId" framing from the
Description. The code is netifyd-only; nDPId analysis is owned
by secubox-ndpid (separate package, separate backend daemon).
The aspirational framing was making the cluster look more
consolidated than it actually is.
- README rewritten to match.
Bump to 1.0.5-1~bookworm1 so the description change ships in apt
metadata.
No code, endpoints, nginx routes, menu entries, or services changed.
secubox-netifyd, secubox-ndpid, secubox-mediaflow not touched.
Decided no cluster merge — the dpi/netifyd/ndpid/mediaflow split is
the right architecture; the only bug was the misleading description.
See [docs/superpowers/plans/2026-05-27-secubox-consolidation-audit.md].
Tier 2 consolidation (Phase 1 audit, magicmirror cluster). The two
packages live on the same host 100% of the time — secubox-mmpm
Recommends secubox-magicmirror, no metapackage references either,
both scaffolded boilerplate with only mechanical sweep edits since
creation. Reduces 2 binary packages → 1 effective + 1 transitional
shell.
Architecture:
- secubox-magicmirror 1.1: api/routers/mmpm.py is the transformed
mmpm code as an APIRouter (drops duplicate /health and /status).
api/main.py mounts it under /mmpm. Frontend moved to
www/magicmirror/mmpm/ (URL: /magicmirror/mmpm/). Menu entry
repointed from /mmpm/ to /magicmirror/mmpm/. nginx config
unchanged — existing /api/v1/magicmirror/ and /magicmirror/
blocks cover the new sub-routes via FastAPI + alias.
- secubox-magicmirror declares Breaks/Replaces secubox-mmpm
(<< 1.1) for clean upgrade.
- secubox-mmpm 1.1.0: Section: oldlibs, Depends on
secubox-magicmirror (>= 1.1), empty rules, postinst stops +
disables + removes the legacy secubox-mmpm.service and
/etc/nginx/secubox.d/mmpm.conf (mirrors the mail-lxc/webmail
transitional pattern from #380).
Operator-facing changes:
- The MMPM frontend URL moves from /mmpm/ to /magicmirror/mmpm/
- The MMPM API base moves from /api/v1/mmpm/ to
/api/v1/magicmirror/mmpm/ (frontend's `const API` updated)
- apt full-upgrade installs secubox-magicmirror 1.1, removes the
old standalone secubox-mmpm payload, leaves the metadata shell
for `apt autoremove` to clean up
Verified: both .py files parse cleanly (ast.parse). No external
references to secubox-mmpm except a roadmap status entry in
secubox-roadmap that documents migration state (deferred — separate
update, doesn't affect packaging).
Not verified: end-to-end dpkg-buildpackage. Recommend rebuilding
both packages on a host with python3-fastapi available before
merging, and a quick apt full-upgrade rehearsal from <1.1 to 1.1
on a canary box (the transitional postinst is the load-bearing
piece for upgrade safety).
Maintainer also corrected from "SecuBox Team <team@secubox.local>"
to "Gerald KERMA <devel@cybermind.fr>" on both packages.
On closer inspection, secubox-streamlit and secubox-streamforge serve
distinct operator workflows that map to two different LuCI apps in the
OpenWrt source. Streamlit is the LXC-backed runtime that runs apps
(pulls in lxc + debootstrap + sudoers); streamforge is the
template/authoring tool (core deps only, 660-line FastAPI).
Merging would force every streamforge operator to install lxc and
debootstrap or require a complicated Recommends-vs-Depends split for
marginal benefit (net -1 package).
Also revises the realistic-reduction table: Tier 2 net is -3, not -4.
The mail consolidation that #136 (Phase 1) landed and #153 (Phase 2,
in-flight) extends has correctly declared these three packages as
transitional at the apt/dpkg level: secubox-mail >= 2.2 carries
Breaks/Replaces against all three, each transitional declares
Section: oldlibs, override_dh_auto_install + override_dh_installsystemd
are both no-ops, and the postinst scripts do legitimate one-shot
cleanup of pre-2.2 leftover units (and /srv → /data migration for
mail-lxc per #319).
What remained was dead source-tree content: api/, nginx/, www/, menu.d/
directories and unused .service unit files that the rules deliberately
don't install but mechanical sweeps (license headers, /data path
migration in #320) kept dragging into changes against dead code.
Pruned:
- packages/secubox-mail-lxc/{api,nginx,www,debian/secubox-mail-lxc.service}
- packages/secubox-webmail/{api,nginx,www,menu.d,README.md,debian/secubox-webmail.service}
- packages/secubox-webmail-lxc/{api,nginx,www,debian/secubox-webmail-lxc.service}
Each package now contains only the debian/ directory with control,
changelog, rules, postinst, prerm — exactly what a transitional
package should ship.
The webmail README was actively misleading (still advertised
`apt install secubox-webmail` + `/api/v1/webmail/*` endpoints that
have moved). The apt Description field already carries the correct
transitional notice; the README added nothing but operator confusion.
Out of scope (left untouched):
- secubox-mail itself — actively modified by #153 (rspamd Phase 2)
- secubox-smtp-relay — distinct purpose (outbound relay)
- Removing the transitional packages themselves — too soon;
operators on 2.1/2.2 still need them as the apt-detectable handle
for autoremove. Keep at least one more release cycle.
Verified via scripts/audit-packages.py: all three packages now show
zero services / nginx_routes / api_modules / ships_files, i.e. empty
payload — the expected state for a transitional package.
Surfaced by the Phase 1 consolidation audit (Tier 1, commit a85129f5).
packages/secubox-daemon/ declared a secondary binary package named
secubox-c3box, but packages/secubox-c3box/ is also a distinct source
package (the Python FastAPI dashboard) that produces a .deb with the
same name. dpkg only allows one binary named secubox-c3box on a host
at a time, so whichever package the build matrix produced last would
silently displace the other on install.
The two packages do completely different things: the Go variant ships
/usr/bin/c3box (Situational Awareness for the WireGuard mesh) and
depends on secubox-daemon; the Python variant ships
/usr/lib/secubox/c3box/ (FastAPI Services Portal) and depends on
secubox-core. File paths don't overlap, but the binary-package-name
collision does.
Rename the Go variant to secubox-daemon-c3box, matching Debian's
source-binary disambiguation convention (cf. linux → linux-image-*).
The Python secubox-c3box keeps the canonical name.
Add Conflicts: secubox-c3box (<< 1.1~) so an old install is removed
cleanly when the new package lands. Operators that currently have
both produce-paths in their build matrix should install only one
package per host.
Files touched:
- packages/secubox-daemon/debian/control (Package: stanza
rename + Conflicts: + description update)
- packages/secubox-daemon/debian/rules (debian/secubox-c3box/
install paths → debian/secubox-daemon-c3box/)
- debian/secubox-c3box.{postinst,prerm} (git mv to
secubox-daemon-c3box.{postinst,prerm})
Surfaced by the Phase 1 consolidation audit (commit a85129f5).
Adds scripts/audit-packages.py to scan packages/secubox-*/ and emit
structured artifacts under out/ (gitignored): packages.json,
packages.dot dependency graph, empty-packages.txt, prefix-based
clusters, and a hand-curated fuzzy-cluster report with per-member
evidence (deps, routes, services).
Findings doc 2026-05-27-secubox-consolidation-audit.md synthesises the
data into Tier 0-4 merge proposals grounded in real package contents,
not the plan stub's illustrative guesses. Key corrections vs the stub:
- 141 real packages (not "100+"); secubox-core is the substrate for 131
- streamlit-idle is already an in-package service variant, not a
separate package — that consolidation is half-done
- meta{blogizer,bolizer,catalog,oblizer} are real distinct services,
not typo proliferation (recommend rename for clarity, not merge)
- Two genuine packaging bugs surfaced: secubox-c3box binary name
collision between Python dashboard and Go daemon (issue #378),
secubox-smart-strip has no debian/ directory at all
Also imports two pre-existing 2026-05-26 plan stubs (consolidation,
build-scripts-refactor) that had been sitting untracked.
Re-run after any packaging change: python3 scripts/audit-packages.py
Boards that don't use Wazuh frequently `systemctl mask
secubox-wazuh.service`. The 2026-05-26 mass-redeploy of 100 secubox
packages on gk2 triggered the wazuh postinst, which called
`systemctl enable` unconditionally. That command exits 1 against a
masked unit, `set -e` in the postinst aborts the script, and dpkg
leaves the package in `half-configured` — `apt-get install -f` then
fails until manual recovery.
Wrap enable+start in a `systemctl is-enabled` check that skips the
calls when the unit is masked. This preserves the operator's masking
choice across upgrades and avoids the half-configured trap.
The same pattern likely belongs in every other secubox-* postinst
that does an unconditional `systemctl enable`, but that's a broader
audit deferred to a separate change.
Discovered: gk2 mass-redeploy with --force-confold left wazuh
half-configured. Recovery on board:
systemctl unmask secubox-wazuh
dpkg --configure secubox-wazuh
systemctl mask secubox-wazuh
After this fix the same redeploy succeeds without intervention.
The source tree had `packages/secubox-soc/www/soc/index.html` at the
44 KB threat-map / Privacynex-style design (introduced 24296084,
April 15) that talks to `/api/v1/soc/*` — a gateway-style frontend
that requires a `secubox-soc-gateway` backend that isn't running on
production boards. The boards have been running a 107 KB
"comprehensive dashboard" that aggregates `/api/v1/hub`,
`/api/v1/waf`, `/api/v1/crowdsec` and `/api/v1/hub/public/firewall_summary`
directly — maintained out-of-tree, never captured back into source.
This commit pulls the production 107 KB file (recovered from
`/srv/www/soc/index.html` on the gk2 board, May 10 07:01 snapshot)
back into the source tree as the canonical SOC frontend.
While re-syncing, drop the dead `.logo`, `.logo-icon`, `.logo-text`,
`.logo-version` CSS overrides at the top of the inline <style> block.
The `.logo-icon` rule redefined what was meant to be a glow-filtered
🔒 emoji into a 40x40 gradient-box, breaking visual consistency with
every other module. `/shared/sidebar.css` owns those classes —
sidebar.js injects markup that the shared stylesheet styles
correctly, and these per-module overrides served no purpose.
Replaced the inline rules with a comment block explaining why no
local `.logo*` redefinition belongs here, to prevent the next
copy-paste round from reintroducing them.
Rationale captured in memory `feedback_source_first_always`:
this re-sync was triggered by mis-shipping `secubox-soc 1.0.1` from
source-tree state that hadn't been verified against the board.
Always source-first; the board frontend was authoritative.
On a production board the WAF threat log reaches 200 k+ JSONL entries
(~50 MB). The old read-through StatsCache (30 s TTL, lazy compute)
ran a full log scan + GeoIP lookups on every cache miss; on gk2 each
scan took ~6 s of CPU, which is longer than the TTL itself, so the
cache never actually warmed. Every dashboard poll blocked uvicorn's
event loop, HAProxy 504'd at 8 s, the WAF widget rendered empty, and
the WAF process burned 50-67 % CPU continuously even with no traffic.
This commit rewrites the data path: a single asyncio background task
(`_warm_loop`, refresh every 30 s) computes everything expensive ONCE
per tick — stats dict, last-500 alerts tail, CrowdSec bans — and
stuffs the results into a module-level `_warm` dict guarded by a
threading lock. The three hot endpoints become O(1) dict reads with a
cold-start fallback (one synchronous compute via asyncio.to_thread on
the very first request, before the warm task completes).
Other improvements bundled in:
* `_read_log_tail()` uses a seek-from-end of 512 KB instead of
`f.readlines()`. The old `/alerts` handler loaded every byte of
the log into Python memory to take the last 500 lines, allocating
~50 MB per call. Memory is now bounded regardless of log size.
* `/bans` no longer calls `cscli decisions list -o json` on the
request path — it was a 5-15 s blocking call shelled out under
sudo. The warm task does it once per refresh window.
* `_decorate_dashboard_stats()` extracts the dashboard-friendly
post-processing from the old /stats handler so the same code path
runs from the background task and the cold-start fallback.
* A `_StatsCacheShim` keeps legacy `stats_cache.{get,set,invalidate}`
call sites compiling — write-side endpoints used the cache to
invalidate after state changes. The shim absorbs them silently;
reads always miss so callers fall through to the warm cache. To
be removed once all references are audited.
Operator directive followed: "les cache double buffer ne doivent pas
tomber et permettre juste d'aller plus vite... un echec de cron et de
cache doit forcer le cron et faire du realtime". The warm cache IS
the fast path; cold start synchronously computes; the warm task is
fire-and-forget (one bad log line cannot kill the loop).
Verified on gk2 (board 192.168.1.200, real traffic, 201 940 threats
in the log, 28 active CrowdSec decisions):
/stats ~200-900 ms (was 6 s or timeout)
/alerts ?limit=5 ~360 ms (was timeout)
/bans ~210 ms (was 5-15 s)
WIP.md updated with v2.12.17, v2.12.18, and this work in context.
Deployment to gk2 surfaced two issues with v2.12.17:
1. Stray cron files (/etc/cron.d/secubox-nft-cache, /etc/cron.d/
secubox-nft-stats) left over from an earlier ad-hoc fix on the
board were racing with the new systemd timer — both wrote to the
SAME cache files, occasionally producing a 0-byte JSON that
parsed to 0 tables/0 chains/0 rules. These crons are NOT shipped
by any package (only mentioned in WIP/HISTORY); they have to be
scrubbed from running boards manually. Documented for the
upgrade-from-v2.12.16 path.
2. The hub runs with NoNewPrivileges=true. That blocks setuid
traversal, which means `sudo` from inside the hub process
ALWAYS fails silently — the realtime fallback in v2.12.17
therefore returned None and the endpoint surfaced zeros with
source=realtime instead of useful data.
Rewrite the endpoint's fallback ladder:
fresh cache (mtime < 60 s) → source=cache (fast path)
stale/missing → try realtime via sudo (best-effort)
realtime failed → fall back to STALE cache contents
source=cache-stale
no cache at all → source=none, zeros (only when the
timer has never run yet)
Showing 5-minute-old firewall stats on the dashboard is strictly
better than showing zeros while the cache populator is between runs.
The systemd timer recovers the cache within at most 30 s.
Sudoers fragment and -j arg ordering are kept verbatim from v2.12.17
so the realtime path lights up the moment an operator drops
NoNewPrivileges from the hub unit.
Verified on gk2:
fresh cache → source=cache, tables=7 chains=10 rules=19
ruleset aged to 5m → source=cache-stale, tables=7 chains=10 rules=19
both files removed → source=none, zeros
+30 s timer tick → source=cache restored
The v2.12.16 fix put the nft cache populator in firstboot.sh, which
broke the rule "cron lives next to the module that consumes it" — the
SOC firewall_summary endpoint is owned by secubox-hub, so the timer
that feeds its cache belongs in the same package.
This commit:
* removes the inline cache populator from image/firstboot.sh
* ships /usr/sbin/secubox-nft-cache + secubox-nft-cache.{service,timer}
inside the secubox-hub package, with the timer auto-enabled via the
timers.target.wants/ symlink
* adds a sudoers fragment authorising user secubox to run
`nft list *`, `nft -j list *`, and `systemctl --no-block start
secubox-nft-cache.service` (and only that one unit)
* rewrites the /firewall_summary endpoint so the cache is a speed
optimisation, not the source of truth: if the cache file is missing
or older than 60 s, fall back to a realtime `sudo nft list`,
surface `source: "realtime"` in the JSON, and nudge the cache
populator via systemctl so the next request lands on a hot cache
Per the operator directive: "les cache double buffer ne doivent pas
tomber et permettre juste d'aller plus vite... un echec de cron et de
cache doit forcer le cron et faire du realtime".
Operator screenshot: SOC dashboard "🔥 FIREWALL nftables / Status
INACTIVE / Tables 7 / Chains 10 / Rules 19" with all counters at 0
(Accept/CAPI/CrowdSec/Manual).
Two bugs found:
1. The /api/v1/hub/public/firewall_summary endpoint docstring says
it reads cache files `/var/cache/secubox/nft-*.txt` "updated by
cron every 30s" — but the cron never existed in the repo. The
table/chain/rule counts were getting populated somehow (probably
a one-shot at firstboot) while the counters file stayed empty,
hence 7/10/19 visible but all hit-counts zero.
2. The endpoint never returned a `status` field, so the React
bundle (secubox-soc-web) defaulted to "INACTIVE" regardless of
whether nftables.service was active or rules were loaded.
Fixes:
* firstboot.sh installs `/usr/local/bin/secubox-nft-cache` +
`secubox-nft-cache.service` + `.timer` (oneshot, every 30s,
OnBootSec=10s). Runs as root via systemd so it can call
`nft -j list ruleset` + `nft list counters`. Writes the JSON
ruleset + counters dump + a `.lastrun` sentinel into
/var/cache/secubox/.
* hub `/firewall_summary` endpoint now:
- calls `systemctl is-active nftables.service` and returns a
`status` field ("active" | "inactive" | "active-manual" | "error")
- "active-manual" = systemd reports inactive but the kernel has
rules loaded (happens when firstboot.sh did `nft -f` directly
and the systemd unit didn't get re-enabled). Treat as active
for dashboard purposes — operator wants to see "firewall is up"
when there ARE rules.
- returns `cache_last_run` so the dashboard can warn on stale
cache.
WAF metrics empty is by design on a live USB without mitmproxy LXC
(THREATS_LOG never gets written → zero counters). Not addressed
here. Operator runs install-lxc.sh once to enable WAF traffic.
rpi400 v2.12.13 image audit (operator-reported "ancien style et il
manque des tas de seccubox tools"):
136 secubox-* packages installed ✓
but only 11 secubox-* binaries on PATH
/etc/motd was vanilla Debian
/etc/issue was vanilla Debian "Debian GNU/Linux 12 \n \l"
no /etc/secubox/build-info.json
build-live-usb.sh creates ~30 secubox-* CLI helpers + the cyber-CRT
boot banner + dynamic MOTD via update-motd.d. None of that was in
build-rpi-usb.sh. Targeted port of the four most-visible blocks
into build-rpi-usb.sh at line ~780 (right after `SecuBox packages
installed`):
* /etc/issue with cybermind banner + getty `\4` IPv4 escape
* /etc/update-motd.d/10-secubox with live `hostname -I` substitution
* /usr/bin/secubox-status (system overview, services, network)
* /usr/bin/secubox-help (command index)
* /etc/secubox/build-info.json (version + git commit + builder)
Broader refactor of the four build-*.sh siblings into shared lib/*.sh
is tracked in docs/superpowers/plans/2026-05-26-build-scripts-refactor.md
— this commit is the tactical port to unblock rpi400 operators now.
The MOTD says "RPI400 LIVE" instead of "LIVE USB MODE" so operators
can tell at a glance which platform they're on without `uname -m`.
secubox-net-detect.service runs on first boot (gated by
!/var/lib/secubox/.net-configured) and REWRITES /etc/netplan/00-secubox.yaml
based on the detected board layout. The "no LAN interfaces" branch
in generate_netplan() was emitting an empty br-lan bridge with a
static 192.168.1.1/24 address. On single-NIC live USB hardware that
made systemd-networkd happily honour the phantom bridge while the
real WAN NIC fell off the DHCP path — the operator saw a working
br-lan in `ip a` but no upstream IP and no internet.
Same shape of bug as the v2.12.7 build-live-usb.sh static-in-DHCP
regression (commit a88c1773): a useless static IP on a non-routing
interface confused networkd into "I'm done, no DHCP needed". Fix:
when there are no LAN interfaces, emit NO bridges block at all.
Operators wire br-lan later via secubox-net-* tools once they've
decided the box is a router vs an endpoint.
Hub + portal stayed `inactive (dead)` on real-hardware boot. The
journal error was `Failed to set up mount namespacing: /run/systemd/
unit-root/run/secubox: No such file or directory` at the NAMESPACE
step. Root cause: 96 services declare `RuntimeDirectory=secubox`
without the matching `RuntimeDirectoryPreserve=yes`. When any of them
stops (including any of the LXC-backed services that fail their
health probe and Restart=on-failure for a few cycles), systemd
removes /run/secubox on the way out. The next service with the same
RuntimeDirectory= that tries to namespace its inputs hits the gap
and falls into 226/NAMESPACE failure — Restart hammers a few times,
then the unit goes failed-permanent.
Earlier fmrelay + sentinelle units got the fix individually
(v2.12.0/v2.12.3 era). Now applied to all 96 remaining units in a
single sweep via:
sed -i '/^RuntimeDirectory=secubox/a RuntimeDirectoryPreserve=yes' "$f"
No version bumps in changelog — `dpkg -i --force-depends` in the
live-USB slipstream picks up the new .deb regardless of version
number.
v2.12.11's rule `udp sport 67 udp dport 68 accept` was invalid nft
syntax — protocol prefix can't be repeated inside a single rule, the
firstboot.sh-generated /etc/nftables.conf failed to load and the
firewall stayed in its previous state (or empty), defeating the
whole point of the DHCP fix.
Just match the destination port: `udp dport 68 accept`. DHCPOFFER /
DHCPACK from any server (sport 67) lands on dport 68 of the client,
so the inbound match is sufficient and unambiguous.
Live-system workaround (no rebuild needed):
nft add rule inet secubox_filter input udp dport 68 accept
Bare-metal box ended up with 192.168.10.250 (assigned by secubox-net-
fallback's ARP-probe loop) instead of the real DHCP lease from the
LAN router. Operator confirmed DHCP server is healthy on the LAN
(dev box on same switch got 192.168.1.13 via DHCP without issue).
Root cause: firstboot.sh writes /etc/nftables.conf with policy=drop
on the input chain and no explicit DHCP allowance. The `ct state
established,related accept` rule does NOT cover DHCP because the
client request leaves from 0.0.0.0:68 and the server reply comes back
broadcast (or unicast direct to the offered IP before it's bound).
Neither matches the original 5-tuple → conntrack treats the reply
as a new packet → policy drop → DHCPOFFER lost → networkd times out.
secubox-net-fallback then picks a random gateway from its probe
list and lands the box on the wrong subnet.
Add `udp sport 67 udp dport 68 accept` to the input chain. This
matches DHCPv4 server-to-client replies specifically (port 67/68
are reserved BOOTP/DHCP). DHCPv6 would need a sibling `udp dport
546 accept` rule but is out of scope here.
Real UEFI hardware kept landing in grub rescue / grub shell across every
v2.12.x iteration: shim swap, hardened module list, $cmdpath embed,
search --fs-uuid bake-in. None of them shipped a working real-UEFI
boot. v2.10.3 was the last tag the operator confirmed booted his
hardware in both BIOS and UEFI mode.
Drop the experimental UUID capture + the embed-cfg fallback chain.
Restore EXACTLY what v2.10.3 had:
# embed-cfg
search --no-floppy --label ESP --set=root
set prefix=($root)/boot/grub
configfile $prefix/grub.cfg
# main grub.cfg LIVE search
search --no-floppy --label LIVE --set=live
Validated other v2.12.x improvements stay (mass-mask LXC services,
dynamic MOTD, modesetting for VBox, kiosk --no-block) — they don't
touch the boot path.
Real UEFI hardware v2.12.7/v2.12.8 still landed in the grub> rescue
shell, and the user reported the same happening in legacy BIOS mode
too — both code paths share the /boot/grub/grub.cfg which begins with
`search --no-floppy --label LIVE --set=live`. If that label lookup
fails (some firmware doesn't surface FAT/ext4 labels, or the firmware
exposes the disk through a path GRUB's label table doesn't recognise)
\$live stays unset, every menuentry's `linux (\$live)/live/vmlinuz`
becomes `linux ()/live/vmlinuz`, the kernel doesn't load, GRUB
drops the operator at the rescue prompt. Same root cause as the
EFI embed-cfg's `search --label ESP` failure.
Capture the ESP + LIVE UUIDs with blkid right after mkfs.* and
substitute them into both configs:
* grub.cfg uses `search --fs-uuid <LIVE_UUID> --set=live` as
primary; falls back to `search --label LIVE` for firmware that
DOES prefer labels (no-op if UUID already resolved).
* embed-cfg uses `\$cmdpath/grub.cfg` first, then falls back to
`search --fs-uuid <ESP_UUID>` + configfile from that root.
UUIDs are deterministic at build time, can't be shadowed by similarly
labelled partitions, and don't depend on the firmware's label
indexing — should work on any firmware that surfaces the disk to
GRUB's block IO at all.
Real UEFI hardware v2.12.7 reported landing at the grub> rescue
prompt even with the simple pre-e1a53297 embed-cfg restored. Cause:
`search --no-floppy --label ESP --set=root` failed silently on that
firmware (FAT label not recognised, predictable rename, etc.), so
\$root stayed unset, prefix was empty, configfile fell through to
the rescue shell.
Replace the embed with a single-line `configfile \$cmdpath/grub.cfg`.
\$cmdpath is set by the EFI firmware itself to the directory it
loaded BOOTX64.EFI from — for us that's (hdX,gptY)/EFI/BOOT, where
the build copies grub.cfg already. The full menu at /EFI/BOOT/grub.cfg
still does its own `search --label LIVE --set=live` to locate the
squashfs partition (unchanged), so this only changes the very first
lookup hop.
Legacy BIOS grub-install --target=i386-pc untouched — it never had
the issue.
Real-hardware report: only the phantom `br-lan` (interfaces: [] +
static 192.168.1.1/24) showed an IP, the physical ethernet stayed
sec. The old netplan declared three separate interface groups
(en*, eth*, all-wifi with empty SSID) plus a member-less bridge with
a static address — that was enough to confuse networkd into
honouring the bridge while dropping the real DHCP request on the
floor.
Cut to the bone: one `eth-all` block matching `e*` (which covers
enpXsY, enoX, ensX, enxAABBCC and ethX), DHCP only, no static, no
bridge, no wifi-with-empty-SSID. Operators wire router-mode br-lan
or wifi later via the secubox-net-* tooling once they've decided
this is an endpoint vs a router.
For the no-DHCP fallback case, secubox-net-fallback.service is
already shipped and runs after networkd to ARP-probe common gateways
and assign a free .250 IP — that remains unchanged.
This means v2.12.6 has both a clean EFI build AND a netplan that
actually requests DHCP on the physical NIC instead of squatting on
the bridge.
Real UEFI hardware boot was broken by my e1a53297 "hardening" — the
gold-plated module list (disk/usb/usbms/ahci/ata/...) and multi-stage
embed-cfg search fallback that I added to chase a VBox EFI Shell drop
turned out to break legitimate UEFI firmware too. None of those
modules exist in x86_64-efi (block IO comes from the EFI firmware
itself), and the if/else cmdpath fallback landed users in a `grub>`
rescue shell on hardware that worked fine with the original config.
Revert the EFI section to the v2.10.x layout that v2.10/v2.11 booted
cleanly on real amd64 boxes: 14 modules, single-line search, plain
grub-mkimage with no compress/no Secure Boot shim. Also drop the
shim-signed + grub-efi-amd64-signed from the host apt install — they
were only there to feed the SB swap that's now removed.
VBox EFI drops to the GRUB rescue shell with this config; that's
acceptable per operator direction ("je m'en fous de booter en legacy
sur vbox, ca marche sur real hw") — VBox testing has always used the
BIOS firmware path which still works perfectly.
Live USB v2.12.4 boot showed `secubox-kiosk-setup enable --x11` running
for 5+ minutes while the kiosk.service stayed `inactive (dead)`. The
synchronous `systemctl start secubox-kiosk.service` at the end of
enable_kiosk was hanging.
Root cause: secubox-kiosk.service declares
Conflicts=getty@tty7.service
TTYPath=/dev/tty7
systemd waits for tty7 to be released and for the conflicting unit's
JobsToStart to clear before kiosk's ExecStart can fire. When the call
comes from a process running inside the tty1 autologin shell (which
is what firstboot.sh's `secubox-kiosk-setup enable --x11` safety net
does), systemd deadlocks itself waiting on a tty operation the caller
can't unblock. The kiosk service appears to start fine when triggered
manually from SSH (proven during debug — once the stuck setup PID was
killed, `systemctl start secubox-kiosk.service` returned in <1s and
the service activated cleanly).
Fix: pass --no-block so systemctl returns as soon as the job is
queued, regardless of the tty wait. systemd then runs the start in
the background after the parent autologin shell exits, breaking the
loop. The exit-code branch is preserved for legitimate failures.
Three regressions surfaced on the v2.12.3 live USB amd64 boot console:
1. /etc/issue + /etc/motd had `<IP>` as literal text — nothing was
substituting at runtime, so operators saw `https://<IP>:9443`
instead of the actual address. The bashrc splash showed real IP
via `hostname -I` but was masked by the static MOTD on top.
Fixes:
* /etc/issue: switch to getty's `\4` escape (resolves to first
IPv4 address at TTY render time).
* /etc/motd: blank out the static file + ship the banner as
/etc/update-motd.d/10-secubox, which pam_motd regenerates on
every interactive login. The script substitutes the live
`hostname -I` first address (falls back to `no-ip` while DHCP
hasn't completed).
2. Kiosk Xorg failed under VirtualBox VMSVGA. The auto-detect picked
the `vmware` Xorg driver, which loads vmwgfx kernel module — vmwgfx
tries to talk to a VMware Workstation host channel that doesn't
exist on VBox, prints "Failed to send host log message" then
bails. The kiosk launcher then logs Xorg failure, retries 3 times,
self-disables. Operator sees no kiosk even after running
`secubox-kiosk-setup enable` (because the launcher disabled the
sentinel again).
Fix: secubox-x11-setup unconditionally picks `modesetting` on
oracle (VirtualBox), regardless of VMSVGA vs VBoxVGA. modesetting
uses DRM/KMS through vmwgfx but skips the broken host channel.
Real VMware Workstation users (VM_TYPE=vmware) still get `vmware`
via the dedicated case branch — only the VBox path changed.
Note: didn't tag yet — wait for user retest after CI rebuilds. If the
boot console comes up with the right IP + the kiosk paints pixels, we
tag v2.12.4 next.
VBox + bare-metal v2.12.2 amd64 tests showed the live USB had no IP at
all (DHCP nor static fallback worked) and ~10 [FAILED] secubox-*
services on the boot console.
Two independent fixes layered together:
1. Inline netplan: my v2.12.1 commit (27645f0a) baked
`addresses: [192.168.1.55/24]` + gw 192.168.1.254 into both
eth-dhcp (match en*) and eth-legacy (match eth*). On a single-NIC
box both interface defs match the same kernel interface and try to
claim 192.168.1.55 simultaneously → systemd-networkd refuses the
apply and the link gets NO IP at all. Even DHCP was broken.
Drop the static block entirely; let secubox-net-fallback.service
(already shipped, runs after networkd) handle the no-DHCP case via
ARP-probe gateway discovery + free .250 IP assignment.
2. INCOMPLETE_MODULES list: extended with the host-side control planes
that proxy to an LXC container — without `install-lxc.sh` having
run, the container at 10.100.0.X doesn't exist, the health probe
fails immediately, and Restart=on-failure spins the service into a
FAILED loop that fills the console and blocks multi-user.target.
Added: grafana, yacy, rustdesk, lyrion, authelia (host), mail,
gitea, matrix, horde, mitmproxy, nextcloud, rbs-sensor.
Operator workflow stays unchanged on real installs: run
`bash install-lxc.sh` to provision the container, then
`systemctl unmask secubox-<name>` + enable + start.
Both modules need physical hardware (RTL-SDR, EP06 modem) and crash
on `import gr_osmosdr` when those aren't there. On the live USB
amd64 build neither is present, so the service died at startup,
systemd restarted in 5s under `Restart=on-failure`, and the cycle
repeated until /var/log filled with hundreds of FAILED lines —
visible on the boot console and blocking the kiosk target from
ever finishing.
Two fixes layered:
1. systemd unit (sentinelle-gsm.service): add StartLimitIntervalSec=30
+ StartLimitBurst=3 + StartLimitAction=none. If start fails 3
times in 30 s, give up instead of looping forever. The kiosk's
multi-user.target settles even when the SDR is missing.
2. build-live-usb.sh: mask secubox-sentinelle-gsm + secubox-fmrelay
in the INCOMPLETE_MODULES list alongside other hardware-gated
modules. Operators enable + unmask by hand once they wire the
physical RTL-SDR / GSM modem.
Operator quick fix on an already-flashed live USB:
sudo systemctl stop secubox-sentinelle-gsm
sudo systemctl mask secubox-sentinelle-gsm
sudo systemctl mask secubox-fmrelay
sudo systemctl daemon-reload
Then `secubox-kiosk-setup enable --x11` should complete normally.
v2.12.1 booted in VBox dropped to the EFI Shell PXE prompt: BOOTX64.EFI
was built but the module set lacked `disk` / `usb` / `usbms` so OVMF
couldn't hand off block IO once GRUB took over. Same image worked in
BIOS mode — the i386-pc grub didn't need those modules.
Hardening:
* GRUB_MODS now includes `disk usb usbms ahci ata` + `loadenv` +
`gfxterm_background gzio png jpeg font sleep reboot halt true exfat
iso9660 ntfs search_fs_uuid search_fs_file` — the full toolbox a
USB / VM EFI handoff needs.
* `grub-mkimage` failure aborts the build instead of warning. The
embed-cfg's `search` chain falls back through label-ESP →
label-LIVE → `$cmdpath/../../boot/grub` so the bootloader still
finds the menu when the partition label was lost (some flashers
rewrite it).
* `--compress=xz` on the EFI binary shaves ~40% off its size.
* Secure Boot: if `shimx64.efi.signed` + `grubx64.efi.signed` are
available on the builder (they're already installed by CI line 108),
ship the Microsoft-trusted shim AS BOOTX64.EFI and the signed grub
alongside. Falls through to plain unsigned grub when those packages
are missing locally, with a noisy warn explaining how to install
them.
* `grub-install --target=i386-pc` failures now propagate via err
instead of warn — silent BIOS install failure was the v2.10.x
"boots into nothing" footgun.
* Mirror BOOTX64.EFI to /EFI/secubox/grubx64.efi so a later
`efibootmgr -c` can register a named entry without clobbering
/EFI/BOOT.
* Local apt-get install adds `grub-efi-amd64-signed` + `shim-signed`
so dev-host builds get the same Secure Boot output as CI.
Operator request: dashboard reachable without SSO from the LAN, gated
by Authelia from the outside.
Two new pieces wired together:
1. /etc/nginx/conf.d/secubox-lan-geo.conf (new)
- geo \$lan_client = 1 for 127/8, 192.168.1.0/24, 192.168.255.0/24,
10.100.0.0/24 (br-lxc), 10.55.0.0/24 (eye-br0), 10.0.3.0/24
- set_real_ip_from 127.0.0.1 + 10.100.0.0/24 with real_ip_header
X-Forwarded-For + real_ip_recursive on. Without this every
request looks local (HAProxy + mitmproxy are on-host) and the
bypass would whitelist the public internet.
2. /etc/nginx/secubox.d/authelia.conf
- location = /__sbx_auth_verify gains `if (\$lan_client) { return 200; }`
as the first directive — short-circuits the Authelia subrequest
for trusted networks. SSO-gated backends keep their existing
`auth_request /__sbx_auth_verify;` + `error_page 401 = @sbx_auth_login;`
directives unchanged; the bypass is invisible to them.
debian/rules installs the geo block into /etc/nginx/conf.d/ so it
loads at http{} level before any include of secubox.d/*.conf inside a
server{} block.
The `location = /` redirect dropped the query string, so when
@sbx_auth_login sent users to `https://sso.gk2.secubox.in/?rd=...`,
the rd= parameter was stripped on the first hop and Authelia's login
form had no return target. Users landed on the hub root after auth
instead of on the protected page they originally requested.
Use `$is_args$args` on the redirect target — present on both vhost
blocks (auth.maegia.tv + sso.gk2.secubox.in). Verified chain:
/fmrelay/ → 302 sso/?rd=…
sso/?rd=… → 302 sso/auth/?rd=… ← now preserves rd
sso/auth/?rd=… → 200 Authelia login
Surfaced while bringing fmrelay live (#377), but affects every
SSO-gated module behind the public sso portal.
Top page bar showed 📦 for any module whose path wasn't in the hardcoded
PAGE_ICONS table. Refactor getPageIcon() to fall back to live menu data via
window._menuDataCache (pre-seeded from localStorage at boot, updated when
the menu API lands). Also re-paints the icon once fresh menu data arrives.
Source-side menu.d entries that still used Font Awesome class names are
swapped to emoji so the icon round-trips through the API instead of
rendering as text:
- secubox-authelia: "fa-key" -> "🔑"
- secubox-hexo: "edit" -> "📝"
- secubox-smtp-relay: "envelope" -> "✉️"
- secubox-metoblizer (Overview entry): "scroll" -> "📜"
