* feat(p2p): DHT, Federation, MasterLink evolutions for P2P-EVO-2026-07-001
Implements three major features for secubox-p2p:
1. DHT (Distributed Hash Table) based on Kademlia algorithm
- Peer discovery across different subnets
- Resilience to temporary node failures
- Scalability for large number of nodes
- REST API endpoints for DHT management
2. Service Federation
- Service registration and discovery
- Multi-version service management
- Automatic health checks
- DHT integration for service propagation
3. Master-Link Hierarchical Topology
- Master/satellite/leaf hierarchical structure
- Automatic master election and failover
- Heartbeat monitoring
- Routing policy management
- OPAD security integration
New files:
- api/dht.py: Kademlia DHT implementation
- api/federation.py: Service federation system
- api/masterlink.py: Hierarchical topology manager
- api/main_evolutions.py: Integrated FastAPI server
- tests/test_dht.py: Unit tests for DHT
Issue: P2P-EVO-2026-07-001
Worktree: secubox-p2p-evolutions
Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
* revert(p2p): remove non-integrating DHT/federation/masterlink code, keep requirements
The DHT/federation/masterlink modules + main_evolutions.py + test_dht.py did not
integrate with the module: tests imported 'secubox.p2p.api.*' (the package uses
'from api import ...' via conftest sys.path) so nothing even collected, and the
modules reinvented standalone aiohttp servers instead of building on the existing
WireGuard mesh (mesh.py) + annuaire-backed registry (registry.py/annuaire_client.py).
Removing them to restart on the real base. The requirements doc
(.github/ISSUES/2026-07-P2P-EVOLUTIONS.md) is kept as the spec source.
* docs(p2p): point the local evolutions note to real issue #774
The '.github/ISSUES/2026-07-P2P-EVOLUTIONS.md' was a local file with a fabricated
id, not a tracked issue. Replace its contents with a pointer to the real GitHub
issue #774 (Kademlia DHT + hierarchical master-link + federation health-checks),
framed against the already-done federation (#766) and directory (#768) work and
the master-link/auto-enrollment issue (#762).
* docs(spec): p2p Kademlia DHT + master-link + federation health-checks (#774)
Clean redesign on the real base (mesh.py + registry/annuaire_client) after
reverting the non-integrating attempt. Custom minimal Kademlia (pure asyncio
UDP, JSON wire, signed reachability records), federation health-checks layered
over the annuaire federation (#766) publishing status via the DHT, and a
deterministic-election master-link (term-based failover, aligns #762). All
feature-flagged off by default (OPAD opt-in), from api import convention, TDD
per module. Refs #774.
* docs(plan): p2p DHT/federation/master-link TDD implementation plan (#774)
17 bite-sized TDD tasks across 3 phases (DHT 1-9, federation health-checks
10-13, master-link 14-16, finalization 17). Each task = failing test -> minimal
impl -> pass -> commit, on the real base (from api import ...). Crypto (Ed25519
sign/verify) and UDP transport are module-level seams injected in unit tests so
the Kademlia primitives, health debounce, and election/term logic are tested
without real sockets. Refs #774.
* feat(p2p): DHT scaffold — node id + xor distance (#774)
* feat(p2p): DHT k-bucket with LRU (#774)
Add DHTNode contact + DHTBucket (k-bucket with LRU ordering).
- DHTNode: dataclass with node_id, did, endpoint, last_seen
- DHTBucket: Kademlia k-bucket with OrderedDict LRU management
- add(node) → bool (True if stored/refreshed, False if full)
- refresh moves node to tail (most-recent)
- oldest() returns head node
- remove(node_id) and nodes property for list access
All Task 1 tests + new Task 2 tests passing.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(p2p): DHT routing table + closest-N (#774)
* feat(p2p): DHT signed reachability records + verify (#774)
Implement canonical_record, sign_record, verify_record functions with
crypto SEAMS (_did_from_pubkey, _verify_sig, _sign_sig) for testing.
Tests use monkeypatching; real crypto wired in Task 8.
Canonical records are deterministic sorted JSON. verify_record checks
for sig field, DID validity via wg_pubkey, and cryptographic signature.
* feat(p2p): DHT JSON UDP codec + hardening (#774)
* feat(p2p): DHTNetwork store+dispatch (transport-injected) (#774)
* feat(p2p): DHT iterative find_node/find_value + announce (#774)
Add asyncio-based iterative Kademlia lookup on top of the injected
DHTNetwork transport from Task 6:
- pending: dict[rpc_id, Future] on DHTNetwork, resolved by handle_message
when a reply (pong/nodes/value/ok) arrives with a matching rpc_id.
- _rpc(node, msg): send + await the matching future with RPC_TIMEOUT,
returns None on timeout.
- iterative_find(target_id, mode): alpha-parallel iterative lookup,
dedups queried nodes, merges discovered contacts into the shortlist
(dedup by node_id, skip self), terminates on no-improvement-in-a-round
or exhausted candidates; mode="value" short-circuits on first verified
record.
- find_peer(did) / announce(): DID resolution and self-record
publication (sign, cache locally, push to the k closest nodes found).
Tests (tests/test_dht.py, +2 async, pytest-asyncio strict mode):
- in-process UDP router delivering via call_soon (never inline) so
awaited RPC futures resolve on a later loop turn;
- find_peer resolves a record announced by a node reachable only
through a bootstrap intermediary;
- dedup: a node introduced twice into the shortlist (via two different
repliers) is queried exactly once.
All 16 tests green (14 prior + 2 new).
* fix(p2p): harden DHT iterative lookup — skip malformed contacts, cap shortlist, tolerate rpc exceptions (#774)
* feat(p2p): DHT record schema (id+wg keys) + real Ed25519 sign/verify (#774)
* feat(p2p): DHT UDP transport + routing persistence + bootstrap (#774)
* feat(p2p): mount DHT — endpoints + startup guard + [dht] config (#774)
Task 9: wire the finished Kademlia DHT (api/dht.py) into the FastAPI
app as a feature-flagged background service, fully backward compatible.
- api/mesh.py: load_p2p_config() now also returns a `dht` sub-dict
(enabled/port/bootstrap/announce/announce_interval/rps) built from an
optional [dht] toml section, with defaults when absent.
- api/main.py: @app.on_event("startup") _dht_startup creates+binds a
DHTNetwork only when [dht].enabled is true; any failure (missing
identity, bind error, ...) is caught and leaves app.state.dht = None
so the app always starts. _dht_shutdown persists the routing table
and closes the UDP transport.
- New endpoints (bare paths, nginx adds /api/v1/p2p prefix): GET
/dht/peers (routing snapshot), POST /dht/announce (JWT-protected,
audit-logged to /var/log/secubox/p2p-audit.log), GET /dht/find/{did}.
Tests: tests/test_dht_wiring.py (config defaults/overrides + disabled
TestClient smoke). All 22 existing DHT tests + 3 new tests green (74
total in the package).
* feat(p2p): federation HealthStore + debounce (#774)
* feat(p2p): federation HealthChecker sweep (#774)
* feat(p2p): federation real probe + registry services (#774)
* fix(p2p): bound TCP-probe close, declare aiohttp dep, guard non-dict service (#774)
* feat(p2p): federation health publish-via-DHT + endpoints + [federation] config (#774)
* fix(p2p): wire federation probe_timeout + harden sweep/health-TTL tests (#774)
* feat(p2p): master-link elect() + term store (#774)
* feat(p2p): master-link state machine + failover (#774)
* fix(p2p): master-link equal-term tie-break (no zero-master window) + defensive peers (#774)
* feat(p2p): master-link signed heartbeats + UDP transport + tick loop (#774)
* feat(p2p): mount master-link — endpoints + startup guard + [masterlink] config (#774)
- masterlink.MasterLink.request_promotion(): on-demand promotion attempt
(bump term, run election, become MASTER + heartbeat on win, else CANDIDATE)
- mesh.load_p2p_config(): add [masterlink] section defaults/overrides
(enabled, role_preference, priority, heartbeat_interval, election_timeout,
port, peer_addrs) alongside existing [dht]/[federation]
- main.py: _masterlink_startup/_masterlink_shutdown mirror the DHT/federation
defensive startup pattern (feature-flagged, never breaks app startup);
GET /masterlink/topology (public) + POST /masterlink/promote (JWT, audited
to p2p-audit.log)
* fix(p2p): master-link request_promotion records winner on loss + strengthen test (#774)
* docs(p2p): document DHT/federation/master-link evolutions (#774)
Add an "Evolutions (#774)" section to packages/secubox-p2p/README.md
covering the DHT (api/dht.py), federation health-checks (api/federation.py)
and hierarchical master-link (api/masterlink.py) subsystems: what each does,
their API endpoints, the real p2p.toml config defaults from api/mesh.py,
and the audit log. All three remain OFF by default / OPAD opt-in. Docs only,
no behavior change. Append a matching dated entry to .claude/HISTORY.md.
* fix(p2p): final-review fixes — audit-log path, master-link peers_fn, wg_pubkey, handle_message hardening (#774)
* fix(p2p): implement mesh visualization tab — load on activation + master-link roles (#774)
The Mesh tab canvas was never rendered (the tab-switch handler didn't call
initMesh, and drawing into a hidden tab gave clientWidth 0 -> blank canvas).
Now: initMesh fires when the Mesh tab is activated, retries if the canvas has
no layout yet, and enriches the star graph with the live master-link state
(role/term/master + DHT peer count from /masterlink/topology + /dht/peers) —
center node coloured gold=MASTER / cyan=SATELLITE, peers by online status.
* docs(p2p): poster GPT + roadmap des évolutions DHT/federation/master-link; maj WIP/HISTORY/TODO (ref #774)
---------
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
Co-authored-by: Mistral Vibe <vibe@mistral.ai>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
The live-usb kiosk stack (dbus, X11, chromium) aborted its postinst in the
init-less chroot ('Failed to connect to system message bus', invoke-rc.d
errors), failing the build. Add /usr/sbin/policy-rc.d (exit 101) before the
installs and remove it before squashfs, so packages don't try to start
services at build time but the booted system still does. Keep kiosk ON for
amd64 USB (extra_args=--kiosk). Do NOT disable kiosk.
dpi/toolbox-ng/waf-ng are CGO_ENABLED=0 cross-compiles; the go toolchain is
present (golang-1.22-go) but dpkg-checkbuilddeps trips on the golang-go
metapackage. Skip the dep check for just these three (-mod=vendor, self-contained).
The discover matrix filter only compared requested_arch against amd64/arm64/
empty, so the 'both' workflow_dispatch option produced an EMPTY matrix (no
builds, collect failed). Normalize 'both' -> empty (= all arches).
secubox-dpi and secubox-toolbox-ng are CGO_ENABLED=0, GOARCH=arm64,
-mod=vendor offline cross-compiles. They failed the arm64 build only
because golang-go was never installed, so dpkg-checkbuilddeps aborted on
their `Build-Depends: golang-go (>= 1.22)`. Not a CGO cross-toolchain
gap. ubuntu-24.04 ships golang-go >= 1.22; installing it lets the pure-Go
arm64 cross-compile run on the amd64 runner.
Unblocks a complete amd64+arm64 package set for the apt repo + arm64 images.
Decision (operator): nDPId is a permanent capture daemon + nDPIsrvd process →
continuous CPU/RAM on an already-saturated board (load ~4.6/4 cores), whereas the
ndpiReader producer runs in bounded niced 60s windows (~1% CPU) and frees the
core between passes. The richer-JSON / no-respawn gain doesn't justify the perf
risk, and the QEMU cross-build failed first try (fragile path). The exfil
pipeline was never cut over, so live is unaffected.
Reverts #722/#723: removes .github/workflows/build-ndpid.yml and restores
secubox-ndpid to its prior arch:all stub. DPI Phase 3 = ASN (#719) + history/
timeline (#721); nDPId dropped. ndpiReader remains the producer.
nDPId is a C daemon needing libnDPI >= 5.0 (bookworm ships 4.2) and the sandbox
cross-toolchain can't link C, so:
- .github/workflows/build-ndpid.yml — QEMU arm64 NATIVE build inside
debian:bookworm (cmake -DBUILD_NDPI=ON bundles the right libnDPI), commits
nDPId + nDPIsrvd to packages/secubox-ndpid/bin/ + uploads an artifact.
Manual (workflow_dispatch), ndpid_ref input to bump.
- secubox-ndpid: Architecture all → arm64, Depends libpcap0.8 (dropped the
external ndpid|ndpi-reader dep), debian/rules ships bin/nDPId + bin/nDPIsrvd to
/usr/sbin (guarded so it builds before CI populates bin/). 1.1.0.
Follow-on (after CI yields a validated binary): daemon service capturing
wg-toolbox → JSON socket, and secubox-dpi-flowcap consuming nDPId JSON instead of
the ndpiReader CSV (PoC stays as fallback).
Publishing webext-v* with the default make_latest would steal the
"latest" release pointer from the Android APK release, breaking the
APK endpoint's /releases/latest/download/secubox-toolbox-android.apk
fallback. Publish the webext release with make_latest:false and point
the /wg/toolbox.xpi endpoint + fetch helper at the tag-pinned download
URL instead.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
web-ext lint: 0 errors, 2 benign warnings (AMO data-collection
declaration is submission-time, tied to the signing follow-up;
service_worker-ignored is the intentional cross-browser pattern).
optional_host_permissions needed FF128 and was unused at MVP.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The two boards fail in the cross-arm64 chroot stage of build-image.yml
and (even with fail-fast: false) block the downstream release.yml job
from publishing the OTHER boards' images. Releases v2.13.9 through
v2.13.12 all hit this trap.
This change keeps both entries available via workflow_dispatch (operators
can still build them on-demand), but removes them from the push-tag
scheduled matrix so mochabin / vm-x64 / rpi400 actually ship on tag.
Board support files (board/espressobin-*/, image/build-image.sh
--board espressobin-*) stay in tree.
- build-image.yml matrix excludes espressobin-v7 + espressobin-ultra
on push and workflow_call.
- build-image.yml workflow_dispatch choice list keeps both entries
flagged as on-demand only.
- Release notes template drops the two image rows, adds a note
explaining the on-demand path.
- Install instruction adjusted (MOCHAbin + Raspberry Pi 400).
The bindeb-pkg target builds linux-image-*.deb FIRST then attempts the
linux-headers-*.deb. The headers build can fail (silent in CI log) while
linux-image is already on disk and ready to deploy. The deployment-
critical artifact for the MochaBin mesh integration is linux-image —
headers/libc-dev are needed only for out-of-tree module builds against
this kernel (which we don't do today).
Add `if: always()` to both Collect and Upload steps so partial
artifacts survive a late bindeb-pkg failure. Downgrade if-no-files-found
from `error` to `warn` so a fully empty /tmp/artifacts/ doesn't mask
the real upstream failure with a misleading artifact error.
Previous attempt used a heredoc with EOF marker at column 0, which
YAML parses as a new top-level key, breaking the workflow file.
Replace with the same loop pattern used elsewhere in the step
(echo | tee -a with iteration over codename suites).
Ubuntu's security.ubuntu.com and archive.ubuntu.com host amd64/i386
only. arm64 packages live on ports.ubuntu.com.
After `dpkg --add-architecture arm64` + `apt-get update`, the runner
404s on:
https://security.ubuntu.com/ubuntu/dists/noble/main/binary-arm64/Packages
Fix: pin existing entries to [arch=amd64] and add [arch=arm64] entries
pointing to http://ports.ubuntu.com/ubuntu-ports/. Detected codename
via /etc/os-release so the workflow stays compatible across runner
updates (jammy/noble/numbat).
Unblocks libssl-dev:arm64 install for cross-arm64 kernel .deb build.
dpkg-buildpackage -aarm64 (cross-build) resolves Build-Depends to the
target arch. The kernel package's debian/control declares
`Build-Depends: libssl-dev` without arch suffix, so dpkg-checkbuilddeps
looks for libssl-dev:arm64 and fails when only libssl-dev (amd64) is
installed.
Add `dpkg --add-architecture arm64` before `apt-get update` so the
arm64 package index is available, then install libssl-dev:arm64
alongside the host libssl-dev.
This is a pre-existing CI gap, exposed by run 26816056566. Unrelated
to the secubox-mesh fragment (signed-regdb was a red herring).
Append WIRELESS/WIFI block to OpenWrt-merged kernel fragment so the next
build-kernel.yml run ships:
- mac80211 + 802.11s mesh + HWMP (=m)
- ath10k_pci for QCA9880 / WLE900VX backhaul mesh
- mt76x2u for MT7632U USB access client
- ath9k_htc for AR9271 firmware-free audit dongle
- signed regulatory DB (FR domain lock via secubox-mesh postinst)
Add four sanity-check greps in build-kernel.yml so a future fragment
regression fails the CI before producing a kernel .deb without WiFi.
Manual workflow_dispatch of build-kernel.yml required to materialise
the new linux-image-*.deb (artifact path not yet wired into
build-image.yml — see tracking issue).
After #427 added '-a matrix.arch' to dpkg-buildpackage, debhelper now
correctly invokes the cross-toolchain binaries:
- aarch64-linux-gnu-strip (used by dh_strip)
- aarch64-linux-gnu-objdump (used by dh_makeshlibs)
…which were not installed on the amd64 CI runner. Result: arm64 builds
of arch-specific packages shipping prebuilt arm64 ELF binaries failed:
- secubox-sentinelle-gsm — dh_strip on bin/secubox-redsea
- secubox-daemon — dh_makeshlibs on secuboxctl
Install binutils-aarch64-linux-gnu (~5 MB) only when matrix.arch == arm64.
Folded into the existing apt step to keep one network round-trip.
Single workflow change > per-package debian/rules no-op overrides:
- no need to revisit every future arch-specific package
- preserves real strip benefit (smaller .deb)
- fixes the CI gap at the source
This is the final piece behind #427 + #425 to unblock APT publish on
arm64 since v2.13.0.
build-packages.yml ran `dpkg-buildpackage -us -uc -b` without -a, so on the
amd64 ubuntu-latest runner the arm64 matrix slot defaulted to amd64. For
Architecture-specific packages (notably secubox-sentinelle-gsm =
Architecture: arm64) that means no binary is produced and dpkg-genbuildinfo
fails with "binary build with no binary artifacts found" → that job has
been red since v2.13.0 and gated the APT publish step.
For amd64 matrix jobs the change is a no-op. For arm64 matrix jobs on the
amd64 runner, packages that don't compile native code (Python only, or
prebuilt arm64 binaries as in sentinelle-gsm) cross-stamp the .deb fine
with -a arm64. Compiling-from-source arm64 packages aren't a thing in this
repo today.
Verified locally: dpkg-buildpackage -a arm64 -us -uc -b on
secubox-sentinelle-gsm produces a clean
secubox-sentinelle-gsm_0.4.0-1~bookworm1_arm64.deb (with #425's shlibdeps
-Xsecubox-redsea, already on master).
The --kiosk flag in image/build-rpi-usb.sh was parsed but never acted on
(INCLUDE_KIOSK toggled but no install block existed), so the rpi400 image
shipped without chromium / X / kiosk service. The boot menu's apply_mode
"kiosk" branch even references a secubox-kiosk.service that was never
created. Adding all the missing pieces:
- New Step 5.4 block (gated by INCLUDE_KIOSK=1):
- apt install xserver-xorg xinit openbox chromium x11-xserver-utils
- install image/kiosk/secubox-kiosk.sh → /usr/share/secubox/kiosk/
- install image/kiosk/xinitrc → /root/.xinitrc
- create /etc/systemd/system/secubox-kiosk.service (startx on vt7,
Conflicts with getty@tty7; matches apply_mode's expected unit name)
- seed /var/lib/secubox/boot-mode = "kiosk" so first boot enters kiosk
- set default systemd target to graphical.target + enable the unit
- CI build-all-live-usb.yml: add `extra_args: "--kiosk"` to the rpi400
matrix entry and append `${{ matrix.extra_args }}` to the build invocation.
Pi 400 = keyboard + HDMI, kiosk-by-default is the obvious user-facing
shape; other platforms keep their headless default.
Build #5 cleared the ZRAM choice sed fix (#326) and reached the
bindeb-pkg step, then failed at:
dpkg-checkbuilddeps: error: Unmet build dependencies:
build-essential:native libssl-dev
bindeb-pkg auto-generates a debian/control that Build-Depends on
`build-essential:native`. The previous install list pulled in gcc /
make / libc-dev individually but never the meta-package, so
dpkg-checkbuilddeps failed even though all the underlying tools were
present.
libssl-dev is also in the same error line but it IS already installed
— it gets satisfied once build-essential lands because of the way
checkbuilddeps re-resolves the full Build-Depends graph in one pass.
Ref #319#323#325#326.
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
After three rounds of failed attempts (#323, #325, #326), the root cause
is clear: kconfig `choice` blocks are re-resolved by olddefconfig from
their declared `default <member>` in Kconfig, regardless of what's in
.config. `scripts/config --enable ZRAM_DEF_COMP_ZSTD` applied BEFORE
olddefconfig is silently undone by olddefconfig's choice pass; merging
a fragment that sets `CONFIG_ZRAM_DEF_COMP_ZSTD=y` is overridden the
same way.
The only reliable approach is post-olddefconfig sed surgery on the
choice members. Choice members are leaf symbols with no downstream
build-time deps, so no further olddefconfig is needed after the
override.
Workflow change:
- move scripts/config out of the choice (only set ZSMALLOC/CRYPTO_*)
- run olddefconfig to resolve the rest of the tree
- sed the choice into ZSTD (LZORLE off, ZSTD on)
- log .config before/after the fixup for diagnostics
- keep the final assertion `^CONFIG_ZRAM_DEF_COMP_ZSTD=y`
Ref #323#325#326.
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
merge_config.sh + olddefconfig don't always honour choice-symbol
overrides set in a fragment (kconfig special-cases choices). Run
scripts/config after the merge to set ZRAM_DEF_COMP_ZSTD explicitly
and disable the upstream-default ZRAM_DEF_COMP_LZORLE.
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
CI run 26222437600 failed at the sanity-check: kconfig didn't emit
CONFIG_ZRAM_DEF_COMP="zstd" verbatim because that symbol is the
auto-derived output of the CONFIG_ZRAM_DEF_COMP_<X> choice family,
not a settable variable. Check the choice itself instead.
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
The discover step previously scheduled an amd64 build for every
non-arch-all package, ignoring an explicit `Architecture: arm64`
or `Architecture: amd64` in debian/control. The amd64 job then
failed with `dpkg-genbuildinfo: error: binary build with no
binary artifacts found` because dpkg produces zero binaries for
an arch-foreign declared package on the build host.
Replace the implicit "amd64-always" logic with a case on the
binary stanza's Architecture field:
all -> amd64 (the arch-all .deb deployable on any arch)
any -> amd64 + arm64 (preserve current behavior)
amd64 -> amd64 only
arm64 -> arm64 only
Local dry-run on the current 133-package catalog: 132 amd64 + 1
arm64 combos (secubox-daemon is the only arch-any package), well
under the 256-jobs-per-matrix GH Actions cap.
Read the Architecture from the BINARY stanza (after Package:) so
the Source stanza's Architecture (which Debian convention sometimes
omits) doesn't interfere — awk pattern is `/^Package:/{p=1} p && /^Architecture:/{print $2; exit}`.
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Anti-regression for the double-cache pattern documented in CLAUDE.md
(« Performance Patterns — Double Caching »). Catches the bug class that
caused PR #146 (secubox-waf 17 % sustained CPU under SOC polling)
before it lands.
What's added
============
- scripts/check-dashboard-cache.py — AST-based audit. Walks every
packages/secubox-*/api/main.py, flags hot dashboard routes
(/stats, /alerts, /summary, /dashboard, /metrics, /overview,
/events, /history, /recent, /bans, /sessions, /connections)
whose handler body does per-request I/O (open, read_text,
read_bytes, subprocess.run, Popen) AND doesn't go through a cache
(module-level _cache instance + .get() call, OR
asyncio.create_task(refresh_*) at startup).
* --check : exit 1 on unjustified violations (CI mode)
* --report : human-readable table (default)
* --json : machine-readable
- .cache-lint-allowlist.toml — TOML allowlist for justified
exceptions. Each entry needs a daemon + route + justification
(issue link expected). Seeded with the 4 current legacy cases:
secubox-waf (covered by PR #146 — will drop the entries when
merged), secubox-config-advisor /history (admin drill-in, not
polled), secubox-netdiag /connections (real-time TCP state),
secubox-tor /summary (small file read, follow-up cleanup).
- .github/workflows/dashboard-cache-check.yml — mirror of
license-check.yml. Runs the lint + the self-tests on PRs that
touch packages/**/api/main.py, the lint script, the allowlist,
or the workflow itself.
- docs/CACHE-PATTERN.md — short remediation guide. Three accepted
shapes (read-through, background-refresh, pure in-memory), code
example, allowlist syntax, references to the canonical
implementations (secubox-crowdsec for read-through, secubox-system
for background-refresh).
Tests (14/14 pass)
==================
- StatsCache pattern detection (compliant cases)
- Non-compliant detection: open, subprocess.run, multiple I/O ops
- Pure in-memory handler = compliant
- Background-refresh task = compliant
- Non-hot route (/health) not audited
- @router.get treated like @app.get
- Allowlist suppresses by exact (daemon, route) key
- Allowlist doesn't cross-pollinate across routes
- Missing allowlist file = no error
- CLI --check exits 1 on unjustified
- CLI --check exits 0 when clean
- CLI --json output well-formed
- **Real repo audit with seeded allowlist passes --check**
(sanity gate: catches regression if anyone removes an allowlist
entry without fixing the daemon)
Baseline against the real packages/ tree
========================================
122 daemons audited, 118 compliant, 4 with findings (all
allowlisted). When PR #146 merges, the secubox-waf entries can be
removed from the allowlist — the next lint run will then prove the
fix is wired correctly.
Follow-up
=========
- Drop secubox-waf allowlist entries when #146 merges.
- Per-daemon follow-up issues for the remaining 3 (config-advisor,
netdiag, tor) — not blocking, allowlist documents the rationale.
- Consider extending HOT_ROUTES to /health / /status once the small
daemons that call `systemctl is-active` adopt a 10 s cache.
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Five package builds failed on v2.9.0 first attempt, and the release
pipeline cascaded to skip all downstream jobs (collect/publish/images/
live-usb/create-release) because GH Actions treats *any* matrix failure
as a global failure for `needs:` purposes. This commit fixes both
root causes:
Package fixes:
- secubox-eye-square: drop debian/compat (conflicted with control's
`Build-Depends: debhelper-compat (= 13)`)
- secubox-defaults: same — drop debian/compat
- secubox-metoblizer: switch from legacy `Build-Depends: debhelper
(>= 13)` (which then requires a debian/compat file) to the modern
`Build-Depends: debhelper-compat (= 13)` virtual package
- secubox-haproxy: override dh_usrlocal — it can only rehome /usr/local
directories (Policy 9.1.2), not the individual admin tools we
deliberately drop there per issue #44
Pipeline resilience:
- build-packages.yml `collect` job: `if: always() && != cancelled` so we
bundle whatever .deb files the matrix produced even when entries failed
- build-packages.yml `publish` job: predicate now reads collect's success
rather than the matrix's overall conclusion
- release.yml `build-images`, `build-live-usb`, `publish`,
`create-release`: all gain `if: always() && needs.build-packages.result
!= 'cancelled'` so a partial build matrix doesn't black-hole the
release. `create-release` already gracefully skips its
`if: needs.build-images.result == 'success'` download steps when
images failed, so the partial release ships what's available.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Follow-up to the previous matrix-cap fix:
1. YAML/shell quoting hell: the inline ${{ github.event.inputs.arch }}
substitution inside a single-quoted heredoc broke parsing on push:tags
events (where inputs are null). Moved both inputs to step `env:` block
so the shell script reads them as ordinary env vars.
2. Find pattern was `-name control -path "*/debian/*"`, which also matched
dpkg-build artifacts at `packages/<pkg>/debian/<pkg>/DEBIAN/control` on
any non-clean checkout. CI's runners are clean so it worked there, but
the pattern is fragile. Tightened to `-path "*/debian/control" -not
-path "*/debian/*/DEBIAN/control"`.
Verified locally: 132 amd64 + 2 arm64-eligible = 134 combos, well under
GH Actions' 256 jobs-per-matrix cap.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Discover now emits a single flat list of {package, arch} entries
pre-filtered to skip arch-all+arm64 combos. Build uses `matrix.include`
instead of a cross-product, so the job count == filtered combo count
regardless of package catalog size.
Why: with 134 packages, the previous package × [amd64, arm64] expansion
produced 268 matrix jobs, exceeding GitHub Actions' hard 256 cap. The
build job was silently skipped (zero matrix entries scheduled), which
in turn skipped collect → publish → build-images → build-live-usb →
create-release. Releases since v2.7.x have shipped only the installer
ISO artifact for this reason.
The post-discover skip step for "arch-all + arm64" becomes dead code
but remains harmless (the condition can never fire now); leaving in
place so a future re-introduction of cross-product matrices doesn't
silently re-break.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Add secubox-metoblizer package (centralized log aggregator)
- Streamlit: double-buffer caching, power management UI, autostart
- Streamlit UI: consolidate Components into Status tab
- GitHub Actions: replace espressobin-v7 with mochabin for live USB builds
- Various UI improvements
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Fix glob pattern in bash test (glob in [[ -f ]] doesn't work correctly)
- Use find command instead of glob for reliable kernel detection
- Add fallback to extract ARM64 kernel from live USB image if not in rootfs
- Add verification step in GitHub Actions to check boot files after build
- Sort kernel files by version to get latest when multiple exist
Fixes missing vmlinuz/Image on multiboot USB issue.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Install live-boot package and rebuild initramfs with live-boot scripts
- Create squashfs filesystem (878MB) on data partition sda4
- Update boot.scr with live boot parameters (boot=live, toram)
- Fix wiki sidebar links from [[Page|Display]] to [Display](Page)
- Add Eye-Remote wiki page documentation
- Add sync-wiki.sh script for wiki repository sync
- Add patch-multiboot-efi.sh for post-build EFI patching
Partition layout:
- sda1 (512MB): EFI with kernel, initrd, dtbs, boot.scr
- sda2 (3GB): ARM64 rootfs reference
- sda3 (3GB): x86 rootfs for VirtualBox/QEMU
- sda4 (9.5GB): Data + /live/filesystem.squashfs
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add test-menu-system job for pytest validation
- Update VERSION to 2.2.0
- Add create_release manual trigger option
- Update release notes with radial menu features
- Change tag pattern to eye-remote-v* for clarity
- Add PR trigger for CI on changes
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Eye-Remote-Hardware.md: GPIO pinout, USB OTG, DPI timings, gadget modes
- Update CI workflow to v2.0.0
- Fix build script: add DNS for chroot network access
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Complete rewrite of build-eye-remote-image.sh for pure offline boot:
- Use QEMU ARM chroot to pre-install all packages at build time
- Expand image by 1GB for pre-installed packages (~500MB)
- Pre-installed: chromium, nginx, lightdm, openbox, python3-pil
- Configure lightdm autologin, openbox autostart, nginx in chroot
- Create secubox user with all groups during build
- No rc.local/firstrun needed - boots directly into kiosk
Build requirements: qemu-user-static, binfmt-support
Boot time comparison:
- v1.8.x: ~10 min (package download required)
- v1.9.0: ~60 sec (ready immediately, no internet)
Closes#30
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Downloads RPi OS Lite automatically
- Builds complete Eye Remote SD card image
- Compresses with xz and generates checksums
- Uploads to releases on version tags
- Supports manual trigger with WiFi/hostname options
- Part of Eye Remote v1.8.0
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- build-all-live-usb.yml: Check for .img.gz first since build-image.sh
already compresses and removes the .img file
- build-rpi-usb.sh: Handle multiple kernel/initrd files by selecting
the latest version instead of globbing which breaks cp
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add ARP-based IP collision detection for multi-device environments
- MAC-based pseudo-random IP offset to spread devices across range
- Gratuitous ARP announcement to prevent IP conflicts
- Fix EspressoBin DSA network: target wan interface, not eth0 CPU port
- Static IP fallback 192.168.255.250 when DHCP unavailable
- Sync all build scripts to version 1.7.0
- Add screenshot script with 90+ module URLs
- Add mock HTML screenshots for documentation
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
GitHub Actions doesn't allow matrix.* in job-level 'if' conditions.
Moved platform filtering to step-level with skip output variable.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add build-all-live-usb.yml with matrix for x64, EspressoBin V7, and RPi 400
- Update release.yml to include live USB builds in release pipeline
- Add RPi 400 netplan configuration for Ethernet and WiFi
- Deprecate build-live-usb.yml (now wrapper to unified workflow)
Platforms supported:
- x64 (amd64) - GRUB UEFI/BIOS boot
- EspressoBin V7 (arm64) - U-Boot distroboot with embedded eMMC image
- Raspberry Pi 400 (arm64) - Native Pi firmware boot
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Skip signing and deploy steps when secrets are not configured
instead of failing the entire workflow.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add create-qemu-arm64-vm.sh script for ARM64 emulation on x86 hosts
- Add wiki/QEMU-ARM64.md documentation
- Update CI workflows to include RPi 400 and VM scripts in releases
- Update release notes to document all platforms
Platforms now supported:
- MOCHAbin (Armada 7040)
- ESPRESSObin v7/Ultra (Armada 3720)
- Raspberry Pi 400 / Pi 4
- VirtualBox x64
- QEMU ARM64 emulation
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
