C2: fix IP collision — picobrew's 10.100.0.140 is already used by
secubox-frigate on br-lxc (verified /data/lxc/frigate/config); reassign to
10.100.0.150 everywhere (ctl, unbound drop-in, tests, specs/plans docs).
C3: postinst no longer aborts under set -e when the unit is masked
(unmask before enable, tolerate failure like the start line already does).
C4: drop ReadWritePaths=/sys/bus/w1 from the unit — the path doesn't exist
on the board and phase 1 ships no sensor controller (preserved untouched in
lib/stillwatch/); no other sensor paths were present to remove.
C5: run the API as User=secubox/Group=secubox instead of root, matching the
sudoers grant (secubox -> picobrewctl) and the repo's single-privileged-
surface rule; drop the now-pointless ExecStartPost socket chown/chmod;
NoNewPrivileges=no kept and documented (required for sudo to work). No
shared parent (/run/secubox, /etc/secubox, /var/log/secubox) is touched.
I1: prerm no longer removes the nginx conffile on upgrade (only
remove/purge), so an upgrade keeps the API/panel route alive.
I2: /status, /start, /stop become plain `def` handlers — they call the
synchronous, blocking `_ctl` (subprocess.run, 20s timeout) and must not run
on the async event loop.
I3: add python3-fastapi | python3-pip to Depends (api/main.py imports
fastapi directly).
I4: drop ConditionArchitecture=arm64 and the stale 1-Wire/I2C sensor
comment — phase 1 is a sensor-less LXC appliance, nothing arch-specific.
I5/I6: rewrite debian/control's Description and README.md to describe what
phase 1 actually ships (LXC-hosted picobrew_pico server + DNS rewrite +
panel), moving fermentation/sensor tracking to "later phase, not shipped".
I7: panel now links to the real PicoBrew interface (https://<lxc-ip>/),
shown only while the container is running.
I8: document in the plan's manual verification recipe that LAN -> br-lxc
forwarding (default-DROP nftables) must be checked before the first real
device test, since that test is also the first test of that network path.
Mineurs: panel act()/load() gain try/catch and response.ok checks with
visible error feedback (previously implemented alongside C1, already
committed: argv fix + guard test, _wait_running polling loop replacing the
fixed `sleep 3` in cmd_install, cmd_status validating the pinned SHA before
emitting it, and a comment documenting PICOBREW_SESSION_FILE as test-only).
Revue pré-vol : la config nginx faisait écouter nginx sur :80 tout en
proxifiant vers 127.0.0.1:80, où Flask écoute déjà. nginx se serait parlé à
lui-même. nginx ne prend que le 443 ; les appareils non-Z joignent Flask
directement en :80.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Plan d'implémentation de la phase 1 de la spec : 9 tâches TDD, 46 étapes.
Le provisionnement réel (debootstrap, root, réseau) n'étant pas testable en
unitaire, les parties décidables sont isolées derrière des entrypoints cachés
testables : __guard (validateur de SHA), __emit-config (config LXC) et
__emit-nginx (terminaison TLS). Le reste est couvert par une recette de
vérification manuelle explicite plutôt que par de faux tests.
Auto-revue : écart trouvé et corrigé — la spec place le TLS série Z en phase 1
alors que le plan le renvoyait hors périmètre ; or la série Z n'accepte que
HTTPS, un appareil Z aurait donc été inutilisable. Tâche 8 ajoutée.
Le contrôleur de capteurs (992 lignes) est déplacé intact dans lib/stillwatch/
plutôt que supprimé : la phase 2 le reprendra sans archéologie git.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Task 1: reconnaître modeEscalate (fail-closed préservé). Task 2: champ
escalateBan (compteur SÉPARÉ, défauts 24h/3), branche self-contained qui observe
(action=detect, passe) sous le seuil et banne au seuil via crowdsec+writeBan —
gatée sur escalateBan, JAMAIS s.ban, sinon les signaux se mélangent. nil →
comme detect. Task 3: déploiement + preuve live (observation; le ban au seuil
est prouvé déterministement par les tests unitaires).
Un seul site d'appel de Match. Mutation obligatoire par tâche.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Task 1: parser `mode` par catégorie (défaut block, fail-closed sur valeur
inconnue). Task 2: Match remonte le mode; le chemin detect journalise
action=detect et laisse passer, sans jamais bannir. Task 3: déploiement gk2 +
preuve sur trafic réel (bascule cve_2024 en detect à chaud, puis restauration).
Un seul site d'appel de Match (main.go:376) — rayon de souffle contenu.
Chaque tâche exige une preuve par mutation: casser le comportement, constater
l'échec du test, restaurer.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
6 tâches TDD : manifestes, résolution d'état, sondes, diff ordonné, profiler
scan, CLI + paquet. 55 tests.
Phase 1 ne peut rien allumer ni éteindre — un test verrouille l'absence de la
commande apply, et observe.py n'exécute que des commandes de lecture.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
* docs(device-guardian): Project A consolidation design — merge 5 device modules into nac (ref #817)
* docs(device-guardian): Project A implementation plan — 9 tasks (ref #817)
* feat(nac): SQLite device store + idempotent legacy migration (ref #817)
Add packages/secubox-nac/api/store.py — pure-stdlib sqlite3 module with
canon_mac(), DeviceStore (WAL, best-value upsert, list/get/count,
history), and migrate_legacy() for idempotent, per-source best-effort
import of mac-guard/device-intel/iot-guard legacy shapes. Reserved
Project-B columns (plane/provenance/geo_cc/geo_asn) created but unused.
Task 1 of the Device Guardian consolidation plan (docs/superpowers/plans/
2026-07-05-device-guardian-consolidation.md).
* fix(nac): store — preserve first_seen, skip corrupt legacy sources, no stray db, SPDX (ref #817)
* feat(nac): unified dnsmasq+ISC+ARP discovery merge (ref #817)
* fix(nac): discovery — latest ISC lease wins the IP on renewal (ref #817)
The merge loop only updated ip/source on a strict rank increase
(incoming_rank > existing_rank), so equal-rank sightings for the same
MAC never touched ip/source. Since dnsmasq -> isc -> arp is processed
in non-increasing rank order, multiple ISC dhcpd.leases blocks for one
MAC (a fresh block per renewal) are equal-rank ties, and the first
(oldest) block's ip was frozen instead of the latest one winning, as
the original secubox-device-intel._get_dhcp_leases did.
Change the condition to incoming_rank >= existing_rank so the latest
same-or-higher-rank sighting updates source, and overwrites ip when
the incoming ip is non-empty. Hostname rule and the lower-rank branch
are unchanged, so a later ARP sighting still cannot clobber a
higher-rank ip/hostname/source.
Add test_isc_latest_lease_wins: two ISC lease blocks for one MAC with
ip/hostname renewal, asserting the latest block wins. Confirmed this
test fails against the pre-fix >-only logic.
* feat(nac): absorbed enrichers — OUI vendor + device-type/risk + OpenWrt fingerprint (ref #817)
* fix(nac): enrich — is_router strictly vendor-based; cover classify/oui edge cases (ref #817)
* feat(nac): background collector + double-cache; handlers def, no inline scan (ref #817)
Task 4 of the Device Guardian consolidation: a Collector background
loop now owns all discover -> enrich -> upsert work (replacing
_monitor_clients), publishing an in-memory snapshot(). The status,
clients, and client/{mac} handlers become plain def reading the SQLite
store / collector cache instead of calling _discover_clients() inline
on every request -- the #808 aggregator SPOF.
* fix(nac): finish #808 — def-convert alerts/list_quarantine/summary; first_seen + recent_events (ref #817)
alerts/list_quarantine were still `async def` calling blocking
_discover_clients() inline on the shared aggregator loop; summary
awaited alerts on top of that (double loop-block). Converted all three
to plain `def` reading the SQLite store/collector snapshot instead,
completing the #808 SPOF fix started for status/clients/client.
Also: collector.cycle_once() now stamps first_seen on newly discovered
devices (store.upsert keeps the earliest via COALESCE), and get_client's
recent_events merges the SQLite device_history (client_joined) with the
JSON event log so a device's join event shows up again.
* feat(nac): allow/deny actions + def-convert action handlers (finish #808) (ref #817)
Task 5: fold mac-guard's allow/deny into nac's zones/nft. New
POST /allow/{mac} and /deny/{mac} set allow_state in the SQLite store
and mirror it into nft's existing inet secubox_nac blocked/lan_allowed
sets via _nft_add_element/_nft_delete_element (mac-guard's separate
inet secubox_mac_guard table stays retired; its element migration is
Task 9).
Carried from Task 4 review: zones, add_to_zone, remove_from_zone,
approve_client, ban_client, update_client, quarantine_client and
unquarantine were still async def wrapping blocking subprocess/nft
calls on the shared aggregator loop (#808). Converted to plain def so
FastAPI threadpools them off the loop; add_to_zone/ban_client's webhook
notify now fires via a new _fire_webhook_sync helper
(asyncio.run_coroutine_threadsafe against the loop captured at
startup) since a plain def can no longer await _notify_webhooks
directly.
Tests: tests/test_actions.py mocks nft via monkeypatched
_nft_add_element/_nft_delete_element/_nft_list_set (+ subprocess.run,
since ban_client/unban_client build raw nft argv) into an in-memory
set-membership dict; covers deny/allow/allow-then-deny, zones,
add_to_zone+approve_client, and ban_client+unban_client.
* feat(nac): absorbed endpoints — vendors/scan/probe/mdns/groups/export (ref #817)
* fix(nac): /scan enriches + validates subnet; /probe IP + CSV injection guards (ref #817)
* feat(nac): retire mac-guard/device-intel/iot-guard via 308 redirects; auth-gate network-anomaly (ref #817)
secubox-mac-guard, secubox-device-intel and secubox-iot-guard are
superseded by secubox-nac's consolidated Device Guardian. Their
api/main.py are reduced to thin shims: every route 308-redirects to
its secubox-nac equivalent (/devices->/clients, /device/{mac}->
/client/{mac}, /whitelist|/allow->/allow, /blacklist|/deny->/deny,
/scan, /vendors, /groups, /export/* pass through unchanged, and a
catch-all forwards anything else), with an X-SecuBox-Deprecated
header. /health stays a plain 200 so liveness probes never 308-loop.
Package removal is a separate, gated follow-up.
secubox-network-anomaly stays (separate traffic-anomaly engine) but
its /status endpoint was unauthenticated while every sibling route
already required a JWT; it now matches.
* feat(nac): webui — vendor/type/risk columns, groups view, export; drop retired webuis (ref #817)
- clients.js: render enriched /clients fields — oui_vendor, device_type,
risk_level (color-coded badge)/risk_score, and OpenWrt/SecuBox/Router
fingerprint badges. All values are attacker-influenceable (DHCP
hostname/vendor) and are rendered exclusively via E()/textContent,
never innerHTML. Null/missing device_type/vendor/risk render as
"unknown"/"—", never "null". Adds an Export CSV button (direct link to
GET /export/csv — auth via the existing SSO session cookie, same as
every other same-origin nav link in this app; no new auth scheme).
- groups.js (new): list/create/delete device groups and assign a device
to a group, hitting GET/POST/DELETE /groups* — mirrors zones.js's
structure and auth (sbxFetch).
- nav.js: register the new "Groupes" tab alongside Zones.
- Remove packages/secubox-{mac-guard,device-intel,iot-guard}/www — their
functionality is now consolidated into nac; secubox-network-anomaly/www
is untouched (stays standalone per plan).
Note for Task 9: device-intel/iot-guard's debian/rules unconditionally
`cp -r www/<name>/.` — now broken since www/ is gone; mac-guard's rule is
already guarded (`[ -d www ] && ... || true`). Needs a debian/rules fix
when Task 9 retires/redirects these packages.
* chore(nac): packaging — ieee-data dep, nft migration, retirement redirects, changelog 3.0.0; fix retired debian/rules www (ref #817)
* fix(nac): whole-branch — lazy init for in-aggregator mount, collector off-loop, sqlite lock, batched zone lookup, sentinel-preserve (ref #817)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(nac): risk_score split — known router low, unknown router high (rogue-AP signal) (ref #817)
Split the router risk axis out of the generic device-type/open-ports
scoring: a router/gateway already known (allow-listed or already in the
store) is trusted infrastructure -> low risk; a newly-seen/unknown
router is a rogue-AP signal -> high risk, regardless of open ports.
Removes the old flat -10 "trusted router" discount. Non-router devices
keep the existing scoring unchanged.
risk_score() gains an is_known=False parameter (backward compatible).
collector.cycle_once() and main.scan() now compute is_known from the
prior store row (existing row or allow_state == "allow") before
upserting, reusing the row they already fetch for is_new/enrichment.
* feat(nac): mesh-sync (p2p peers as devices) + fingerprinted-router classification for rogue-AP risk (ref #817)
Part A: absorb iot-guard's missed "mesh-sync" capability. GET /mesh/peers
and POST /mesh/sync now live in nac, reading live P2P peers over
/run/secubox/p2p.sock (HTTP-over-UDS GET /peers, fail-safe -> [] on any
error/timeout/absent socket) and upserting one sb:-prefixed synthetic
device per peer (iot-guard's exact md5-hash MAC scheme), device_type
"mesh_node", source "mesh". iot-guard's retired redirect-shim catch-all
already forwards /mesh/* here, so no shim change needed.
Part B: classify_device_type's "router" keyword list is much narrower
than ROUTER_VENDORS (misses tp-link/asus/linksys/d-link/gl.inet/xiaomi/
huawei), so a router-vendor MAC with a generic hostname fingerprinted
is_router=True still classified as device_type="unknown" and skipped
risk scoring entirely (the omit-unknown guard) -- the known-router-vs-
rogue-AP HIGH signal never fired for the most common vendor case. Both
enrichment call sites (Collector.cycle_once and /scan) now reclassify
device_type="unknown"+is_router to "router" before scoring.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(nac): postinst ensures secubox_nac table + zone sets (incl lxc_zone)
Nothing shipped created nac's inet secubox_nac table or its zone sets — they
were assumed to pre-exist (they don't: the live board had no such table), so
operator 'move to zone' failed for every zone and the mac-guard element
migration silently no-op'd. postinst now creates the table + all ZONES sets
(lan_allowed, lxc_zone, iot_zone, guest_zone, quarantine_zone) + blocked,
idempotently and best-effort, before the migration. Makes operator moves to
the LXC Containers zone work on a fresh install. nft commands validated on
gk2 (create/idempotent re-add/add-element/list), no live drift left.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
---------
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* docs(sentinel): design — inline IOC gate + async Go analyzer + spyware packs (ref #823)
* docs(sentinel): implementation plan — 12 tasks (ref #823)
* feat(sentinel): IOC model + IOCSet matchers (ref #823)
* feat(sentinel): pack loader + live-overlay merge + hot-reload (ref #823)
Add internal/sentinel/pack.go: Pack (JSON base/overlay content bundle),
LoadBasePack, MergePacks (base + N overlays, override by Type+Value via
IOCSet's per-type maps), and Loader (NewLoader/Set/MaybeReload/YaraRules).
Loader mirrors cmd/sbxmitm/policy.go's LoadPolicy/maybeReload mtime-based
hot-reload pattern, adapted for a directory of a DYNAMIC *.json file set
(glob-based) rather than policy.go's fixed Target list. Fail-safe: a
corrupt overlay pack is skipped+logged, never wipes the base or panics;
MaybeReload is a cheap stat-only no-op when nothing on disk changed;
Set()/YaraRules() read via atomic.Pointer so a reload swap never torn-reads
a concurrent hot-path lookup.
Adds JSON tags to Task 1's IOC struct (type/value/class/severity/source/
action) so packs round-trip through encoding/json.
* fix(sentinel): pack loader — fail-closed on base-wipe + throttle MaybeReload + url_regex override + yara dedup (ref #823)
Review found 1 Critical + 2 Important + 1 Minor in Task 2 (pack loader):
- Critical: a vanished/degraded base pack directory (deleted dir, mount
blip, mid-dpkg-upgrade window) could silently empty the live detection
set on the next MaybeReload. loaderState now tracks baseCount; a reload
that would drop base IOCs from >0 to 0 is refused (prior state kept,
warning logged) instead of swapped in.
- Important: MaybeReload() is called per-flow by Task 3's inline gate: add
a 5s default throttle (mirrors cmd/sbxmitm/policy.go's reloadThrottle)
so a hot-path call is a pure no-op until the window elapses.
- Important: url_regex IOCs previously only appended, so a live overlay
could never override a base pack's URL rule (base always matched
first). Add urlIndex (pattern -> slice index) so re-adding the same
pattern replaces the entry in place, keeping deterministic match order.
- Minor: YaraRules() now dedups identical rule bodies across base+overlay
packs instead of emitting duplicates.
go test ./internal/sentinel/: 21/21 PASS. go vet/gofmt/go build clean.
* feat(sentinel): inline gate matcher (fail-open, highest-severity) (ref #823)
* feat(sentinel): bbolt verdict store (pure-Go, WAL-free, TTL prune) (ref #823)
* feat(sentinel): action scorer — heuristic/low-confidence forced report-only (ref #823)
* feat(sentinel): bounded fire-and-forget mirror channel (drop-with-count) (ref #823)
* fix(sentinel): mirror Emit always copies body — no aliasing of caller's hot-path buffer (ref #823)
* feat(sbxmitm): inline Sentinel gate — neutralize high-confidence, mirror rest, fail-open (ref #823)
Wire internal/sentinel's inline IOC gate into the shared mitmPipeline response
path (covers both the transparent R3 and CONNECT accept paths without drift):
build a FlowMeta from Host/URL/ClientIP/MacHash after up.Do, call the gate, and
on a HIGH-CONFIDENCE known-infra hit neutralize inline — block/sinkhole serve a
Sentinel block page instead of the upstream response, strip serves the response
with an emptied body. Everything else (miss, heuristic, low-confidence, report)
is mirrored bounded fire-and-forget to the async analyzer and proceeds unchanged.
Fail-open everywhere: SENTINEL_ENABLED unset/false, a pack-load failure, a nil
hook, or a gate panic all degrade to a transparent passthrough — a Sentinel
setup problem can never break browsing. With the engine off, behavior is
byte-identical to today. Env: SENTINEL_ENABLED / SENTINEL_PACK_DIR /
SENTINEL_OVERLAY_DIR / SENTINEL_MIRROR_SOCK.
Hot path (BenchmarkSentinelInspectMiss, -benchmem): 101.0 ns/op, 0 B/op,
0 allocs/op — one throttled gate Match + a non-blocking mirror Emit; no YARA or
heavy analysis inline.
go test ./cmd/sbxmitm/... -race → ok (all pre-existing + 7 new green).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(sentinel): sbx-sentinel async daemon scaffold + store wiring (ref #823)
* feat(sentinel): YARA engine (cgo, build-tagged) + no-cgo stub (ref #823)
Add internal/sentinel/yara.go (cgo && yara) wrapping
github.com/hillu/go-yara/v4 for compiled-rule body scanning, and
yara_stub.go (!yara) as the no-op default: NewYaraEngine always
succeeds, Scan/Analyze always return nil. Both expose identical
signatures (YaraEngine, NewYaraEngine, Scan, Analyze, Close) so
sbx-sentinel/sbxmitm can wire a YaraEngine unconditionally.
yara_test.go (untagged) proves the default build works with YARA
disabled. yara_real_test.go (yara tag) compiles a trivial EICAR rule
and exercises the real libyara-backed engine — split into its own
file since Go build constraints are file-scoped and can't mix a
tagged and untagged test in one file.
go-yara is now an indirect go.mod requirement and vendored, but is
only reachable via the yara build tag: default `go build ./...` and
`CGO_ENABLED=0 go build ./...` both stay clean, no libyara/cgo
required. `-tags yara` needs libyara-dev + a C toolchain (documented
in yara.go's header).
* fix(sentinel): YARA verdict confidence 90 (>= threshold) so a confirmed malware match actually strips, not report-only (ref #823)
* feat(sentinel): behavioral engine — beaconing + zero-click heuristics (report-only) (ref #823)
* fix(sentinel): tighten one-time-link heuristic — exclude file-extension + base64 shapes (cut zero-click FPs) (ref #823)
* feat(sentinel): spyware indicator engine + commercial-spyware base pack (MVT/CitizenLab) (ref #823)
* fix(sentinel): base-pack declared-block IOCs severity>=85 so they actually block + general guard test (ref #823)
Three shipped base-pack IOCs declared action:"block" but severity:80,
below scorer.go's HighConfidenceThreshold=85 — FinalizeAction silently
downgrades any Confidence below that threshold to report-only, so these
entries never actually blocked despite being declared as known-bad:
- packs/base/malware.json: ip 198.51.100.23 (class malware)
- packs/base/malware.json: cert_sha1 ...01234567 (class trojan)
- packs/base/botnet.json: ja3 deadbeef... (class botnet_c2)
Bumped all three to severity 90 so they survive FinalizeAction and
actually block, matching every other declared-block entry in the base
packs.
Added TestBasePackDeclaredBlockIsHighSeverity, a general guard over
every base pack file/entry: any IOC with an auto-neutralize action
(block/strip/sinkhole) must carry Severity >= HighConfidenceThreshold.
This generalizes the narrower TestBasePackKnownInfraIsHighSeverity
(spyware classes only) so a future pack entry in any class can't ship
a "block" that silently never blocks. Confirmed the new test fails red
against the pre-fix JSON (all 3 offenders reported) and passes green
after the severity bump.
* feat(sentinel): proposal/solution reporter — RenderReport(Verdict) (ref #823)
* feat(sentinel): wire real analyzers into sbx-sentinel + read-only status HTTP surface (ref #823)
* feat(sentinel): live threat-feed → overlay pack fetcher (fail-safe, atomic) (ref #823)
* feat(sentinel): live feed overlay + reporter + daemon analyzer wiring + packaging (ref #823)
* fix(sentinel): feeds fetcher — add curl --fail so HTTP 4xx/5xx is treated as a fetch failure (ref #823)
* fix(sbxmitm): drop mis-wired ClientIP from inline Sentinel FlowMeta (source IP must not match destination-C2 IP IOCs) (ref #823)
---------
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* docs(media-buffer): design spec — dashcam capture + replay + metatag lifecycle (ref #812)
* docs(media-buffer): phase-1 implementation plan (capture+janitor+API+DPI tab) (ref #812)
* feat(sbxmitm): media buffer store + metatag writer (ref #812)
* fix(sbxmitm): media-buffer review — 4096 atomic-append cap + *string buffer_ref (ref #812)
Task 1 review (Important): mediaBufferLineMax 8192→4096 to match mediacatch.go's
PIPE_BUF atomic-append guarantee across 4 workers; BufferRef string→*string so the
janitor (Task 3) can flip it to JSON null on eviction.
* feat(sbxmitm): tee media up/download bodies into the buffer (ref #812)
Wire the R4 media buffer tees into mitmPipeline: the download hook wraps
resp.Body and the upload path wraps req.Body via teeReadCloser, capturing
whole-file media into the rolling /data buffer. Add the mbuf field + flags
(--media-buffer default OFF, --media-buffer-root, --media-buffer-per-object).
Design upgrade over the plan sketch: ObjectWriter.Write is now NON-BLOCKING.
Instead of io.TeeReader driving a synchronous os.File.Write on the client read
path (which would add latency / could stall large media on ARM eMMC), Write
copies each chunk into a bounded channel and returns len(p),nil immediately; a
background goroutine started at Capture drains the channel to disk. A full
channel drops the chunk and flags truncated — never blocks, never errors the
proxied flow (design 4.2/7). Close signals the drainer, waits for it to flush
queued chunks + close the file, then appends the metatag; it is idempotent,
deadlock-free (sole closer of ch) and safe if Write is never called or races
Close.
* fix(sbxmitm): media download tee — defer closure so w.Close runs (ref #812)
defer resp.Body.Close() bound the ORIGINAL upstream body at the defer
statement, before the media-buffer tee reassigned resp.Body — so the
tee's Close() (and w.Close, which flushes the sink/metatag/drain
goroutine) never ran on any download. Wrap the close in a closure so
it reads resp.Body at return time instead.
Adds TestTeeDeferOrderingMatchesMainGo, which replicates main.go's
exact defer-then-reassign ordering and fails against the old plain
defer form (verified manually, then restored the fix).
Also: drop the dead write-only ObjectWriter.dropped field (truncated
already covers it), document Capture's reserved contentLen param, and
note that ObjectWriter.Close is allowed to block (post-stream
finalisation), unlike Write.
* feat(sbxmitm): media buffer janitor — time + LRU eviction, metatag survives (ref #812)
SweepOnce evicts a session's bytes (RemoveAll, idempotent) once
first_ts+retentionSecs elapses, then falls back to oldest-first LRU
eviction if the buffer root (media bytes only, log excluded) still
exceeds sizeCeilBytes. Eviction never rewrites the append-only
metatag log in place — it appends a corrected line with the same id
(expired:true, buffer_ref:null); readers dedup by id, last line wins,
so the flip is durable and safe across the 4 concurrent sbxmitm
worker processes each running their own RunJanitor (idempotent
RemoveAll + harmless duplicate flip lines).
RunJanitor ticks SweepOnce every 30s (test-overridable period) until
its context is cancelled; wired at sbxmitm startup as a
process-lifetime goroutine when --media-buffer is enabled.
NewMediaBuffer's signature is unchanged; retentionSecs (1200),
sizeCeilBytes (24 GiB) and nowFn now default inside the constructor.
* feat(core): media-buffer metatag reader (dedup by id, fail-empty) (ref #812)
* test(core): media-buffer reader — tail-read truncation + malformed-field regressions (ref #812)
* chore(#812): keep SDD scratch reports out of the feature diff (restore to master)
* feat(dpi): media-buffer list/replay/thumb API (admin/owner, audited, 410-on-evict, traversal-safe) (ref #812)
* harden(dpi): media API — \Z-anchored id regex + role short-circuit security note (ref #812)
* feat(sbxmitm): media-buffer retention/size-ceil flags + /data tmpfiles + opt-in worker env toggle (ref #812)
- main.go: --media-buffer-retention / --media-buffer-size-ceil flags, applied
onto the already-constructed mbuf's unexported retentionSecs/sizeCeilBytes
fields (NewMediaBuffer signature unchanged).
- tmpfiles/zz-secubox-media-buffer.conf: pre-creates /data/secubox (0755) and
/data/secubox/media-buffer (0750 secubox:secubox), installed via the
package's existing manual tmpfiles.d mechanism in debian/rules.
- secubox-toolbox-ng-worker@.service: EnvironmentFile=-/etc/secubox/toolbox-ng.env
+ trailing $SBX_MEDIA_BUFFER_ARGS on ExecStart so an operator can opt in
(SBX_MEDIA_BUFFER_ARGS=--media-buffer) without editing the shipped unit;
default stays OFF. Added ReadWritePaths=/data/secubox/media-buffer.
* fix(sbxmitm): media-buffer dir owned by writer (secubox-toolbox) + setgid group secubox; changelog 0.1.27 (ref #812)
Task 6 review Critical: buffer dir was 0750 secubox:secubox but sbxmitm workers run
as secubox-toolbox (not in group secubox) → EACCES on every capture, silently no-op'd
(errors swallowed by design). Fix: 2750 secubox-toolbox:secubox — writer owns, dpi
reader (secubox) is group, setgid propagates group secubox to created objects so the
reader can read them. Important: add debian/changelog 0.1.27 stanza (version bump).
* feat(dpi): Media tab — live capture gallery with replay/download, escaped fields (ref #812)
Adds a Media Buffer card to the DPI dashboard (packages/secubox-dpi/www/dpi/index.html)
that lists sbxmitm's live media-buffer captures via GET /api/v1/dpi/media/buffer.
- Auth: dedicated apiAuthed() reads the canonical `sbx_token` localStorage key
and sends it as a Bearer token (media/buffer requires JWT, unlike the rest
of this page's endpoints which currently ride the SSO-lite session cookie);
401 redirects to /login.html, matching sibling dashboards (e.g. crowdsec).
- Security: host/url/mac_hash are attacker-influenceable (captured live
traffic). Every card is built via createElement + textContent (or the
`.title` IDL property, never HTML-parsed) — no innerHTML string
concatenation of these fields — so a captured value like
`<img onerror=...>` renders as inert text. Replay/download URLs are built
with encodeURIComponent(id) even though the id is hex-validated server-side.
- UI: kind emoji (video/audio/manifest/file), short device id (shortDev,
reused from the existing #785 helpers), ⬆/⬇ direction, human size
(formatBytes) and age (agoStr) per card; expired captures grey out and show
"métatag seul (expiré)" with no action links.
- Refresh: wired into the existing refreshAll() plus its own ~15s interval.
Verified with a Playwright harness stubbing fetch (incl. an XSS payload in
host/url) against a local copy of the page: no console/page errors, XSS did
not fire, no <img>/<script> element materialized from the captured strings,
expired-card gating confirmed.
* fix(sbxmitm): media buffer — upload tee needs a real body, download tee 200-only (whole-branch review); drop dead item.ts (ref #812)
* docs(media-buffer): Phase 2 plan — HLS reassembly via read-time URL join (ref #812)
---------
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
* docs(spec): rich PDF character sheet + persona/dpi/media route parity (ref #790)
* docs(plan): rich PDF character sheet + route parity implementation plan (ref #790)
* feat(toolbox): rich PDF character sheet — pips, on/off inventory, quests (ref #790)
* feat(toolbox): factor _enrich_report_data + apply to token/admin PDF routes (ref #790)
* feat(toolbox): populate Quêtes/menaces from raw DPI alerts (dest/detail) — HTML + PDF (closes#792, ref #790)
_dpi_stats now exposes alerts_raw (kind/service/dst/detail) alongside the donut
'alerts' ({label,count}). The persona Quêtes section (PDF _persona_block + HTML
.nr card) reads alerts_raw so each threat shows its destination + detail instead
of a dangling em-dash; donut 'alerts' kept for the DPI-Exfil tab; graceful
fallback to the donut label (no dangling dash) when alerts_raw is absent.
---------
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
* docs(spec): rapport kbin fidèle + media types + WebUI DPI (ref #785)
* docs(plan): rapport kbin media types implementation plan (ref #785)
* feat(core): shared media-catch JSONL aggregator (ref #785)
* fix(core): guard media-catch aggregate against malformed fields + full SPDX header (ref #785)
* feat(toolbox): _media_stats + wire media_exfil into report routes (ref #785)
* feat(toolbox): PDF DPI-exfil/overall donut-grids + media-type block (ref #785)
Replaces the text-bullet DPI/EXFIL section (per-device + overall) with
_pdf_donut_grid 4-donut grids and adds a new "TYPES DE MEDIAS CAPTES"
section (kinds/content-types donut grid + top-hosts emoji table), for
parity with the HTML report. Also guards the PDF footer's italic
set_font against DejaVu-Oblique not being registered (fonts-dejavu-core
ships Regular+Bold only) — a pre-existing latent crash on any full
render_pdf() call, uncovered because no prior test exercised the whole
function end-to-end.
* fix(toolbox): per-grid captions so network/media donuts aren't mislabeled as device stats (ref #785)
* feat(toolbox): media-type cards in report web page (me + overall) (ref #785)
* fix(toolbox): close media card divs so Overall tab + footer aren't nested in DPI pane (ref #785)
* feat(dpi): WebUI card — services by category (bytes) (ref #785)
* feat(dpi): /media_types endpoint (MIME media-catch, fail-empty) (ref #785)
* feat(dpi): WebUI card — MIME media types + refreshAll wiring (ref #785)
* chore: gitignore SDD run scratch + drop stray task report (ref #785)
* perf(core): bounded tail-read for media-catch + import-in-try for uniform fail-empty (ref #785)
* fix(dpi): backport board-live async control endpoints (drift reconciliation, ref #785)
The board ran a manually-deployed main.py with status/restart/start/stop/logs/
interface_list/tc_status/remove_mirred as 'async def' — never committed to git.
Backported so the next .deb install doesn't revert them. Signature-only change,
verified identical to the live-validated deployed file.
* fix(toolbox): PDF render must not wedge the event loop (#785 incident)
Heavier #785 report PDFs (more matplotlib donut-grids, ~9s render) + the WAF
504-page auto-retry storm pegged the single uvicorn worker and 504'd the whole
toolbox vhost. Fixes:
- run render off the event loop (threadpool) so HTML report / landing stay live
- serialize renders through one asyncio lock (pyplot is not thread-safe; also
bounds CPU so retries can't render in parallel)
- short per-device PDF cache with a double-checked lock: a retry storm now
triggers exactly ONE render, all other hits return cached bytes instantly
- persistent MPLCONFIGDIR (systemd drop-in + postinst dir) so matplotlib builds
its font cache once instead of on every process start
Validated live on gk2: cold render caches, 8-request retry storm all served
from cache (~0.1s), /landing 0.012s throughout.
---------
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
* feat(p2p): DHT, Federation, MasterLink evolutions for P2P-EVO-2026-07-001
Implements three major features for secubox-p2p:
1. DHT (Distributed Hash Table) based on Kademlia algorithm
- Peer discovery across different subnets
- Resilience to temporary node failures
- Scalability for large number of nodes
- REST API endpoints for DHT management
2. Service Federation
- Service registration and discovery
- Multi-version service management
- Automatic health checks
- DHT integration for service propagation
3. Master-Link Hierarchical Topology
- Master/satellite/leaf hierarchical structure
- Automatic master election and failover
- Heartbeat monitoring
- Routing policy management
- OPAD security integration
New files:
- api/dht.py: Kademlia DHT implementation
- api/federation.py: Service federation system
- api/masterlink.py: Hierarchical topology manager
- api/main_evolutions.py: Integrated FastAPI server
- tests/test_dht.py: Unit tests for DHT
Issue: P2P-EVO-2026-07-001
Worktree: secubox-p2p-evolutions
Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
* revert(p2p): remove non-integrating DHT/federation/masterlink code, keep requirements
The DHT/federation/masterlink modules + main_evolutions.py + test_dht.py did not
integrate with the module: tests imported 'secubox.p2p.api.*' (the package uses
'from api import ...' via conftest sys.path) so nothing even collected, and the
modules reinvented standalone aiohttp servers instead of building on the existing
WireGuard mesh (mesh.py) + annuaire-backed registry (registry.py/annuaire_client.py).
Removing them to restart on the real base. The requirements doc
(.github/ISSUES/2026-07-P2P-EVOLUTIONS.md) is kept as the spec source.
* docs(p2p): point the local evolutions note to real issue #774
The '.github/ISSUES/2026-07-P2P-EVOLUTIONS.md' was a local file with a fabricated
id, not a tracked issue. Replace its contents with a pointer to the real GitHub
issue #774 (Kademlia DHT + hierarchical master-link + federation health-checks),
framed against the already-done federation (#766) and directory (#768) work and
the master-link/auto-enrollment issue (#762).
* docs(spec): p2p Kademlia DHT + master-link + federation health-checks (#774)
Clean redesign on the real base (mesh.py + registry/annuaire_client) after
reverting the non-integrating attempt. Custom minimal Kademlia (pure asyncio
UDP, JSON wire, signed reachability records), federation health-checks layered
over the annuaire federation (#766) publishing status via the DHT, and a
deterministic-election master-link (term-based failover, aligns #762). All
feature-flagged off by default (OPAD opt-in), from api import convention, TDD
per module. Refs #774.
* docs(plan): p2p DHT/federation/master-link TDD implementation plan (#774)
17 bite-sized TDD tasks across 3 phases (DHT 1-9, federation health-checks
10-13, master-link 14-16, finalization 17). Each task = failing test -> minimal
impl -> pass -> commit, on the real base (from api import ...). Crypto (Ed25519
sign/verify) and UDP transport are module-level seams injected in unit tests so
the Kademlia primitives, health debounce, and election/term logic are tested
without real sockets. Refs #774.
* feat(p2p): DHT scaffold — node id + xor distance (#774)
* feat(p2p): DHT k-bucket with LRU (#774)
Add DHTNode contact + DHTBucket (k-bucket with LRU ordering).
- DHTNode: dataclass with node_id, did, endpoint, last_seen
- DHTBucket: Kademlia k-bucket with OrderedDict LRU management
- add(node) → bool (True if stored/refreshed, False if full)
- refresh moves node to tail (most-recent)
- oldest() returns head node
- remove(node_id) and nodes property for list access
All Task 1 tests + new Task 2 tests passing.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(p2p): DHT routing table + closest-N (#774)
* feat(p2p): DHT signed reachability records + verify (#774)
Implement canonical_record, sign_record, verify_record functions with
crypto SEAMS (_did_from_pubkey, _verify_sig, _sign_sig) for testing.
Tests use monkeypatching; real crypto wired in Task 8.
Canonical records are deterministic sorted JSON. verify_record checks
for sig field, DID validity via wg_pubkey, and cryptographic signature.
* feat(p2p): DHT JSON UDP codec + hardening (#774)
* feat(p2p): DHTNetwork store+dispatch (transport-injected) (#774)
* feat(p2p): DHT iterative find_node/find_value + announce (#774)
Add asyncio-based iterative Kademlia lookup on top of the injected
DHTNetwork transport from Task 6:
- pending: dict[rpc_id, Future] on DHTNetwork, resolved by handle_message
when a reply (pong/nodes/value/ok) arrives with a matching rpc_id.
- _rpc(node, msg): send + await the matching future with RPC_TIMEOUT,
returns None on timeout.
- iterative_find(target_id, mode): alpha-parallel iterative lookup,
dedups queried nodes, merges discovered contacts into the shortlist
(dedup by node_id, skip self), terminates on no-improvement-in-a-round
or exhausted candidates; mode="value" short-circuits on first verified
record.
- find_peer(did) / announce(): DID resolution and self-record
publication (sign, cache locally, push to the k closest nodes found).
Tests (tests/test_dht.py, +2 async, pytest-asyncio strict mode):
- in-process UDP router delivering via call_soon (never inline) so
awaited RPC futures resolve on a later loop turn;
- find_peer resolves a record announced by a node reachable only
through a bootstrap intermediary;
- dedup: a node introduced twice into the shortlist (via two different
repliers) is queried exactly once.
All 16 tests green (14 prior + 2 new).
* fix(p2p): harden DHT iterative lookup — skip malformed contacts, cap shortlist, tolerate rpc exceptions (#774)
* feat(p2p): DHT record schema (id+wg keys) + real Ed25519 sign/verify (#774)
* feat(p2p): DHT UDP transport + routing persistence + bootstrap (#774)
* feat(p2p): mount DHT — endpoints + startup guard + [dht] config (#774)
Task 9: wire the finished Kademlia DHT (api/dht.py) into the FastAPI
app as a feature-flagged background service, fully backward compatible.
- api/mesh.py: load_p2p_config() now also returns a `dht` sub-dict
(enabled/port/bootstrap/announce/announce_interval/rps) built from an
optional [dht] toml section, with defaults when absent.
- api/main.py: @app.on_event("startup") _dht_startup creates+binds a
DHTNetwork only when [dht].enabled is true; any failure (missing
identity, bind error, ...) is caught and leaves app.state.dht = None
so the app always starts. _dht_shutdown persists the routing table
and closes the UDP transport.
- New endpoints (bare paths, nginx adds /api/v1/p2p prefix): GET
/dht/peers (routing snapshot), POST /dht/announce (JWT-protected,
audit-logged to /var/log/secubox/p2p-audit.log), GET /dht/find/{did}.
Tests: tests/test_dht_wiring.py (config defaults/overrides + disabled
TestClient smoke). All 22 existing DHT tests + 3 new tests green (74
total in the package).
* feat(p2p): federation HealthStore + debounce (#774)
* feat(p2p): federation HealthChecker sweep (#774)
* feat(p2p): federation real probe + registry services (#774)
* fix(p2p): bound TCP-probe close, declare aiohttp dep, guard non-dict service (#774)
* feat(p2p): federation health publish-via-DHT + endpoints + [federation] config (#774)
* fix(p2p): wire federation probe_timeout + harden sweep/health-TTL tests (#774)
* feat(p2p): master-link elect() + term store (#774)
* feat(p2p): master-link state machine + failover (#774)
* fix(p2p): master-link equal-term tie-break (no zero-master window) + defensive peers (#774)
* feat(p2p): master-link signed heartbeats + UDP transport + tick loop (#774)
* feat(p2p): mount master-link — endpoints + startup guard + [masterlink] config (#774)
- masterlink.MasterLink.request_promotion(): on-demand promotion attempt
(bump term, run election, become MASTER + heartbeat on win, else CANDIDATE)
- mesh.load_p2p_config(): add [masterlink] section defaults/overrides
(enabled, role_preference, priority, heartbeat_interval, election_timeout,
port, peer_addrs) alongside existing [dht]/[federation]
- main.py: _masterlink_startup/_masterlink_shutdown mirror the DHT/federation
defensive startup pattern (feature-flagged, never breaks app startup);
GET /masterlink/topology (public) + POST /masterlink/promote (JWT, audited
to p2p-audit.log)
* fix(p2p): master-link request_promotion records winner on loss + strengthen test (#774)
* docs(p2p): document DHT/federation/master-link evolutions (#774)
Add an "Evolutions (#774)" section to packages/secubox-p2p/README.md
covering the DHT (api/dht.py), federation health-checks (api/federation.py)
and hierarchical master-link (api/masterlink.py) subsystems: what each does,
their API endpoints, the real p2p.toml config defaults from api/mesh.py,
and the audit log. All three remain OFF by default / OPAD opt-in. Docs only,
no behavior change. Append a matching dated entry to .claude/HISTORY.md.
* fix(p2p): final-review fixes — audit-log path, master-link peers_fn, wg_pubkey, handle_message hardening (#774)
* fix(p2p): implement mesh visualization tab — load on activation + master-link roles (#774)
The Mesh tab canvas was never rendered (the tab-switch handler didn't call
initMesh, and drawing into a hidden tab gave clientWidth 0 -> blank canvas).
Now: initMesh fires when the Mesh tab is activated, retries if the canvas has
no layout yet, and enriches the star graph with the live master-link state
(role/term/master + DHT peer count from /masterlink/topology + /dht/peers) —
center node coloured gold=MASTER / cyan=SATELLITE, peers by online status.
* docs(p2p): poster GPT + roadmap des évolutions DHT/federation/master-link; maj WIP/HISTORY/TODO (ref #774)
---------
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
Co-authored-by: Mistral Vibe <vibe@mistral.ai>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
First slice of gondwana P1: make the annuaire the distributed directory.
Adds Op.NODE_PUBLISH/CONFIG_PUBLISH/CONFIG_REVOKE and the NodeRecord (signed
mesh peer registry, public wg key only) + ConfigBlob (signed, versioned,
single-writer config distribution) models — modelled on ServiceOffer.
Carries the consolidated clone/distribute/emancipate roadmap doc.
TDD: tests/test_directory.py (9 tests); full annuaire suite 136 green.
