Le module secubox-cve-triage existe déjà et fait plus que prévu (dpkg, NVD,
CVSS, EPSS réel, flux KEV CISA, triage, panel). Il lui manque une gâchette
(aucun .timer), une voix (zéro notify) et un filtre anti-bruit. Ce spec livre
l'appelant, pas un module.
Filtre: KEV ∧ installé ∧ (exposé public ∨ rançongiciel) — un ET, pas un OU.
L'exposition vient de l'inventaire secubox-profiles livré ce jour (16 modules
publics sur 187). Anti-bruit: alerte sur le delta, ré-alerte sur escalade,
acquittement, digest, plancher anti-tempête.
CrowdSec CTI écarté: clé obligatoire, 40 requêtes/mois en gratuit, dépendance
cloud pour une donnée que le KEV CISA fournit librement (1647 CVE, sans clé).
XMPP chiffré honnêtement: le module jabber ne sait pas envoyer: c'est un
gestionnaire Prosody. Le sink exige slixmpp (présent dans bookworm), un compte
bot, un secret et un client — une tâche isolée, pas un branchement.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
6 tâches TDD : manifestes, résolution d'état, sondes, diff ordonné, profiler
scan, CLI + paquet. 55 tests.
Phase 1 ne peut rien allumer ni éteindre — un test verrouille l'absence de la
commande apply, et observe.py n'exécute que des commandes de lecture.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Bascule de contexte au quotidien (média / sécurité / démo) : profils exhaustifs
+ pins persistants, toggle individuel, taxonomie des modules, dépendances
minimisées, priorités présentées.
Architecture retenue : manifestes plats + moteur de réconciliation. Les .target
systemd sont écartées : LXC, routes WAF et menus ne sont pas des units, et une
target n'exprime pas 'éteint'.
Le design est contraint par le réel mesuré sur gk2 : 187 units (118 actives),
load 5.4 sur 4 cœurs, ~2 Go libres, 24 LXC, 134 menu.d, et 80 units en
Requires=secubox-core (un oneshot mkdir) — d'où application séquentielle,
extinction avant allumage, et conversion Requires -> Wants en prérequis.
Noyau protégé non négociable : un profil ne doit pas pouvoir éteindre ce qui
permet de le rallumer.
Métriques mesh explicitement hors périmètre (spec séparé).
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
* docs(device-guardian): Project A consolidation design — merge 5 device modules into nac (ref #817)
* docs(device-guardian): Project A implementation plan — 9 tasks (ref #817)
* feat(nac): SQLite device store + idempotent legacy migration (ref #817)
Add packages/secubox-nac/api/store.py — pure-stdlib sqlite3 module with
canon_mac(), DeviceStore (WAL, best-value upsert, list/get/count,
history), and migrate_legacy() for idempotent, per-source best-effort
import of mac-guard/device-intel/iot-guard legacy shapes. Reserved
Project-B columns (plane/provenance/geo_cc/geo_asn) created but unused.
Task 1 of the Device Guardian consolidation plan (docs/superpowers/plans/
2026-07-05-device-guardian-consolidation.md).
* fix(nac): store — preserve first_seen, skip corrupt legacy sources, no stray db, SPDX (ref #817)
* feat(nac): unified dnsmasq+ISC+ARP discovery merge (ref #817)
* fix(nac): discovery — latest ISC lease wins the IP on renewal (ref #817)
The merge loop only updated ip/source on a strict rank increase
(incoming_rank > existing_rank), so equal-rank sightings for the same
MAC never touched ip/source. Since dnsmasq -> isc -> arp is processed
in non-increasing rank order, multiple ISC dhcpd.leases blocks for one
MAC (a fresh block per renewal) are equal-rank ties, and the first
(oldest) block's ip was frozen instead of the latest one winning, as
the original secubox-device-intel._get_dhcp_leases did.
Change the condition to incoming_rank >= existing_rank so the latest
same-or-higher-rank sighting updates source, and overwrites ip when
the incoming ip is non-empty. Hostname rule and the lower-rank branch
are unchanged, so a later ARP sighting still cannot clobber a
higher-rank ip/hostname/source.
Add test_isc_latest_lease_wins: two ISC lease blocks for one MAC with
ip/hostname renewal, asserting the latest block wins. Confirmed this
test fails against the pre-fix >-only logic.
* feat(nac): absorbed enrichers — OUI vendor + device-type/risk + OpenWrt fingerprint (ref #817)
* fix(nac): enrich — is_router strictly vendor-based; cover classify/oui edge cases (ref #817)
* feat(nac): background collector + double-cache; handlers def, no inline scan (ref #817)
Task 4 of the Device Guardian consolidation: a Collector background
loop now owns all discover -> enrich -> upsert work (replacing
_monitor_clients), publishing an in-memory snapshot(). The status,
clients, and client/{mac} handlers become plain def reading the SQLite
store / collector cache instead of calling _discover_clients() inline
on every request -- the #808 aggregator SPOF.
* fix(nac): finish #808 — def-convert alerts/list_quarantine/summary; first_seen + recent_events (ref #817)
alerts/list_quarantine were still `async def` calling blocking
_discover_clients() inline on the shared aggregator loop; summary
awaited alerts on top of that (double loop-block). Converted all three
to plain `def` reading the SQLite store/collector snapshot instead,
completing the #808 SPOF fix started for status/clients/client.
Also: collector.cycle_once() now stamps first_seen on newly discovered
devices (store.upsert keeps the earliest via COALESCE), and get_client's
recent_events merges the SQLite device_history (client_joined) with the
JSON event log so a device's join event shows up again.
* feat(nac): allow/deny actions + def-convert action handlers (finish #808) (ref #817)
Task 5: fold mac-guard's allow/deny into nac's zones/nft. New
POST /allow/{mac} and /deny/{mac} set allow_state in the SQLite store
and mirror it into nft's existing inet secubox_nac blocked/lan_allowed
sets via _nft_add_element/_nft_delete_element (mac-guard's separate
inet secubox_mac_guard table stays retired; its element migration is
Task 9).
Carried from Task 4 review: zones, add_to_zone, remove_from_zone,
approve_client, ban_client, update_client, quarantine_client and
unquarantine were still async def wrapping blocking subprocess/nft
calls on the shared aggregator loop (#808). Converted to plain def so
FastAPI threadpools them off the loop; add_to_zone/ban_client's webhook
notify now fires via a new _fire_webhook_sync helper
(asyncio.run_coroutine_threadsafe against the loop captured at
startup) since a plain def can no longer await _notify_webhooks
directly.
Tests: tests/test_actions.py mocks nft via monkeypatched
_nft_add_element/_nft_delete_element/_nft_list_set (+ subprocess.run,
since ban_client/unban_client build raw nft argv) into an in-memory
set-membership dict; covers deny/allow/allow-then-deny, zones,
add_to_zone+approve_client, and ban_client+unban_client.
* feat(nac): absorbed endpoints — vendors/scan/probe/mdns/groups/export (ref #817)
* fix(nac): /scan enriches + validates subnet; /probe IP + CSV injection guards (ref #817)
* feat(nac): retire mac-guard/device-intel/iot-guard via 308 redirects; auth-gate network-anomaly (ref #817)
secubox-mac-guard, secubox-device-intel and secubox-iot-guard are
superseded by secubox-nac's consolidated Device Guardian. Their
api/main.py are reduced to thin shims: every route 308-redirects to
its secubox-nac equivalent (/devices->/clients, /device/{mac}->
/client/{mac}, /whitelist|/allow->/allow, /blacklist|/deny->/deny,
/scan, /vendors, /groups, /export/* pass through unchanged, and a
catch-all forwards anything else), with an X-SecuBox-Deprecated
header. /health stays a plain 200 so liveness probes never 308-loop.
Package removal is a separate, gated follow-up.
secubox-network-anomaly stays (separate traffic-anomaly engine) but
its /status endpoint was unauthenticated while every sibling route
already required a JWT; it now matches.
* feat(nac): webui — vendor/type/risk columns, groups view, export; drop retired webuis (ref #817)
- clients.js: render enriched /clients fields — oui_vendor, device_type,
risk_level (color-coded badge)/risk_score, and OpenWrt/SecuBox/Router
fingerprint badges. All values are attacker-influenceable (DHCP
hostname/vendor) and are rendered exclusively via E()/textContent,
never innerHTML. Null/missing device_type/vendor/risk render as
"unknown"/"—", never "null". Adds an Export CSV button (direct link to
GET /export/csv — auth via the existing SSO session cookie, same as
every other same-origin nav link in this app; no new auth scheme).
- groups.js (new): list/create/delete device groups and assign a device
to a group, hitting GET/POST/DELETE /groups* — mirrors zones.js's
structure and auth (sbxFetch).
- nav.js: register the new "Groupes" tab alongside Zones.
- Remove packages/secubox-{mac-guard,device-intel,iot-guard}/www — their
functionality is now consolidated into nac; secubox-network-anomaly/www
is untouched (stays standalone per plan).
Note for Task 9: device-intel/iot-guard's debian/rules unconditionally
`cp -r www/<name>/.` — now broken since www/ is gone; mac-guard's rule is
already guarded (`[ -d www ] && ... || true`). Needs a debian/rules fix
when Task 9 retires/redirects these packages.
* chore(nac): packaging — ieee-data dep, nft migration, retirement redirects, changelog 3.0.0; fix retired debian/rules www (ref #817)
* fix(nac): whole-branch — lazy init for in-aggregator mount, collector off-loop, sqlite lock, batched zone lookup, sentinel-preserve (ref #817)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(nac): risk_score split — known router low, unknown router high (rogue-AP signal) (ref #817)
Split the router risk axis out of the generic device-type/open-ports
scoring: a router/gateway already known (allow-listed or already in the
store) is trusted infrastructure -> low risk; a newly-seen/unknown
router is a rogue-AP signal -> high risk, regardless of open ports.
Removes the old flat -10 "trusted router" discount. Non-router devices
keep the existing scoring unchanged.
risk_score() gains an is_known=False parameter (backward compatible).
collector.cycle_once() and main.scan() now compute is_known from the
prior store row (existing row or allow_state == "allow") before
upserting, reusing the row they already fetch for is_new/enrichment.
* feat(nac): mesh-sync (p2p peers as devices) + fingerprinted-router classification for rogue-AP risk (ref #817)
Part A: absorb iot-guard's missed "mesh-sync" capability. GET /mesh/peers
and POST /mesh/sync now live in nac, reading live P2P peers over
/run/secubox/p2p.sock (HTTP-over-UDS GET /peers, fail-safe -> [] on any
error/timeout/absent socket) and upserting one sb:-prefixed synthetic
device per peer (iot-guard's exact md5-hash MAC scheme), device_type
"mesh_node", source "mesh". iot-guard's retired redirect-shim catch-all
already forwards /mesh/* here, so no shim change needed.
Part B: classify_device_type's "router" keyword list is much narrower
than ROUTER_VENDORS (misses tp-link/asus/linksys/d-link/gl.inet/xiaomi/
huawei), so a router-vendor MAC with a generic hostname fingerprinted
is_router=True still classified as device_type="unknown" and skipped
risk scoring entirely (the omit-unknown guard) -- the known-router-vs-
rogue-AP HIGH signal never fired for the most common vendor case. Both
enrichment call sites (Collector.cycle_once and /scan) now reclassify
device_type="unknown"+is_router to "router" before scoring.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(nac): postinst ensures secubox_nac table + zone sets (incl lxc_zone)
Nothing shipped created nac's inet secubox_nac table or its zone sets — they
were assumed to pre-exist (they don't: the live board had no such table), so
operator 'move to zone' failed for every zone and the mac-guard element
migration silently no-op'd. postinst now creates the table + all ZONES sets
(lan_allowed, lxc_zone, iot_zone, guest_zone, quarantine_zone) + blocked,
idempotently and best-effort, before the migration. Makes operator moves to
the LXC Containers zone work on a fresh install. nft commands validated on
gk2 (create/idempotent re-add/add-element/list), no live drift left.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
---------
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* docs(sentinel): design — inline IOC gate + async Go analyzer + spyware packs (ref #823)
* docs(sentinel): implementation plan — 12 tasks (ref #823)
* feat(sentinel): IOC model + IOCSet matchers (ref #823)
* feat(sentinel): pack loader + live-overlay merge + hot-reload (ref #823)
Add internal/sentinel/pack.go: Pack (JSON base/overlay content bundle),
LoadBasePack, MergePacks (base + N overlays, override by Type+Value via
IOCSet's per-type maps), and Loader (NewLoader/Set/MaybeReload/YaraRules).
Loader mirrors cmd/sbxmitm/policy.go's LoadPolicy/maybeReload mtime-based
hot-reload pattern, adapted for a directory of a DYNAMIC *.json file set
(glob-based) rather than policy.go's fixed Target list. Fail-safe: a
corrupt overlay pack is skipped+logged, never wipes the base or panics;
MaybeReload is a cheap stat-only no-op when nothing on disk changed;
Set()/YaraRules() read via atomic.Pointer so a reload swap never torn-reads
a concurrent hot-path lookup.
Adds JSON tags to Task 1's IOC struct (type/value/class/severity/source/
action) so packs round-trip through encoding/json.
* fix(sentinel): pack loader — fail-closed on base-wipe + throttle MaybeReload + url_regex override + yara dedup (ref #823)
Review found 1 Critical + 2 Important + 1 Minor in Task 2 (pack loader):
- Critical: a vanished/degraded base pack directory (deleted dir, mount
blip, mid-dpkg-upgrade window) could silently empty the live detection
set on the next MaybeReload. loaderState now tracks baseCount; a reload
that would drop base IOCs from >0 to 0 is refused (prior state kept,
warning logged) instead of swapped in.
- Important: MaybeReload() is called per-flow by Task 3's inline gate: add
a 5s default throttle (mirrors cmd/sbxmitm/policy.go's reloadThrottle)
so a hot-path call is a pure no-op until the window elapses.
- Important: url_regex IOCs previously only appended, so a live overlay
could never override a base pack's URL rule (base always matched
first). Add urlIndex (pattern -> slice index) so re-adding the same
pattern replaces the entry in place, keeping deterministic match order.
- Minor: YaraRules() now dedups identical rule bodies across base+overlay
packs instead of emitting duplicates.
go test ./internal/sentinel/: 21/21 PASS. go vet/gofmt/go build clean.
* feat(sentinel): inline gate matcher (fail-open, highest-severity) (ref #823)
* feat(sentinel): bbolt verdict store (pure-Go, WAL-free, TTL prune) (ref #823)
* feat(sentinel): action scorer — heuristic/low-confidence forced report-only (ref #823)
* feat(sentinel): bounded fire-and-forget mirror channel (drop-with-count) (ref #823)
* fix(sentinel): mirror Emit always copies body — no aliasing of caller's hot-path buffer (ref #823)
* feat(sbxmitm): inline Sentinel gate — neutralize high-confidence, mirror rest, fail-open (ref #823)
Wire internal/sentinel's inline IOC gate into the shared mitmPipeline response
path (covers both the transparent R3 and CONNECT accept paths without drift):
build a FlowMeta from Host/URL/ClientIP/MacHash after up.Do, call the gate, and
on a HIGH-CONFIDENCE known-infra hit neutralize inline — block/sinkhole serve a
Sentinel block page instead of the upstream response, strip serves the response
with an emptied body. Everything else (miss, heuristic, low-confidence, report)
is mirrored bounded fire-and-forget to the async analyzer and proceeds unchanged.
Fail-open everywhere: SENTINEL_ENABLED unset/false, a pack-load failure, a nil
hook, or a gate panic all degrade to a transparent passthrough — a Sentinel
setup problem can never break browsing. With the engine off, behavior is
byte-identical to today. Env: SENTINEL_ENABLED / SENTINEL_PACK_DIR /
SENTINEL_OVERLAY_DIR / SENTINEL_MIRROR_SOCK.
Hot path (BenchmarkSentinelInspectMiss, -benchmem): 101.0 ns/op, 0 B/op,
0 allocs/op — one throttled gate Match + a non-blocking mirror Emit; no YARA or
heavy analysis inline.
go test ./cmd/sbxmitm/... -race → ok (all pre-existing + 7 new green).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(sentinel): sbx-sentinel async daemon scaffold + store wiring (ref #823)
* feat(sentinel): YARA engine (cgo, build-tagged) + no-cgo stub (ref #823)
Add internal/sentinel/yara.go (cgo && yara) wrapping
github.com/hillu/go-yara/v4 for compiled-rule body scanning, and
yara_stub.go (!yara) as the no-op default: NewYaraEngine always
succeeds, Scan/Analyze always return nil. Both expose identical
signatures (YaraEngine, NewYaraEngine, Scan, Analyze, Close) so
sbx-sentinel/sbxmitm can wire a YaraEngine unconditionally.
yara_test.go (untagged) proves the default build works with YARA
disabled. yara_real_test.go (yara tag) compiles a trivial EICAR rule
and exercises the real libyara-backed engine — split into its own
file since Go build constraints are file-scoped and can't mix a
tagged and untagged test in one file.
go-yara is now an indirect go.mod requirement and vendored, but is
only reachable via the yara build tag: default `go build ./...` and
`CGO_ENABLED=0 go build ./...` both stay clean, no libyara/cgo
required. `-tags yara` needs libyara-dev + a C toolchain (documented
in yara.go's header).
* fix(sentinel): YARA verdict confidence 90 (>= threshold) so a confirmed malware match actually strips, not report-only (ref #823)
* feat(sentinel): behavioral engine — beaconing + zero-click heuristics (report-only) (ref #823)
* fix(sentinel): tighten one-time-link heuristic — exclude file-extension + base64 shapes (cut zero-click FPs) (ref #823)
* feat(sentinel): spyware indicator engine + commercial-spyware base pack (MVT/CitizenLab) (ref #823)
* fix(sentinel): base-pack declared-block IOCs severity>=85 so they actually block + general guard test (ref #823)
Three shipped base-pack IOCs declared action:"block" but severity:80,
below scorer.go's HighConfidenceThreshold=85 — FinalizeAction silently
downgrades any Confidence below that threshold to report-only, so these
entries never actually blocked despite being declared as known-bad:
- packs/base/malware.json: ip 198.51.100.23 (class malware)
- packs/base/malware.json: cert_sha1 ...01234567 (class trojan)
- packs/base/botnet.json: ja3 deadbeef... (class botnet_c2)
Bumped all three to severity 90 so they survive FinalizeAction and
actually block, matching every other declared-block entry in the base
packs.
Added TestBasePackDeclaredBlockIsHighSeverity, a general guard over
every base pack file/entry: any IOC with an auto-neutralize action
(block/strip/sinkhole) must carry Severity >= HighConfidenceThreshold.
This generalizes the narrower TestBasePackKnownInfraIsHighSeverity
(spyware classes only) so a future pack entry in any class can't ship
a "block" that silently never blocks. Confirmed the new test fails red
against the pre-fix JSON (all 3 offenders reported) and passes green
after the severity bump.
* feat(sentinel): proposal/solution reporter — RenderReport(Verdict) (ref #823)
* feat(sentinel): wire real analyzers into sbx-sentinel + read-only status HTTP surface (ref #823)
* feat(sentinel): live threat-feed → overlay pack fetcher (fail-safe, atomic) (ref #823)
* feat(sentinel): live feed overlay + reporter + daemon analyzer wiring + packaging (ref #823)
* fix(sentinel): feeds fetcher — add curl --fail so HTTP 4xx/5xx is treated as a fetch failure (ref #823)
* fix(sbxmitm): drop mis-wired ClientIP from inline Sentinel FlowMeta (source IP must not match destination-C2 IP IOCs) (ref #823)
---------
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* docs(media-buffer): design spec — dashcam capture + replay + metatag lifecycle (ref #812)
* docs(media-buffer): phase-1 implementation plan (capture+janitor+API+DPI tab) (ref #812)
* feat(sbxmitm): media buffer store + metatag writer (ref #812)
* fix(sbxmitm): media-buffer review — 4096 atomic-append cap + *string buffer_ref (ref #812)
Task 1 review (Important): mediaBufferLineMax 8192→4096 to match mediacatch.go's
PIPE_BUF atomic-append guarantee across 4 workers; BufferRef string→*string so the
janitor (Task 3) can flip it to JSON null on eviction.
* feat(sbxmitm): tee media up/download bodies into the buffer (ref #812)
Wire the R4 media buffer tees into mitmPipeline: the download hook wraps
resp.Body and the upload path wraps req.Body via teeReadCloser, capturing
whole-file media into the rolling /data buffer. Add the mbuf field + flags
(--media-buffer default OFF, --media-buffer-root, --media-buffer-per-object).
Design upgrade over the plan sketch: ObjectWriter.Write is now NON-BLOCKING.
Instead of io.TeeReader driving a synchronous os.File.Write on the client read
path (which would add latency / could stall large media on ARM eMMC), Write
copies each chunk into a bounded channel and returns len(p),nil immediately; a
background goroutine started at Capture drains the channel to disk. A full
channel drops the chunk and flags truncated — never blocks, never errors the
proxied flow (design 4.2/7). Close signals the drainer, waits for it to flush
queued chunks + close the file, then appends the metatag; it is idempotent,
deadlock-free (sole closer of ch) and safe if Write is never called or races
Close.
* fix(sbxmitm): media download tee — defer closure so w.Close runs (ref #812)
defer resp.Body.Close() bound the ORIGINAL upstream body at the defer
statement, before the media-buffer tee reassigned resp.Body — so the
tee's Close() (and w.Close, which flushes the sink/metatag/drain
goroutine) never ran on any download. Wrap the close in a closure so
it reads resp.Body at return time instead.
Adds TestTeeDeferOrderingMatchesMainGo, which replicates main.go's
exact defer-then-reassign ordering and fails against the old plain
defer form (verified manually, then restored the fix).
Also: drop the dead write-only ObjectWriter.dropped field (truncated
already covers it), document Capture's reserved contentLen param, and
note that ObjectWriter.Close is allowed to block (post-stream
finalisation), unlike Write.
* feat(sbxmitm): media buffer janitor — time + LRU eviction, metatag survives (ref #812)
SweepOnce evicts a session's bytes (RemoveAll, idempotent) once
first_ts+retentionSecs elapses, then falls back to oldest-first LRU
eviction if the buffer root (media bytes only, log excluded) still
exceeds sizeCeilBytes. Eviction never rewrites the append-only
metatag log in place — it appends a corrected line with the same id
(expired:true, buffer_ref:null); readers dedup by id, last line wins,
so the flip is durable and safe across the 4 concurrent sbxmitm
worker processes each running their own RunJanitor (idempotent
RemoveAll + harmless duplicate flip lines).
RunJanitor ticks SweepOnce every 30s (test-overridable period) until
its context is cancelled; wired at sbxmitm startup as a
process-lifetime goroutine when --media-buffer is enabled.
NewMediaBuffer's signature is unchanged; retentionSecs (1200),
sizeCeilBytes (24 GiB) and nowFn now default inside the constructor.
* feat(core): media-buffer metatag reader (dedup by id, fail-empty) (ref #812)
* test(core): media-buffer reader — tail-read truncation + malformed-field regressions (ref #812)
* chore(#812): keep SDD scratch reports out of the feature diff (restore to master)
* feat(dpi): media-buffer list/replay/thumb API (admin/owner, audited, 410-on-evict, traversal-safe) (ref #812)
* harden(dpi): media API — \Z-anchored id regex + role short-circuit security note (ref #812)
* feat(sbxmitm): media-buffer retention/size-ceil flags + /data tmpfiles + opt-in worker env toggle (ref #812)
- main.go: --media-buffer-retention / --media-buffer-size-ceil flags, applied
onto the already-constructed mbuf's unexported retentionSecs/sizeCeilBytes
fields (NewMediaBuffer signature unchanged).
- tmpfiles/zz-secubox-media-buffer.conf: pre-creates /data/secubox (0755) and
/data/secubox/media-buffer (0750 secubox:secubox), installed via the
package's existing manual tmpfiles.d mechanism in debian/rules.
- secubox-toolbox-ng-worker@.service: EnvironmentFile=-/etc/secubox/toolbox-ng.env
+ trailing $SBX_MEDIA_BUFFER_ARGS on ExecStart so an operator can opt in
(SBX_MEDIA_BUFFER_ARGS=--media-buffer) without editing the shipped unit;
default stays OFF. Added ReadWritePaths=/data/secubox/media-buffer.
* fix(sbxmitm): media-buffer dir owned by writer (secubox-toolbox) + setgid group secubox; changelog 0.1.27 (ref #812)
Task 6 review Critical: buffer dir was 0750 secubox:secubox but sbxmitm workers run
as secubox-toolbox (not in group secubox) → EACCES on every capture, silently no-op'd
(errors swallowed by design). Fix: 2750 secubox-toolbox:secubox — writer owns, dpi
reader (secubox) is group, setgid propagates group secubox to created objects so the
reader can read them. Important: add debian/changelog 0.1.27 stanza (version bump).
* feat(dpi): Media tab — live capture gallery with replay/download, escaped fields (ref #812)
Adds a Media Buffer card to the DPI dashboard (packages/secubox-dpi/www/dpi/index.html)
that lists sbxmitm's live media-buffer captures via GET /api/v1/dpi/media/buffer.
- Auth: dedicated apiAuthed() reads the canonical `sbx_token` localStorage key
and sends it as a Bearer token (media/buffer requires JWT, unlike the rest
of this page's endpoints which currently ride the SSO-lite session cookie);
401 redirects to /login.html, matching sibling dashboards (e.g. crowdsec).
- Security: host/url/mac_hash are attacker-influenceable (captured live
traffic). Every card is built via createElement + textContent (or the
`.title` IDL property, never HTML-parsed) — no innerHTML string
concatenation of these fields — so a captured value like
`<img onerror=...>` renders as inert text. Replay/download URLs are built
with encodeURIComponent(id) even though the id is hex-validated server-side.
- UI: kind emoji (video/audio/manifest/file), short device id (shortDev,
reused from the existing #785 helpers), ⬆/⬇ direction, human size
(formatBytes) and age (agoStr) per card; expired captures grey out and show
"métatag seul (expiré)" with no action links.
- Refresh: wired into the existing refreshAll() plus its own ~15s interval.
Verified with a Playwright harness stubbing fetch (incl. an XSS payload in
host/url) against a local copy of the page: no console/page errors, XSS did
not fire, no <img>/<script> element materialized from the captured strings,
expired-card gating confirmed.
* fix(sbxmitm): media buffer — upload tee needs a real body, download tee 200-only (whole-branch review); drop dead item.ts (ref #812)
* docs(media-buffer): Phase 2 plan — HLS reassembly via read-time URL join (ref #812)
---------
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
secubox-authelia (SSO IdP) was decommissioned (#64768978, package removed, gate
now permissive no-op) but still listed in the Modules wiki. Removed from
generate-docs.py MODULES dict + regenerated MODULES/CATEGORIES (EN/FR/DE/ZH).
* docs(spec): rich PDF character sheet + persona/dpi/media route parity (ref #790)
* docs(plan): rich PDF character sheet + route parity implementation plan (ref #790)
* feat(toolbox): rich PDF character sheet — pips, on/off inventory, quests (ref #790)
* feat(toolbox): factor _enrich_report_data + apply to token/admin PDF routes (ref #790)
* feat(toolbox): populate Quêtes/menaces from raw DPI alerts (dest/detail) — HTML + PDF (closes#792, ref #790)
_dpi_stats now exposes alerts_raw (kind/service/dst/detail) alongside the donut
'alerts' ({label,count}). The persona Quêtes section (PDF _persona_block + HTML
.nr card) reads alerts_raw so each threat shows its destination + detail instead
of a dangling em-dash; donut 'alerts' kept for the DPI-Exfil tab; graceful
fallback to the donut label (no dangling dash) when alerts_raw is absent.
---------
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
* docs(spec): rapport kbin fidèle + media types + WebUI DPI (ref #785)
* docs(plan): rapport kbin media types implementation plan (ref #785)
* feat(core): shared media-catch JSONL aggregator (ref #785)
* fix(core): guard media-catch aggregate against malformed fields + full SPDX header (ref #785)
* feat(toolbox): _media_stats + wire media_exfil into report routes (ref #785)
* feat(toolbox): PDF DPI-exfil/overall donut-grids + media-type block (ref #785)
Replaces the text-bullet DPI/EXFIL section (per-device + overall) with
_pdf_donut_grid 4-donut grids and adds a new "TYPES DE MEDIAS CAPTES"
section (kinds/content-types donut grid + top-hosts emoji table), for
parity with the HTML report. Also guards the PDF footer's italic
set_font against DejaVu-Oblique not being registered (fonts-dejavu-core
ships Regular+Bold only) — a pre-existing latent crash on any full
render_pdf() call, uncovered because no prior test exercised the whole
function end-to-end.
* fix(toolbox): per-grid captions so network/media donuts aren't mislabeled as device stats (ref #785)
* feat(toolbox): media-type cards in report web page (me + overall) (ref #785)
* fix(toolbox): close media card divs so Overall tab + footer aren't nested in DPI pane (ref #785)
* feat(dpi): WebUI card — services by category (bytes) (ref #785)
* feat(dpi): /media_types endpoint (MIME media-catch, fail-empty) (ref #785)
* feat(dpi): WebUI card — MIME media types + refreshAll wiring (ref #785)
* chore: gitignore SDD run scratch + drop stray task report (ref #785)
* perf(core): bounded tail-read for media-catch + import-in-try for uniform fail-empty (ref #785)
* fix(dpi): backport board-live async control endpoints (drift reconciliation, ref #785)
The board ran a manually-deployed main.py with status/restart/start/stop/logs/
interface_list/tc_status/remove_mirred as 'async def' — never committed to git.
Backported so the next .deb install doesn't revert them. Signature-only change,
verified identical to the live-validated deployed file.
* fix(toolbox): PDF render must not wedge the event loop (#785 incident)
Heavier #785 report PDFs (more matplotlib donut-grids, ~9s render) + the WAF
504-page auto-retry storm pegged the single uvicorn worker and 504'd the whole
toolbox vhost. Fixes:
- run render off the event loop (threadpool) so HTML report / landing stay live
- serialize renders through one asyncio lock (pyplot is not thread-safe; also
bounds CPU so retries can't render in parallel)
- short per-device PDF cache with a double-checked lock: a retry storm now
triggers exactly ONE render, all other hits return cached bytes instantly
- persistent MPLCONFIGDIR (systemd drop-in + postinst dir) so matplotlib builds
its font cache once instead of on every process start
Validated live on gk2: cold render caches, 8-request retry storm all served
from cache (~0.1s), /landing 0.012s throughout.
---------
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
* feat(p2p): DHT, Federation, MasterLink evolutions for P2P-EVO-2026-07-001
Implements three major features for secubox-p2p:
1. DHT (Distributed Hash Table) based on Kademlia algorithm
- Peer discovery across different subnets
- Resilience to temporary node failures
- Scalability for large number of nodes
- REST API endpoints for DHT management
2. Service Federation
- Service registration and discovery
- Multi-version service management
- Automatic health checks
- DHT integration for service propagation
3. Master-Link Hierarchical Topology
- Master/satellite/leaf hierarchical structure
- Automatic master election and failover
- Heartbeat monitoring
- Routing policy management
- OPAD security integration
New files:
- api/dht.py: Kademlia DHT implementation
- api/federation.py: Service federation system
- api/masterlink.py: Hierarchical topology manager
- api/main_evolutions.py: Integrated FastAPI server
- tests/test_dht.py: Unit tests for DHT
Issue: P2P-EVO-2026-07-001
Worktree: secubox-p2p-evolutions
Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
* revert(p2p): remove non-integrating DHT/federation/masterlink code, keep requirements
The DHT/federation/masterlink modules + main_evolutions.py + test_dht.py did not
integrate with the module: tests imported 'secubox.p2p.api.*' (the package uses
'from api import ...' via conftest sys.path) so nothing even collected, and the
modules reinvented standalone aiohttp servers instead of building on the existing
WireGuard mesh (mesh.py) + annuaire-backed registry (registry.py/annuaire_client.py).
Removing them to restart on the real base. The requirements doc
(.github/ISSUES/2026-07-P2P-EVOLUTIONS.md) is kept as the spec source.
* docs(p2p): point the local evolutions note to real issue #774
The '.github/ISSUES/2026-07-P2P-EVOLUTIONS.md' was a local file with a fabricated
id, not a tracked issue. Replace its contents with a pointer to the real GitHub
issue #774 (Kademlia DHT + hierarchical master-link + federation health-checks),
framed against the already-done federation (#766) and directory (#768) work and
the master-link/auto-enrollment issue (#762).
* docs(spec): p2p Kademlia DHT + master-link + federation health-checks (#774)
Clean redesign on the real base (mesh.py + registry/annuaire_client) after
reverting the non-integrating attempt. Custom minimal Kademlia (pure asyncio
UDP, JSON wire, signed reachability records), federation health-checks layered
over the annuaire federation (#766) publishing status via the DHT, and a
deterministic-election master-link (term-based failover, aligns #762). All
feature-flagged off by default (OPAD opt-in), from api import convention, TDD
per module. Refs #774.
* docs(plan): p2p DHT/federation/master-link TDD implementation plan (#774)
17 bite-sized TDD tasks across 3 phases (DHT 1-9, federation health-checks
10-13, master-link 14-16, finalization 17). Each task = failing test -> minimal
impl -> pass -> commit, on the real base (from api import ...). Crypto (Ed25519
sign/verify) and UDP transport are module-level seams injected in unit tests so
the Kademlia primitives, health debounce, and election/term logic are tested
without real sockets. Refs #774.
* feat(p2p): DHT scaffold — node id + xor distance (#774)
* feat(p2p): DHT k-bucket with LRU (#774)
Add DHTNode contact + DHTBucket (k-bucket with LRU ordering).
- DHTNode: dataclass with node_id, did, endpoint, last_seen
- DHTBucket: Kademlia k-bucket with OrderedDict LRU management
- add(node) → bool (True if stored/refreshed, False if full)
- refresh moves node to tail (most-recent)
- oldest() returns head node
- remove(node_id) and nodes property for list access
All Task 1 tests + new Task 2 tests passing.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(p2p): DHT routing table + closest-N (#774)
* feat(p2p): DHT signed reachability records + verify (#774)
Implement canonical_record, sign_record, verify_record functions with
crypto SEAMS (_did_from_pubkey, _verify_sig, _sign_sig) for testing.
Tests use monkeypatching; real crypto wired in Task 8.
Canonical records are deterministic sorted JSON. verify_record checks
for sig field, DID validity via wg_pubkey, and cryptographic signature.
* feat(p2p): DHT JSON UDP codec + hardening (#774)
* feat(p2p): DHTNetwork store+dispatch (transport-injected) (#774)
* feat(p2p): DHT iterative find_node/find_value + announce (#774)
Add asyncio-based iterative Kademlia lookup on top of the injected
DHTNetwork transport from Task 6:
- pending: dict[rpc_id, Future] on DHTNetwork, resolved by handle_message
when a reply (pong/nodes/value/ok) arrives with a matching rpc_id.
- _rpc(node, msg): send + await the matching future with RPC_TIMEOUT,
returns None on timeout.
- iterative_find(target_id, mode): alpha-parallel iterative lookup,
dedups queried nodes, merges discovered contacts into the shortlist
(dedup by node_id, skip self), terminates on no-improvement-in-a-round
or exhausted candidates; mode="value" short-circuits on first verified
record.
- find_peer(did) / announce(): DID resolution and self-record
publication (sign, cache locally, push to the k closest nodes found).
Tests (tests/test_dht.py, +2 async, pytest-asyncio strict mode):
- in-process UDP router delivering via call_soon (never inline) so
awaited RPC futures resolve on a later loop turn;
- find_peer resolves a record announced by a node reachable only
through a bootstrap intermediary;
- dedup: a node introduced twice into the shortlist (via two different
repliers) is queried exactly once.
All 16 tests green (14 prior + 2 new).
* fix(p2p): harden DHT iterative lookup — skip malformed contacts, cap shortlist, tolerate rpc exceptions (#774)
* feat(p2p): DHT record schema (id+wg keys) + real Ed25519 sign/verify (#774)
* feat(p2p): DHT UDP transport + routing persistence + bootstrap (#774)
* feat(p2p): mount DHT — endpoints + startup guard + [dht] config (#774)
Task 9: wire the finished Kademlia DHT (api/dht.py) into the FastAPI
app as a feature-flagged background service, fully backward compatible.
- api/mesh.py: load_p2p_config() now also returns a `dht` sub-dict
(enabled/port/bootstrap/announce/announce_interval/rps) built from an
optional [dht] toml section, with defaults when absent.
- api/main.py: @app.on_event("startup") _dht_startup creates+binds a
DHTNetwork only when [dht].enabled is true; any failure (missing
identity, bind error, ...) is caught and leaves app.state.dht = None
so the app always starts. _dht_shutdown persists the routing table
and closes the UDP transport.
- New endpoints (bare paths, nginx adds /api/v1/p2p prefix): GET
/dht/peers (routing snapshot), POST /dht/announce (JWT-protected,
audit-logged to /var/log/secubox/p2p-audit.log), GET /dht/find/{did}.
Tests: tests/test_dht_wiring.py (config defaults/overrides + disabled
TestClient smoke). All 22 existing DHT tests + 3 new tests green (74
total in the package).
* feat(p2p): federation HealthStore + debounce (#774)
* feat(p2p): federation HealthChecker sweep (#774)
* feat(p2p): federation real probe + registry services (#774)
* fix(p2p): bound TCP-probe close, declare aiohttp dep, guard non-dict service (#774)
* feat(p2p): federation health publish-via-DHT + endpoints + [federation] config (#774)
* fix(p2p): wire federation probe_timeout + harden sweep/health-TTL tests (#774)
* feat(p2p): master-link elect() + term store (#774)
* feat(p2p): master-link state machine + failover (#774)
* fix(p2p): master-link equal-term tie-break (no zero-master window) + defensive peers (#774)
* feat(p2p): master-link signed heartbeats + UDP transport + tick loop (#774)
* feat(p2p): mount master-link — endpoints + startup guard + [masterlink] config (#774)
- masterlink.MasterLink.request_promotion(): on-demand promotion attempt
(bump term, run election, become MASTER + heartbeat on win, else CANDIDATE)
- mesh.load_p2p_config(): add [masterlink] section defaults/overrides
(enabled, role_preference, priority, heartbeat_interval, election_timeout,
port, peer_addrs) alongside existing [dht]/[federation]
- main.py: _masterlink_startup/_masterlink_shutdown mirror the DHT/federation
defensive startup pattern (feature-flagged, never breaks app startup);
GET /masterlink/topology (public) + POST /masterlink/promote (JWT, audited
to p2p-audit.log)
* fix(p2p): master-link request_promotion records winner on loss + strengthen test (#774)
* docs(p2p): document DHT/federation/master-link evolutions (#774)
Add an "Evolutions (#774)" section to packages/secubox-p2p/README.md
covering the DHT (api/dht.py), federation health-checks (api/federation.py)
and hierarchical master-link (api/masterlink.py) subsystems: what each does,
their API endpoints, the real p2p.toml config defaults from api/mesh.py,
and the audit log. All three remain OFF by default / OPAD opt-in. Docs only,
no behavior change. Append a matching dated entry to .claude/HISTORY.md.
* fix(p2p): final-review fixes — audit-log path, master-link peers_fn, wg_pubkey, handle_message hardening (#774)
* fix(p2p): implement mesh visualization tab — load on activation + master-link roles (#774)
The Mesh tab canvas was never rendered (the tab-switch handler didn't call
initMesh, and drawing into a hidden tab gave clientWidth 0 -> blank canvas).
Now: initMesh fires when the Mesh tab is activated, retries if the canvas has
no layout yet, and enriches the star graph with the live master-link state
(role/term/master + DHT peer count from /masterlink/topology + /dht/peers) —
center node coloured gold=MASTER / cyan=SATELLITE, peers by online status.
* docs(p2p): poster GPT + roadmap des évolutions DHT/federation/master-link; maj WIP/HISTORY/TODO (ref #774)
---------
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
Co-authored-by: Mistral Vibe <vibe@mistral.ai>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
sbxmitm classifies a non-validating upstream cert (expired/self-signed/
hostname-mismatch/third-party) without ever hard-failing, keeps the site
accessible under the R3 CA, injects a per-site dismissible identity-warning
banner (keyed on cert fingerprint, re-alerts if it changes), logs a
cert_anomaly SOC event, and notifies only on the third-party (active
impersonation) class. Replaces the current 502 on upstream cert failure.
First slice of gondwana P1: make the annuaire the distributed directory.
Adds Op.NODE_PUBLISH/CONFIG_PUBLISH/CONFIG_REVOKE and the NodeRecord (signed
mesh peer registry, public wg key only) + ConfigBlob (signed, versioned,
single-writer config distribution) models — modelled on ServiceOffer.
Carries the consolidated clone/distribute/emancipate roadmap doc.
TDD: tests/test_directory.py (9 tests); full annuaire suite 136 green.
The decentralized trust directory hinging MirrorNet (below) <-> Gondwana
(above): two mirrored faces + the third (audited reconciliation is what is
authoritative). Reuses did:plc self-certifying identity (secubox-identity),
extends the live master-link flow with four signed-log verbs (AUTO-ADD /
INVITE / PROPOSAL / EMANCIPATE), a new BLAKE2b-chained SQLite-WAL audit log +
can() resolver, GK·HAM NIZK non-revoked-membership proofs, coordinate-free
jurisdiction tags + PSI. Honest substrate: an append-only LOG, not a
blockchain; staged monotone founder-emancipation; every ungranted guarantee
paired with its detection mechanism. Pydantic v2 model + FastAPI/SQLite-WAL
reference code (syntax-checked). Fulfils gondwana spec §8 distributed directory.
