* docs(sentinel): design — inline IOC gate + async Go analyzer + spyware packs (ref #823)
* docs(sentinel): implementation plan — 12 tasks (ref #823)
* feat(sentinel): IOC model + IOCSet matchers (ref #823)
* feat(sentinel): pack loader + live-overlay merge + hot-reload (ref #823)
Add internal/sentinel/pack.go: Pack (JSON base/overlay content bundle),
LoadBasePack, MergePacks (base + N overlays, override by Type+Value via
IOCSet's per-type maps), and Loader (NewLoader/Set/MaybeReload/YaraRules).
Loader mirrors cmd/sbxmitm/policy.go's LoadPolicy/maybeReload mtime-based
hot-reload pattern, adapted for a directory of a DYNAMIC *.json file set
(glob-based) rather than policy.go's fixed Target list. Fail-safe: a
corrupt overlay pack is skipped+logged, never wipes the base or panics;
MaybeReload is a cheap stat-only no-op when nothing on disk changed;
Set()/YaraRules() read via atomic.Pointer so a reload swap never torn-reads
a concurrent hot-path lookup.
Adds JSON tags to Task 1's IOC struct (type/value/class/severity/source/
action) so packs round-trip through encoding/json.
* fix(sentinel): pack loader — fail-closed on base-wipe + throttle MaybeReload + url_regex override + yara dedup (ref #823)
Review found 1 Critical + 2 Important + 1 Minor in Task 2 (pack loader):
- Critical: a vanished/degraded base pack directory (deleted dir, mount
blip, mid-dpkg-upgrade window) could silently empty the live detection
set on the next MaybeReload. loaderState now tracks baseCount; a reload
that would drop base IOCs from >0 to 0 is refused (prior state kept,
warning logged) instead of swapped in.
- Important: MaybeReload() is called per-flow by Task 3's inline gate: add
a 5s default throttle (mirrors cmd/sbxmitm/policy.go's reloadThrottle)
so a hot-path call is a pure no-op until the window elapses.
- Important: url_regex IOCs previously only appended, so a live overlay
could never override a base pack's URL rule (base always matched
first). Add urlIndex (pattern -> slice index) so re-adding the same
pattern replaces the entry in place, keeping deterministic match order.
- Minor: YaraRules() now dedups identical rule bodies across base+overlay
packs instead of emitting duplicates.
go test ./internal/sentinel/: 21/21 PASS. go vet/gofmt/go build clean.
* feat(sentinel): inline gate matcher (fail-open, highest-severity) (ref #823)
* feat(sentinel): bbolt verdict store (pure-Go, WAL-free, TTL prune) (ref #823)
* feat(sentinel): action scorer — heuristic/low-confidence forced report-only (ref #823)
* feat(sentinel): bounded fire-and-forget mirror channel (drop-with-count) (ref #823)
* fix(sentinel): mirror Emit always copies body — no aliasing of caller's hot-path buffer (ref #823)
* feat(sbxmitm): inline Sentinel gate — neutralize high-confidence, mirror rest, fail-open (ref #823)
Wire internal/sentinel's inline IOC gate into the shared mitmPipeline response
path (covers both the transparent R3 and CONNECT accept paths without drift):
build a FlowMeta from Host/URL/ClientIP/MacHash after up.Do, call the gate, and
on a HIGH-CONFIDENCE known-infra hit neutralize inline — block/sinkhole serve a
Sentinel block page instead of the upstream response, strip serves the response
with an emptied body. Everything else (miss, heuristic, low-confidence, report)
is mirrored bounded fire-and-forget to the async analyzer and proceeds unchanged.
Fail-open everywhere: SENTINEL_ENABLED unset/false, a pack-load failure, a nil
hook, or a gate panic all degrade to a transparent passthrough — a Sentinel
setup problem can never break browsing. With the engine off, behavior is
byte-identical to today. Env: SENTINEL_ENABLED / SENTINEL_PACK_DIR /
SENTINEL_OVERLAY_DIR / SENTINEL_MIRROR_SOCK.
Hot path (BenchmarkSentinelInspectMiss, -benchmem): 101.0 ns/op, 0 B/op,
0 allocs/op — one throttled gate Match + a non-blocking mirror Emit; no YARA or
heavy analysis inline.
go test ./cmd/sbxmitm/... -race → ok (all pre-existing + 7 new green).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(sentinel): sbx-sentinel async daemon scaffold + store wiring (ref #823)
* feat(sentinel): YARA engine (cgo, build-tagged) + no-cgo stub (ref #823)
Add internal/sentinel/yara.go (cgo && yara) wrapping
github.com/hillu/go-yara/v4 for compiled-rule body scanning, and
yara_stub.go (!yara) as the no-op default: NewYaraEngine always
succeeds, Scan/Analyze always return nil. Both expose identical
signatures (YaraEngine, NewYaraEngine, Scan, Analyze, Close) so
sbx-sentinel/sbxmitm can wire a YaraEngine unconditionally.
yara_test.go (untagged) proves the default build works with YARA
disabled. yara_real_test.go (yara tag) compiles a trivial EICAR rule
and exercises the real libyara-backed engine — split into its own
file since Go build constraints are file-scoped and can't mix a
tagged and untagged test in one file.
go-yara is now an indirect go.mod requirement and vendored, but is
only reachable via the yara build tag: default `go build ./...` and
`CGO_ENABLED=0 go build ./...` both stay clean, no libyara/cgo
required. `-tags yara` needs libyara-dev + a C toolchain (documented
in yara.go's header).
* fix(sentinel): YARA verdict confidence 90 (>= threshold) so a confirmed malware match actually strips, not report-only (ref #823)
* feat(sentinel): behavioral engine — beaconing + zero-click heuristics (report-only) (ref #823)
* fix(sentinel): tighten one-time-link heuristic — exclude file-extension + base64 shapes (cut zero-click FPs) (ref #823)
* feat(sentinel): spyware indicator engine + commercial-spyware base pack (MVT/CitizenLab) (ref #823)
* fix(sentinel): base-pack declared-block IOCs severity>=85 so they actually block + general guard test (ref #823)
Three shipped base-pack IOCs declared action:"block" but severity:80,
below scorer.go's HighConfidenceThreshold=85 — FinalizeAction silently
downgrades any Confidence below that threshold to report-only, so these
entries never actually blocked despite being declared as known-bad:
- packs/base/malware.json: ip 198.51.100.23 (class malware)
- packs/base/malware.json: cert_sha1 ...01234567 (class trojan)
- packs/base/botnet.json: ja3 deadbeef... (class botnet_c2)
Bumped all three to severity 90 so they survive FinalizeAction and
actually block, matching every other declared-block entry in the base
packs.
Added TestBasePackDeclaredBlockIsHighSeverity, a general guard over
every base pack file/entry: any IOC with an auto-neutralize action
(block/strip/sinkhole) must carry Severity >= HighConfidenceThreshold.
This generalizes the narrower TestBasePackKnownInfraIsHighSeverity
(spyware classes only) so a future pack entry in any class can't ship
a "block" that silently never blocks. Confirmed the new test fails red
against the pre-fix JSON (all 3 offenders reported) and passes green
after the severity bump.
* feat(sentinel): proposal/solution reporter — RenderReport(Verdict) (ref #823)
* feat(sentinel): wire real analyzers into sbx-sentinel + read-only status HTTP surface (ref #823)
* feat(sentinel): live threat-feed → overlay pack fetcher (fail-safe, atomic) (ref #823)
* feat(sentinel): live feed overlay + reporter + daemon analyzer wiring + packaging (ref #823)
* fix(sentinel): feeds fetcher — add curl --fail so HTTP 4xx/5xx is treated as a fetch failure (ref #823)
* fix(sbxmitm): drop mis-wired ClientIP from inline Sentinel FlowMeta (source IP must not match destination-C2 IP IOCs) (ref #823)
---------
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* docs(media-buffer): design spec — dashcam capture + replay + metatag lifecycle (ref #812)
* docs(media-buffer): phase-1 implementation plan (capture+janitor+API+DPI tab) (ref #812)
* feat(sbxmitm): media buffer store + metatag writer (ref #812)
* fix(sbxmitm): media-buffer review — 4096 atomic-append cap + *string buffer_ref (ref #812)
Task 1 review (Important): mediaBufferLineMax 8192→4096 to match mediacatch.go's
PIPE_BUF atomic-append guarantee across 4 workers; BufferRef string→*string so the
janitor (Task 3) can flip it to JSON null on eviction.
* feat(sbxmitm): tee media up/download bodies into the buffer (ref #812)
Wire the R4 media buffer tees into mitmPipeline: the download hook wraps
resp.Body and the upload path wraps req.Body via teeReadCloser, capturing
whole-file media into the rolling /data buffer. Add the mbuf field + flags
(--media-buffer default OFF, --media-buffer-root, --media-buffer-per-object).
Design upgrade over the plan sketch: ObjectWriter.Write is now NON-BLOCKING.
Instead of io.TeeReader driving a synchronous os.File.Write on the client read
path (which would add latency / could stall large media on ARM eMMC), Write
copies each chunk into a bounded channel and returns len(p),nil immediately; a
background goroutine started at Capture drains the channel to disk. A full
channel drops the chunk and flags truncated — never blocks, never errors the
proxied flow (design 4.2/7). Close signals the drainer, waits for it to flush
queued chunks + close the file, then appends the metatag; it is idempotent,
deadlock-free (sole closer of ch) and safe if Write is never called or races
Close.
* fix(sbxmitm): media download tee — defer closure so w.Close runs (ref #812)
defer resp.Body.Close() bound the ORIGINAL upstream body at the defer
statement, before the media-buffer tee reassigned resp.Body — so the
tee's Close() (and w.Close, which flushes the sink/metatag/drain
goroutine) never ran on any download. Wrap the close in a closure so
it reads resp.Body at return time instead.
Adds TestTeeDeferOrderingMatchesMainGo, which replicates main.go's
exact defer-then-reassign ordering and fails against the old plain
defer form (verified manually, then restored the fix).
Also: drop the dead write-only ObjectWriter.dropped field (truncated
already covers it), document Capture's reserved contentLen param, and
note that ObjectWriter.Close is allowed to block (post-stream
finalisation), unlike Write.
* feat(sbxmitm): media buffer janitor — time + LRU eviction, metatag survives (ref #812)
SweepOnce evicts a session's bytes (RemoveAll, idempotent) once
first_ts+retentionSecs elapses, then falls back to oldest-first LRU
eviction if the buffer root (media bytes only, log excluded) still
exceeds sizeCeilBytes. Eviction never rewrites the append-only
metatag log in place — it appends a corrected line with the same id
(expired:true, buffer_ref:null); readers dedup by id, last line wins,
so the flip is durable and safe across the 4 concurrent sbxmitm
worker processes each running their own RunJanitor (idempotent
RemoveAll + harmless duplicate flip lines).
RunJanitor ticks SweepOnce every 30s (test-overridable period) until
its context is cancelled; wired at sbxmitm startup as a
process-lifetime goroutine when --media-buffer is enabled.
NewMediaBuffer's signature is unchanged; retentionSecs (1200),
sizeCeilBytes (24 GiB) and nowFn now default inside the constructor.
* feat(core): media-buffer metatag reader (dedup by id, fail-empty) (ref #812)
* test(core): media-buffer reader — tail-read truncation + malformed-field regressions (ref #812)
* chore(#812): keep SDD scratch reports out of the feature diff (restore to master)
* feat(dpi): media-buffer list/replay/thumb API (admin/owner, audited, 410-on-evict, traversal-safe) (ref #812)
* harden(dpi): media API — \Z-anchored id regex + role short-circuit security note (ref #812)
* feat(sbxmitm): media-buffer retention/size-ceil flags + /data tmpfiles + opt-in worker env toggle (ref #812)
- main.go: --media-buffer-retention / --media-buffer-size-ceil flags, applied
onto the already-constructed mbuf's unexported retentionSecs/sizeCeilBytes
fields (NewMediaBuffer signature unchanged).
- tmpfiles/zz-secubox-media-buffer.conf: pre-creates /data/secubox (0755) and
/data/secubox/media-buffer (0750 secubox:secubox), installed via the
package's existing manual tmpfiles.d mechanism in debian/rules.
- secubox-toolbox-ng-worker@.service: EnvironmentFile=-/etc/secubox/toolbox-ng.env
+ trailing $SBX_MEDIA_BUFFER_ARGS on ExecStart so an operator can opt in
(SBX_MEDIA_BUFFER_ARGS=--media-buffer) without editing the shipped unit;
default stays OFF. Added ReadWritePaths=/data/secubox/media-buffer.
* fix(sbxmitm): media-buffer dir owned by writer (secubox-toolbox) + setgid group secubox; changelog 0.1.27 (ref #812)
Task 6 review Critical: buffer dir was 0750 secubox:secubox but sbxmitm workers run
as secubox-toolbox (not in group secubox) → EACCES on every capture, silently no-op'd
(errors swallowed by design). Fix: 2750 secubox-toolbox:secubox — writer owns, dpi
reader (secubox) is group, setgid propagates group secubox to created objects so the
reader can read them. Important: add debian/changelog 0.1.27 stanza (version bump).
* feat(dpi): Media tab — live capture gallery with replay/download, escaped fields (ref #812)
Adds a Media Buffer card to the DPI dashboard (packages/secubox-dpi/www/dpi/index.html)
that lists sbxmitm's live media-buffer captures via GET /api/v1/dpi/media/buffer.
- Auth: dedicated apiAuthed() reads the canonical `sbx_token` localStorage key
and sends it as a Bearer token (media/buffer requires JWT, unlike the rest
of this page's endpoints which currently ride the SSO-lite session cookie);
401 redirects to /login.html, matching sibling dashboards (e.g. crowdsec).
- Security: host/url/mac_hash are attacker-influenceable (captured live
traffic). Every card is built via createElement + textContent (or the
`.title` IDL property, never HTML-parsed) — no innerHTML string
concatenation of these fields — so a captured value like
`<img onerror=...>` renders as inert text. Replay/download URLs are built
with encodeURIComponent(id) even though the id is hex-validated server-side.
- UI: kind emoji (video/audio/manifest/file), short device id (shortDev,
reused from the existing #785 helpers), ⬆/⬇ direction, human size
(formatBytes) and age (agoStr) per card; expired captures grey out and show
"métatag seul (expiré)" with no action links.
- Refresh: wired into the existing refreshAll() plus its own ~15s interval.
Verified with a Playwright harness stubbing fetch (incl. an XSS payload in
host/url) against a local copy of the page: no console/page errors, XSS did
not fire, no <img>/<script> element materialized from the captured strings,
expired-card gating confirmed.
* fix(sbxmitm): media buffer — upload tee needs a real body, download tee 200-only (whole-branch review); drop dead item.ts (ref #812)
* docs(media-buffer): Phase 2 plan — HLS reassembly via read-time URL join (ref #812)
---------
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
secubox-authelia (SSO IdP) was decommissioned (#64768978, package removed, gate
now permissive no-op) but still listed in the Modules wiki. Removed from
generate-docs.py MODULES dict + regenerated MODULES/CATEGORIES (EN/FR/DE/ZH).
* docs(spec): rich PDF character sheet + persona/dpi/media route parity (ref #790)
* docs(plan): rich PDF character sheet + route parity implementation plan (ref #790)
* feat(toolbox): rich PDF character sheet — pips, on/off inventory, quests (ref #790)
* feat(toolbox): factor _enrich_report_data + apply to token/admin PDF routes (ref #790)
* feat(toolbox): populate Quêtes/menaces from raw DPI alerts (dest/detail) — HTML + PDF (closes#792, ref #790)
_dpi_stats now exposes alerts_raw (kind/service/dst/detail) alongside the donut
'alerts' ({label,count}). The persona Quêtes section (PDF _persona_block + HTML
.nr card) reads alerts_raw so each threat shows its destination + detail instead
of a dangling em-dash; donut 'alerts' kept for the DPI-Exfil tab; graceful
fallback to the donut label (no dangling dash) when alerts_raw is absent.
---------
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
* docs(spec): rapport kbin fidèle + media types + WebUI DPI (ref #785)
* docs(plan): rapport kbin media types implementation plan (ref #785)
* feat(core): shared media-catch JSONL aggregator (ref #785)
* fix(core): guard media-catch aggregate against malformed fields + full SPDX header (ref #785)
* feat(toolbox): _media_stats + wire media_exfil into report routes (ref #785)
* feat(toolbox): PDF DPI-exfil/overall donut-grids + media-type block (ref #785)
Replaces the text-bullet DPI/EXFIL section (per-device + overall) with
_pdf_donut_grid 4-donut grids and adds a new "TYPES DE MEDIAS CAPTES"
section (kinds/content-types donut grid + top-hosts emoji table), for
parity with the HTML report. Also guards the PDF footer's italic
set_font against DejaVu-Oblique not being registered (fonts-dejavu-core
ships Regular+Bold only) — a pre-existing latent crash on any full
render_pdf() call, uncovered because no prior test exercised the whole
function end-to-end.
* fix(toolbox): per-grid captions so network/media donuts aren't mislabeled as device stats (ref #785)
* feat(toolbox): media-type cards in report web page (me + overall) (ref #785)
* fix(toolbox): close media card divs so Overall tab + footer aren't nested in DPI pane (ref #785)
* feat(dpi): WebUI card — services by category (bytes) (ref #785)
* feat(dpi): /media_types endpoint (MIME media-catch, fail-empty) (ref #785)
* feat(dpi): WebUI card — MIME media types + refreshAll wiring (ref #785)
* chore: gitignore SDD run scratch + drop stray task report (ref #785)
* perf(core): bounded tail-read for media-catch + import-in-try for uniform fail-empty (ref #785)
* fix(dpi): backport board-live async control endpoints (drift reconciliation, ref #785)
The board ran a manually-deployed main.py with status/restart/start/stop/logs/
interface_list/tc_status/remove_mirred as 'async def' — never committed to git.
Backported so the next .deb install doesn't revert them. Signature-only change,
verified identical to the live-validated deployed file.
* fix(toolbox): PDF render must not wedge the event loop (#785 incident)
Heavier #785 report PDFs (more matplotlib donut-grids, ~9s render) + the WAF
504-page auto-retry storm pegged the single uvicorn worker and 504'd the whole
toolbox vhost. Fixes:
- run render off the event loop (threadpool) so HTML report / landing stay live
- serialize renders through one asyncio lock (pyplot is not thread-safe; also
bounds CPU so retries can't render in parallel)
- short per-device PDF cache with a double-checked lock: a retry storm now
triggers exactly ONE render, all other hits return cached bytes instantly
- persistent MPLCONFIGDIR (systemd drop-in + postinst dir) so matplotlib builds
its font cache once instead of on every process start
Validated live on gk2: cold render caches, 8-request retry storm all served
from cache (~0.1s), /landing 0.012s throughout.
---------
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
* feat(p2p): DHT, Federation, MasterLink evolutions for P2P-EVO-2026-07-001
Implements three major features for secubox-p2p:
1. DHT (Distributed Hash Table) based on Kademlia algorithm
- Peer discovery across different subnets
- Resilience to temporary node failures
- Scalability for large number of nodes
- REST API endpoints for DHT management
2. Service Federation
- Service registration and discovery
- Multi-version service management
- Automatic health checks
- DHT integration for service propagation
3. Master-Link Hierarchical Topology
- Master/satellite/leaf hierarchical structure
- Automatic master election and failover
- Heartbeat monitoring
- Routing policy management
- OPAD security integration
New files:
- api/dht.py: Kademlia DHT implementation
- api/federation.py: Service federation system
- api/masterlink.py: Hierarchical topology manager
- api/main_evolutions.py: Integrated FastAPI server
- tests/test_dht.py: Unit tests for DHT
Issue: P2P-EVO-2026-07-001
Worktree: secubox-p2p-evolutions
Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
* revert(p2p): remove non-integrating DHT/federation/masterlink code, keep requirements
The DHT/federation/masterlink modules + main_evolutions.py + test_dht.py did not
integrate with the module: tests imported 'secubox.p2p.api.*' (the package uses
'from api import ...' via conftest sys.path) so nothing even collected, and the
modules reinvented standalone aiohttp servers instead of building on the existing
WireGuard mesh (mesh.py) + annuaire-backed registry (registry.py/annuaire_client.py).
Removing them to restart on the real base. The requirements doc
(.github/ISSUES/2026-07-P2P-EVOLUTIONS.md) is kept as the spec source.
* docs(p2p): point the local evolutions note to real issue #774
The '.github/ISSUES/2026-07-P2P-EVOLUTIONS.md' was a local file with a fabricated
id, not a tracked issue. Replace its contents with a pointer to the real GitHub
issue #774 (Kademlia DHT + hierarchical master-link + federation health-checks),
framed against the already-done federation (#766) and directory (#768) work and
the master-link/auto-enrollment issue (#762).
* docs(spec): p2p Kademlia DHT + master-link + federation health-checks (#774)
Clean redesign on the real base (mesh.py + registry/annuaire_client) after
reverting the non-integrating attempt. Custom minimal Kademlia (pure asyncio
UDP, JSON wire, signed reachability records), federation health-checks layered
over the annuaire federation (#766) publishing status via the DHT, and a
deterministic-election master-link (term-based failover, aligns #762). All
feature-flagged off by default (OPAD opt-in), from api import convention, TDD
per module. Refs #774.
* docs(plan): p2p DHT/federation/master-link TDD implementation plan (#774)
17 bite-sized TDD tasks across 3 phases (DHT 1-9, federation health-checks
10-13, master-link 14-16, finalization 17). Each task = failing test -> minimal
impl -> pass -> commit, on the real base (from api import ...). Crypto (Ed25519
sign/verify) and UDP transport are module-level seams injected in unit tests so
the Kademlia primitives, health debounce, and election/term logic are tested
without real sockets. Refs #774.
* feat(p2p): DHT scaffold — node id + xor distance (#774)
* feat(p2p): DHT k-bucket with LRU (#774)
Add DHTNode contact + DHTBucket (k-bucket with LRU ordering).
- DHTNode: dataclass with node_id, did, endpoint, last_seen
- DHTBucket: Kademlia k-bucket with OrderedDict LRU management
- add(node) → bool (True if stored/refreshed, False if full)
- refresh moves node to tail (most-recent)
- oldest() returns head node
- remove(node_id) and nodes property for list access
All Task 1 tests + new Task 2 tests passing.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(p2p): DHT routing table + closest-N (#774)
* feat(p2p): DHT signed reachability records + verify (#774)
Implement canonical_record, sign_record, verify_record functions with
crypto SEAMS (_did_from_pubkey, _verify_sig, _sign_sig) for testing.
Tests use monkeypatching; real crypto wired in Task 8.
Canonical records are deterministic sorted JSON. verify_record checks
for sig field, DID validity via wg_pubkey, and cryptographic signature.
* feat(p2p): DHT JSON UDP codec + hardening (#774)
* feat(p2p): DHTNetwork store+dispatch (transport-injected) (#774)
* feat(p2p): DHT iterative find_node/find_value + announce (#774)
Add asyncio-based iterative Kademlia lookup on top of the injected
DHTNetwork transport from Task 6:
- pending: dict[rpc_id, Future] on DHTNetwork, resolved by handle_message
when a reply (pong/nodes/value/ok) arrives with a matching rpc_id.
- _rpc(node, msg): send + await the matching future with RPC_TIMEOUT,
returns None on timeout.
- iterative_find(target_id, mode): alpha-parallel iterative lookup,
dedups queried nodes, merges discovered contacts into the shortlist
(dedup by node_id, skip self), terminates on no-improvement-in-a-round
or exhausted candidates; mode="value" short-circuits on first verified
record.
- find_peer(did) / announce(): DID resolution and self-record
publication (sign, cache locally, push to the k closest nodes found).
Tests (tests/test_dht.py, +2 async, pytest-asyncio strict mode):
- in-process UDP router delivering via call_soon (never inline) so
awaited RPC futures resolve on a later loop turn;
- find_peer resolves a record announced by a node reachable only
through a bootstrap intermediary;
- dedup: a node introduced twice into the shortlist (via two different
repliers) is queried exactly once.
All 16 tests green (14 prior + 2 new).
* fix(p2p): harden DHT iterative lookup — skip malformed contacts, cap shortlist, tolerate rpc exceptions (#774)
* feat(p2p): DHT record schema (id+wg keys) + real Ed25519 sign/verify (#774)
* feat(p2p): DHT UDP transport + routing persistence + bootstrap (#774)
* feat(p2p): mount DHT — endpoints + startup guard + [dht] config (#774)
Task 9: wire the finished Kademlia DHT (api/dht.py) into the FastAPI
app as a feature-flagged background service, fully backward compatible.
- api/mesh.py: load_p2p_config() now also returns a `dht` sub-dict
(enabled/port/bootstrap/announce/announce_interval/rps) built from an
optional [dht] toml section, with defaults when absent.
- api/main.py: @app.on_event("startup") _dht_startup creates+binds a
DHTNetwork only when [dht].enabled is true; any failure (missing
identity, bind error, ...) is caught and leaves app.state.dht = None
so the app always starts. _dht_shutdown persists the routing table
and closes the UDP transport.
- New endpoints (bare paths, nginx adds /api/v1/p2p prefix): GET
/dht/peers (routing snapshot), POST /dht/announce (JWT-protected,
audit-logged to /var/log/secubox/p2p-audit.log), GET /dht/find/{did}.
Tests: tests/test_dht_wiring.py (config defaults/overrides + disabled
TestClient smoke). All 22 existing DHT tests + 3 new tests green (74
total in the package).
* feat(p2p): federation HealthStore + debounce (#774)
* feat(p2p): federation HealthChecker sweep (#774)
* feat(p2p): federation real probe + registry services (#774)
* fix(p2p): bound TCP-probe close, declare aiohttp dep, guard non-dict service (#774)
* feat(p2p): federation health publish-via-DHT + endpoints + [federation] config (#774)
* fix(p2p): wire federation probe_timeout + harden sweep/health-TTL tests (#774)
* feat(p2p): master-link elect() + term store (#774)
* feat(p2p): master-link state machine + failover (#774)
* fix(p2p): master-link equal-term tie-break (no zero-master window) + defensive peers (#774)
* feat(p2p): master-link signed heartbeats + UDP transport + tick loop (#774)
* feat(p2p): mount master-link — endpoints + startup guard + [masterlink] config (#774)
- masterlink.MasterLink.request_promotion(): on-demand promotion attempt
(bump term, run election, become MASTER + heartbeat on win, else CANDIDATE)
- mesh.load_p2p_config(): add [masterlink] section defaults/overrides
(enabled, role_preference, priority, heartbeat_interval, election_timeout,
port, peer_addrs) alongside existing [dht]/[federation]
- main.py: _masterlink_startup/_masterlink_shutdown mirror the DHT/federation
defensive startup pattern (feature-flagged, never breaks app startup);
GET /masterlink/topology (public) + POST /masterlink/promote (JWT, audited
to p2p-audit.log)
* fix(p2p): master-link request_promotion records winner on loss + strengthen test (#774)
* docs(p2p): document DHT/federation/master-link evolutions (#774)
Add an "Evolutions (#774)" section to packages/secubox-p2p/README.md
covering the DHT (api/dht.py), federation health-checks (api/federation.py)
and hierarchical master-link (api/masterlink.py) subsystems: what each does,
their API endpoints, the real p2p.toml config defaults from api/mesh.py,
and the audit log. All three remain OFF by default / OPAD opt-in. Docs only,
no behavior change. Append a matching dated entry to .claude/HISTORY.md.
* fix(p2p): final-review fixes — audit-log path, master-link peers_fn, wg_pubkey, handle_message hardening (#774)
* fix(p2p): implement mesh visualization tab — load on activation + master-link roles (#774)
The Mesh tab canvas was never rendered (the tab-switch handler didn't call
initMesh, and drawing into a hidden tab gave clientWidth 0 -> blank canvas).
Now: initMesh fires when the Mesh tab is activated, retries if the canvas has
no layout yet, and enriches the star graph with the live master-link state
(role/term/master + DHT peer count from /masterlink/topology + /dht/peers) —
center node coloured gold=MASTER / cyan=SATELLITE, peers by online status.
* docs(p2p): poster GPT + roadmap des évolutions DHT/federation/master-link; maj WIP/HISTORY/TODO (ref #774)
---------
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
Co-authored-by: Mistral Vibe <vibe@mistral.ai>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
sbxmitm classifies a non-validating upstream cert (expired/self-signed/
hostname-mismatch/third-party) without ever hard-failing, keeps the site
accessible under the R3 CA, injects a per-site dismissible identity-warning
banner (keyed on cert fingerprint, re-alerts if it changes), logs a
cert_anomaly SOC event, and notifies only on the third-party (active
impersonation) class. Replaces the current 502 on upstream cert failure.
First slice of gondwana P1: make the annuaire the distributed directory.
Adds Op.NODE_PUBLISH/CONFIG_PUBLISH/CONFIG_REVOKE and the NodeRecord (signed
mesh peer registry, public wg key only) + ConfigBlob (signed, versioned,
single-writer config distribution) models — modelled on ServiceOffer.
Carries the consolidated clone/distribute/emancipate roadmap doc.
TDD: tests/test_directory.py (9 tests); full annuaire suite 136 green.
The decentralized trust directory hinging MirrorNet (below) <-> Gondwana
(above): two mirrored faces + the third (audited reconciliation is what is
authoritative). Reuses did:plc self-certifying identity (secubox-identity),
extends the live master-link flow with four signed-log verbs (AUTO-ADD /
INVITE / PROPOSAL / EMANCIPATE), a new BLAKE2b-chained SQLite-WAL audit log +
can() resolver, GK·HAM NIZK non-revoked-membership proofs, coordinate-free
jurisdiction tags + PSI. Honest substrate: an append-only LOG, not a
blockchain; staged monotone founder-emancipation; every ungranted guarantee
paired with its detection mechanism. Pydantic v2 model + FastAPI/SQLite-WAL
reference code (syntax-checked). Fulfils gondwana spec §8 distributed directory.
Fold two design refinements into the roadmap without expanding Phase 1:
- Rendezvous is a role any node can hold (gk2 active now, not hardwired);
each node carries a DDNS name <boxname>.secubox.in as part of identity;
registry is local-first/replicable. Phase 1 = forward-compat only;
availability-based floating failover is Phase 4.
- Cross-cutting distributed directory: shared state (peers/services/threats/
names) migrates to a replicated DNS-structured append-only ledger
(did:plc / Chain of Hamiltonians) underpinning Phases 2-4.
- Phase 4: per-node auto-registration -> <service>.<boxname>.secubox.in
vhosts, DNS to gk2 public IP, Host-routed over wg-mesh.
Satellite-facing endpoint is free-form host:port — accepts a DDNS hostname
(WireGuard re-resolves per handshake, survives WAN IP changes) or a literal
IP. Current deployment pins 82.67.100.75:51822; DDNS is a later one-line
config swap.
Multi-site WireGuard substrate + node identity for the gondwana mesh:
adopt secubox-p2p as the single mesh owner, fix the 10.100.0.0/24 ->
10.10.0.0/24 subnet collision with br-lxc, gk2 = public rendezvous (UDP
51822), hub-and-spoke routing, master-assigned addressing, persistent
per-node identity for the Phase 2 ZKP layer. Zero-disruption cutover via
private-key adoption. Phases 2-4 (ZKP enrollment, signed protection
sharing, service mirroring) scoped out for their own cycles.
