mirror of
https://github.com/CyberMind-FR/secubox-deb.git
synced 2026-07-29 12:34:38 +00:00
90a323e6e8
Some checks are pending
License Headers / check (push) Waiting to run
* feat(p2p): DHT, Federation, MasterLink evolutions for P2P-EVO-2026-07-001 Implements three major features for secubox-p2p: 1. DHT (Distributed Hash Table) based on Kademlia algorithm - Peer discovery across different subnets - Resilience to temporary node failures - Scalability for large number of nodes - REST API endpoints for DHT management 2. Service Federation - Service registration and discovery - Multi-version service management - Automatic health checks - DHT integration for service propagation 3. Master-Link Hierarchical Topology - Master/satellite/leaf hierarchical structure - Automatic master election and failover - Heartbeat monitoring - Routing policy management - OPAD security integration New files: - api/dht.py: Kademlia DHT implementation - api/federation.py: Service federation system - api/masterlink.py: Hierarchical topology manager - api/main_evolutions.py: Integrated FastAPI server - tests/test_dht.py: Unit tests for DHT Issue: P2P-EVO-2026-07-001 Worktree: secubox-p2p-evolutions Generated by Mistral Vibe. Co-Authored-By: Mistral Vibe <vibe@mistral.ai> * revert(p2p): remove non-integrating DHT/federation/masterlink code, keep requirements The DHT/federation/masterlink modules + main_evolutions.py + test_dht.py did not integrate with the module: tests imported 'secubox.p2p.api.*' (the package uses 'from api import ...' via conftest sys.path) so nothing even collected, and the modules reinvented standalone aiohttp servers instead of building on the existing WireGuard mesh (mesh.py) + annuaire-backed registry (registry.py/annuaire_client.py). Removing them to restart on the real base. The requirements doc (.github/ISSUES/2026-07-P2P-EVOLUTIONS.md) is kept as the spec source. * docs(p2p): point the local evolutions note to real issue #774 The '.github/ISSUES/2026-07-P2P-EVOLUTIONS.md' was a local file with a fabricated id, not a tracked issue. Replace its contents with a pointer to the real GitHub issue #774 (Kademlia DHT + hierarchical master-link + federation health-checks), framed against the already-done federation (#766) and directory (#768) work and the master-link/auto-enrollment issue (#762). * docs(spec): p2p Kademlia DHT + master-link + federation health-checks (#774) Clean redesign on the real base (mesh.py + registry/annuaire_client) after reverting the non-integrating attempt. Custom minimal Kademlia (pure asyncio UDP, JSON wire, signed reachability records), federation health-checks layered over the annuaire federation (#766) publishing status via the DHT, and a deterministic-election master-link (term-based failover, aligns #762). All feature-flagged off by default (OPAD opt-in), from api import convention, TDD per module. Refs #774. * docs(plan): p2p DHT/federation/master-link TDD implementation plan (#774) 17 bite-sized TDD tasks across 3 phases (DHT 1-9, federation health-checks 10-13, master-link 14-16, finalization 17). Each task = failing test -> minimal impl -> pass -> commit, on the real base (from api import ...). Crypto (Ed25519 sign/verify) and UDP transport are module-level seams injected in unit tests so the Kademlia primitives, health debounce, and election/term logic are tested without real sockets. Refs #774. * feat(p2p): DHT scaffold — node id + xor distance (#774) * feat(p2p): DHT k-bucket with LRU (#774) Add DHTNode contact + DHTBucket (k-bucket with LRU ordering). - DHTNode: dataclass with node_id, did, endpoint, last_seen - DHTBucket: Kademlia k-bucket with OrderedDict LRU management - add(node) → bool (True if stored/refreshed, False if full) - refresh moves node to tail (most-recent) - oldest() returns head node - remove(node_id) and nodes property for list access All Task 1 tests + new Task 2 tests passing. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(p2p): DHT routing table + closest-N (#774) * feat(p2p): DHT signed reachability records + verify (#774) Implement canonical_record, sign_record, verify_record functions with crypto SEAMS (_did_from_pubkey, _verify_sig, _sign_sig) for testing. Tests use monkeypatching; real crypto wired in Task 8. Canonical records are deterministic sorted JSON. verify_record checks for sig field, DID validity via wg_pubkey, and cryptographic signature. * feat(p2p): DHT JSON UDP codec + hardening (#774) * feat(p2p): DHTNetwork store+dispatch (transport-injected) (#774) * feat(p2p): DHT iterative find_node/find_value + announce (#774) Add asyncio-based iterative Kademlia lookup on top of the injected DHTNetwork transport from Task 6: - pending: dict[rpc_id, Future] on DHTNetwork, resolved by handle_message when a reply (pong/nodes/value/ok) arrives with a matching rpc_id. - _rpc(node, msg): send + await the matching future with RPC_TIMEOUT, returns None on timeout. - iterative_find(target_id, mode): alpha-parallel iterative lookup, dedups queried nodes, merges discovered contacts into the shortlist (dedup by node_id, skip self), terminates on no-improvement-in-a-round or exhausted candidates; mode="value" short-circuits on first verified record. - find_peer(did) / announce(): DID resolution and self-record publication (sign, cache locally, push to the k closest nodes found). Tests (tests/test_dht.py, +2 async, pytest-asyncio strict mode): - in-process UDP router delivering via call_soon (never inline) so awaited RPC futures resolve on a later loop turn; - find_peer resolves a record announced by a node reachable only through a bootstrap intermediary; - dedup: a node introduced twice into the shortlist (via two different repliers) is queried exactly once. All 16 tests green (14 prior + 2 new). * fix(p2p): harden DHT iterative lookup — skip malformed contacts, cap shortlist, tolerate rpc exceptions (#774) * feat(p2p): DHT record schema (id+wg keys) + real Ed25519 sign/verify (#774) * feat(p2p): DHT UDP transport + routing persistence + bootstrap (#774) * feat(p2p): mount DHT — endpoints + startup guard + [dht] config (#774) Task 9: wire the finished Kademlia DHT (api/dht.py) into the FastAPI app as a feature-flagged background service, fully backward compatible. - api/mesh.py: load_p2p_config() now also returns a `dht` sub-dict (enabled/port/bootstrap/announce/announce_interval/rps) built from an optional [dht] toml section, with defaults when absent. - api/main.py: @app.on_event("startup") _dht_startup creates+binds a DHTNetwork only when [dht].enabled is true; any failure (missing identity, bind error, ...) is caught and leaves app.state.dht = None so the app always starts. _dht_shutdown persists the routing table and closes the UDP transport. - New endpoints (bare paths, nginx adds /api/v1/p2p prefix): GET /dht/peers (routing snapshot), POST /dht/announce (JWT-protected, audit-logged to /var/log/secubox/p2p-audit.log), GET /dht/find/{did}. Tests: tests/test_dht_wiring.py (config defaults/overrides + disabled TestClient smoke). All 22 existing DHT tests + 3 new tests green (74 total in the package). * feat(p2p): federation HealthStore + debounce (#774) * feat(p2p): federation HealthChecker sweep (#774) * feat(p2p): federation real probe + registry services (#774) * fix(p2p): bound TCP-probe close, declare aiohttp dep, guard non-dict service (#774) * feat(p2p): federation health publish-via-DHT + endpoints + [federation] config (#774) * fix(p2p): wire federation probe_timeout + harden sweep/health-TTL tests (#774) * feat(p2p): master-link elect() + term store (#774) * feat(p2p): master-link state machine + failover (#774) * fix(p2p): master-link equal-term tie-break (no zero-master window) + defensive peers (#774) * feat(p2p): master-link signed heartbeats + UDP transport + tick loop (#774) * feat(p2p): mount master-link — endpoints + startup guard + [masterlink] config (#774) - masterlink.MasterLink.request_promotion(): on-demand promotion attempt (bump term, run election, become MASTER + heartbeat on win, else CANDIDATE) - mesh.load_p2p_config(): add [masterlink] section defaults/overrides (enabled, role_preference, priority, heartbeat_interval, election_timeout, port, peer_addrs) alongside existing [dht]/[federation] - main.py: _masterlink_startup/_masterlink_shutdown mirror the DHT/federation defensive startup pattern (feature-flagged, never breaks app startup); GET /masterlink/topology (public) + POST /masterlink/promote (JWT, audited to p2p-audit.log) * fix(p2p): master-link request_promotion records winner on loss + strengthen test (#774) * docs(p2p): document DHT/federation/master-link evolutions (#774) Add an "Evolutions (#774)" section to packages/secubox-p2p/README.md covering the DHT (api/dht.py), federation health-checks (api/federation.py) and hierarchical master-link (api/masterlink.py) subsystems: what each does, their API endpoints, the real p2p.toml config defaults from api/mesh.py, and the audit log. All three remain OFF by default / OPAD opt-in. Docs only, no behavior change. Append a matching dated entry to .claude/HISTORY.md. * fix(p2p): final-review fixes — audit-log path, master-link peers_fn, wg_pubkey, handle_message hardening (#774) * fix(p2p): implement mesh visualization tab — load on activation + master-link roles (#774) The Mesh tab canvas was never rendered (the tab-switch handler didn't call initMesh, and drawing into a hidden tab gave clientWidth 0 -> blank canvas). Now: initMesh fires when the Mesh tab is activated, retries if the canvas has no layout yet, and enriches the star graph with the live master-link state (role/term/master + DHT peer count from /masterlink/topology + /dht/peers) — center node coloured gold=MASTER / cyan=SATELLITE, peers by online status. * docs(p2p): poster GPT + roadmap des évolutions DHT/federation/master-link; maj WIP/HISTORY/TODO (ref #774) --------- Co-authored-by: CyberMind-FR <gandalf@Gk2.net> Co-authored-by: Mistral Vibe <vibe@mistral.ai> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
7.0 KiB
7.0 KiB
SecuBox P2P — Poster GPT & Roadmap (Gondwana Mesh)
Livrable de synthèse pour les évolutions secubox-p2p DHT / Federation / Master-link (#774 · PR #775 · branche
feature/p2p-dht-federation). Deux parties : (1) un prompt prêt-à-coller pour un générateur d'image GPT (poster), (2) une vue roadmap textuelle des phases livrées et à venir.
1 · Prompt poster — à coller dans GPT (image)
Copier le bloc ci-dessous tel quel. Format cible : affiche verticale A2 (portrait), haute densité, lisible imprimée.
Create a high-detail vertical A2 technical poster, cyberpunk-hermetic aesthetic,
titled "SECUBOX · GONDWANA MESH" in an engraved Cinzel serif at the top, subtitle
"Peer-to-Peer Trust Substrate — DHT · Federation · Master-Link" in JetBrains Mono.
PALETTE (strict): background cosmos-black #0a0a0f with subtle carbon texture; primary
accent hermetic-gold #c9a84c; secondary cyber-cyan #00d4ff; signal matrix-green #00ff41;
depth void-purple #6e40c9; alert cinnabar #e63946; body text warm off-white #e8e6d9,
muted labels #6b6b7a. Everything glows softly against the dark, like an alchemical
circuit board crossed with a star chart.
CENTRAL MOTIF — a triangular 3-node WireGuard mesh forming a Hamiltonian cycle (sacred
geometry nod to GK-HAM). Three glowing nodes joined by luminous encrypted tunnels:
• TOP node "gk2 · 10.10.0.1" wears a small hermetic-GOLD crown labelled "MASTER —
term 1 · prio 10". Brightest, gold halo.
• BOTTOM-LEFT node "c3box · 10.10.0.2" and BOTTOM-RIGHT node "amd64 · 10.10.0.3",
both cyber-cyan, labelled "SATELLITE — following master". Thin heartbeat lines
(matrix-green pulses) travel from satellites up to the crown.
• The three tunnels are labelled "wg-mesh · 51822 · WireGuard".
AROUND THE MESH, three annotated technical rings (like an astrolabe), each a subsystem:
1. DHT ring — a Kademlia constellation: small orbiting record cards reading
"reachability record {did, id_pubkey, wg_pubkey, endpoint, ts, sig}", a wax-seal
icon marked "Ed25519 signed", a bucket ladder, and the label "UDP :51823 ·
iterative α-parallel lookup · peers discovered = 2 per node".
2. FEDERATION ring — a health pulse/EKG line with green "UP" and cinnabar "DOWN"
beacons, a debounce spring icon, label "health-checks · aiohttp+TCP probe ·
published via DHT".
3. MASTER-LINK ring — a crown-and-scepter election glyph over a term counter dial,
label "UDP :51824 · deterministic election · term-based failover · signed
heartbeats · no split-brain".
BOTTOM THIRD — a horizontal ROADMAP TIMELINE band on a faint void-purple rail, left to
right, four milestones as illuminated waypoints:
● "SHIPPED — DHT + Federation + Master-Link · LIVE on 3-node mesh" (gold, checkmark)
● "NEXT — Mesh bans → sbxwaf engine bridge" (cyan)
● "NEXT — macroctl on satellites (privilege path)" (cyan)
● "HORIZON — Mesh phases 2–4 · NIZK GK-HAM binding · new macro kinds" (void-purple, dashed)
FOOTER strip in JetBrains Mono: "17 tasks · 132 tests · subagent-driven TDD · #774 /
PR #775 — CyberMind · secubox.in". A small "OPAD — off by default, opt-in" seal in a
corner.
STYLE: crisp vector-meets-engraving, thin glowing lines, alchemical marginalia and
circuit traces, faint constellation grid in the background, no photographic elements,
no people. Balanced, symmetrical, poster-grade typography. Ultra sharp, print-ready.
Variante courte (si le générateur tronque) : garder le titre, la palette, le motif central 3-nœuds avec la couronne sur gk2, et la bande roadmap 4 jalons ; retirer le détail des trois anneaux.
2 · Vue Roadmap — P2P Gondwana
Légende : ✅ livré & live · 🔜 prochain · 🌀 horizon (conçu, non construit)
✅ SHIPPED — Substrat DHT / Federation / Master-Link (#774 · PR #775)
| Sous-système | Transport | État live |
|---|---|---|
| Kademlia DHT | UDP :51823, JSON, records {did,id_pubkey,wg_pubkey,endpoint,ts,sig} Ed25519 |
✅ 3 nœuds, chacun découvre les 2 autres (peers=2) |
| Federation health-checks | aiohttp GET /health + fallback TCP, debounce up/down, publié via DHT |
✅ sweep actif sur les 3 nœuds |
| Master-link hiérarchique | UDP :51824, élection déterministe + failover par term + heartbeats signés |
✅ gk2 master (term 1, prio 10), pas de split-brain |
| Activation | /etc/secubox/p2p.toml [dht]/[federation]/[masterlink], OPAD off-by-default |
✅ enabled=true sur gk2/c3box/amd64 |
| nginx endpoint gk2 | route /api/v1/p2p/ → p2p.sock (standalone qui porte les daemons) |
✅ /dht/peers reflète le vrai état |
| nft reboot-persist | allow wg-mesh udp {51823,51824} dans /etc/nftables.conf |
✅ c3box + amd64 (gk2 = 10/8 large) |
| Mesh viz UI | onglet Mesh du dashboard p2p (canvas, rôle/term/DHT peers) | ✅ déployé 3 box |
| Auth login-bounce fix | correctif du rebond de login | ✅ déployé 3 box |
| Qualité | 17 tâches, 132 tests, SDD subagent-driven + revue adversariale | ✅ |
🔜 NEXT — court terme
- 🔜 Pont bans mesh → moteur sbxwaf — aujourd'hui les bans fédérés (threatmesh #768)
s'appliquent au niveau nft (
inet secubox_meshban) uniquement. Les faire alimenter le moteur sbxwaf (bouncer CrowdSec) :cscli decisions add --ip X -R "secubox-mesh" -d 4h→ LAPI → événement WAF. Anti-boucle : filtre par reason (secubox-mesh) danssecubox-threatmesh-bridgepour ne pas re-fédérer une décision déjà reçue. - 🔜 macroctl sur satellites — l'unité
secubox-p2pstandalone tourne avecNoNewPrivileges=yes⇒sudo macroctl activaterefusé (« NNP flag is set »). Sur gk2 ça marche car p2p tourne dans l'aggregator (NNP=no). Fixer proprement le chemin privilégié côté satellites sans affaiblir le durcissement (drop-in ciblé / helper setuid vetté). - 🔜 Fenêtre transitoire du socket p2p — pendant un restart de
secubox-p2p, le webui satellite renvoie 502/504 le temps quep2p.socksoit recréé. Lisser (socket-wait /RuntimeDirectoryPreserve) pour éviter les erreursapiGetvisibles à l'écran.
🌀 HORIZON — conçu, non construit
- 🌀 Mesh phases 2–4 (voir
project_mesh_gk2_c3box) — au-delà du full-mesh WireGuard Phase 1 : orchestration, résilience multi-master régionale, exposition contrôlée. - 🌀 Liaison NIZK / PSI GK·HAM — remplacer les stubs
ZKP-HAM-v1par le vraizkp-hamiltonian(cffi) dans les verbes annuaire/p2p. - 🌀 Nouveaux kinds macro —
wg-relay,dns-resolver,http-mirror(chaque kind = pluginmacros.d/<kind>vetté + profil AppArmor, même framework quetor-exit). - 🌀 Macros en mode
pending— fédération cross-nœud des Subscription/APPROVE. - 🌀 Mesh sens master→satellite — pull satellite→master OK ; master→satellite bloqué (nft c3box) ; + Freebox forward UDP 51822 pour le remote.
CyberMind · Gérald Kerma · https://secubox.in — #774 / PR #775