secubox/secubox-deb
1
0
Fork 0
mirror of https://github.com/CyberMind-FR/secubox-deb.git synced 2026-07-29 15:37:03 +00:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
3,219 Commits 381 Branches 117 Tags 280 MiB
940aa64fcf
Commit Graph

3 Commits

Author SHA1 Message Date
CyberMind
90a323e6e8
secubox-p2p: Kademlia DHT + federation health-checks + hierarchical master-link (#774) (#775)
Some checks are pending
License Headers / check (push) Waiting to run
Details 
* feat(p2p): DHT, Federation, MasterLink evolutions for P2P-EVO-2026-07-001

Implements three major features for secubox-p2p:

1. DHT (Distributed Hash Table) based on Kademlia algorithm
   - Peer discovery across different subnets
   - Resilience to temporary node failures
   - Scalability for large number of nodes
   - REST API endpoints for DHT management

2. Service Federation
   - Service registration and discovery
   - Multi-version service management
   - Automatic health checks
   - DHT integration for service propagation

3. Master-Link Hierarchical Topology
   - Master/satellite/leaf hierarchical structure
   - Automatic master election and failover
   - Heartbeat monitoring
   - Routing policy management
   - OPAD security integration

New files:
- api/dht.py: Kademlia DHT implementation
- api/federation.py: Service federation system
- api/masterlink.py: Hierarchical topology manager
- api/main_evolutions.py: Integrated FastAPI server
- tests/test_dht.py: Unit tests for DHT

Issue: P2P-EVO-2026-07-001
Worktree: secubox-p2p-evolutions

Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>

* revert(p2p): remove non-integrating DHT/federation/masterlink code, keep requirements

The DHT/federation/masterlink modules + main_evolutions.py + test_dht.py did not
integrate with the module: tests imported 'secubox.p2p.api.*' (the package uses
'from api import ...' via conftest sys.path) so nothing even collected, and the
modules reinvented standalone aiohttp servers instead of building on the existing
WireGuard mesh (mesh.py) + annuaire-backed registry (registry.py/annuaire_client.py).
Removing them to restart on the real base. The requirements doc
(.github/ISSUES/2026-07-P2P-EVOLUTIONS.md) is kept as the spec source.

* docs(p2p): point the local evolutions note to real issue #774

The '.github/ISSUES/2026-07-P2P-EVOLUTIONS.md' was a local file with a fabricated
id, not a tracked issue. Replace its contents with a pointer to the real GitHub
issue #774 (Kademlia DHT + hierarchical master-link + federation health-checks),
framed against the already-done federation (#766) and directory (#768) work and
the master-link/auto-enrollment issue (#762).

* docs(spec): p2p Kademlia DHT + master-link + federation health-checks (#774)

Clean redesign on the real base (mesh.py + registry/annuaire_client) after
reverting the non-integrating attempt. Custom minimal Kademlia (pure asyncio
UDP, JSON wire, signed reachability records), federation health-checks layered
over the annuaire federation (#766) publishing status via the DHT, and a
deterministic-election master-link (term-based failover, aligns #762). All
feature-flagged off by default (OPAD opt-in), from api import convention, TDD
per module. Refs #774.

* docs(plan): p2p DHT/federation/master-link TDD implementation plan (#774)

17 bite-sized TDD tasks across 3 phases (DHT 1-9, federation health-checks
10-13, master-link 14-16, finalization 17). Each task = failing test -> minimal
impl -> pass -> commit, on the real base (from api import ...). Crypto (Ed25519
sign/verify) and UDP transport are module-level seams injected in unit tests so
the Kademlia primitives, health debounce, and election/term logic are tested
without real sockets. Refs #774.

* feat(p2p): DHT scaffold — node id + xor distance (#774)

* feat(p2p): DHT k-bucket with LRU (#774)

Add DHTNode contact + DHTBucket (k-bucket with LRU ordering).

- DHTNode: dataclass with node_id, did, endpoint, last_seen
- DHTBucket: Kademlia k-bucket with OrderedDict LRU management
  - add(node) → bool (True if stored/refreshed, False if full)
  - refresh moves node to tail (most-recent)
  - oldest() returns head node
  - remove(node_id) and nodes property for list access

All Task 1 tests + new Task 2 tests passing.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(p2p): DHT routing table + closest-N (#774)

* feat(p2p): DHT signed reachability records + verify (#774)

Implement canonical_record, sign_record, verify_record functions with
crypto SEAMS (_did_from_pubkey, _verify_sig, _sign_sig) for testing.
Tests use monkeypatching; real crypto wired in Task 8.

Canonical records are deterministic sorted JSON. verify_record checks
for sig field, DID validity via wg_pubkey, and cryptographic signature.

* feat(p2p): DHT JSON UDP codec + hardening (#774)

* feat(p2p): DHTNetwork store+dispatch (transport-injected) (#774)

* feat(p2p): DHT iterative find_node/find_value + announce (#774)

Add asyncio-based iterative Kademlia lookup on top of the injected
DHTNetwork transport from Task 6:

- pending: dict[rpc_id, Future] on DHTNetwork, resolved by handle_message
  when a reply (pong/nodes/value/ok) arrives with a matching rpc_id.
- _rpc(node, msg): send + await the matching future with RPC_TIMEOUT,
  returns None on timeout.
- iterative_find(target_id, mode): alpha-parallel iterative lookup,
  dedups queried nodes, merges discovered contacts into the shortlist
  (dedup by node_id, skip self), terminates on no-improvement-in-a-round
  or exhausted candidates; mode="value" short-circuits on first verified
  record.
- find_peer(did) / announce(): DID resolution and self-record
  publication (sign, cache locally, push to the k closest nodes found).

Tests (tests/test_dht.py, +2 async, pytest-asyncio strict mode):
- in-process UDP router delivering via call_soon (never inline) so
  awaited RPC futures resolve on a later loop turn;
- find_peer resolves a record announced by a node reachable only
  through a bootstrap intermediary;
- dedup: a node introduced twice into the shortlist (via two different
  repliers) is queried exactly once.

All 16 tests green (14 prior + 2 new).

* fix(p2p): harden DHT iterative lookup — skip malformed contacts, cap shortlist, tolerate rpc exceptions (#774)

* feat(p2p): DHT record schema (id+wg keys) + real Ed25519 sign/verify (#774)

* feat(p2p): DHT UDP transport + routing persistence + bootstrap (#774)

* feat(p2p): mount DHT — endpoints + startup guard + [dht] config (#774)

Task 9: wire the finished Kademlia DHT (api/dht.py) into the FastAPI
app as a feature-flagged background service, fully backward compatible.

- api/mesh.py: load_p2p_config() now also returns a `dht` sub-dict
  (enabled/port/bootstrap/announce/announce_interval/rps) built from an
  optional [dht] toml section, with defaults when absent.
- api/main.py: @app.on_event("startup") _dht_startup creates+binds a
  DHTNetwork only when [dht].enabled is true; any failure (missing
  identity, bind error, ...) is caught and leaves app.state.dht = None
  so the app always starts. _dht_shutdown persists the routing table
  and closes the UDP transport.
- New endpoints (bare paths, nginx adds /api/v1/p2p prefix): GET
  /dht/peers (routing snapshot), POST /dht/announce (JWT-protected,
  audit-logged to /var/log/secubox/p2p-audit.log), GET /dht/find/{did}.

Tests: tests/test_dht_wiring.py (config defaults/overrides + disabled
TestClient smoke). All 22 existing DHT tests + 3 new tests green (74
total in the package).

* feat(p2p): federation HealthStore + debounce (#774)

* feat(p2p): federation HealthChecker sweep (#774)

* feat(p2p): federation real probe + registry services (#774)

* fix(p2p): bound TCP-probe close, declare aiohttp dep, guard non-dict service (#774)

* feat(p2p): federation health publish-via-DHT + endpoints + [federation] config (#774)

* fix(p2p): wire federation probe_timeout + harden sweep/health-TTL tests (#774)

* feat(p2p): master-link elect() + term store (#774)

* feat(p2p): master-link state machine + failover (#774)

* fix(p2p): master-link equal-term tie-break (no zero-master window) + defensive peers (#774)

* feat(p2p): master-link signed heartbeats + UDP transport + tick loop (#774)

* feat(p2p): mount master-link — endpoints + startup guard + [masterlink] config (#774)

- masterlink.MasterLink.request_promotion(): on-demand promotion attempt
  (bump term, run election, become MASTER + heartbeat on win, else CANDIDATE)
- mesh.load_p2p_config(): add [masterlink] section defaults/overrides
  (enabled, role_preference, priority, heartbeat_interval, election_timeout,
  port, peer_addrs) alongside existing [dht]/[federation]
- main.py: _masterlink_startup/_masterlink_shutdown mirror the DHT/federation
  defensive startup pattern (feature-flagged, never breaks app startup);
  GET /masterlink/topology (public) + POST /masterlink/promote (JWT, audited
  to p2p-audit.log)

* fix(p2p): master-link request_promotion records winner on loss + strengthen test (#774)

* docs(p2p): document DHT/federation/master-link evolutions (#774)

Add an "Evolutions (#774)" section to packages/secubox-p2p/README.md
covering the DHT (api/dht.py), federation health-checks (api/federation.py)
and hierarchical master-link (api/masterlink.py) subsystems: what each does,
their API endpoints, the real p2p.toml config defaults from api/mesh.py,
and the audit log. All three remain OFF by default / OPAD opt-in. Docs only,
no behavior change. Append a matching dated entry to .claude/HISTORY.md.

* fix(p2p): final-review fixes — audit-log path, master-link peers_fn, wg_pubkey, handle_message hardening (#774)

* fix(p2p): implement mesh visualization tab — load on activation + master-link roles (#774)

The Mesh tab canvas was never rendered (the tab-switch handler didn't call
initMesh, and drawing into a hidden tab gave clientWidth 0 -> blank canvas).
Now: initMesh fires when the Mesh tab is activated, retries if the canvas has
no layout yet, and enriches the star graph with the live master-link state
(role/term/master + DHT peer count from /masterlink/topology + /dht/peers) —
center node coloured gold=MASTER / cyan=SATELLITE, peers by online status.

* docs(p2p): poster GPT + roadmap des évolutions DHT/federation/master-link; maj WIP/HISTORY/TODO (ref #774)

---------

Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
Co-authored-by: Mistral Vibe <vibe@mistral.ai>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-03 06:59:35 +02:00
CyberMind-FR
4b4dfc9b5f fix(auth): consolidate sessions/audit file constants, harden _DATA_DIR.mkdir, fix pytest.ini footgun (ref #120) 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-05-13 09:25:08 +02:00
CyberMind-FR
a8abe9737b fix(auth): startup() no longer clobbers session callback; document per-dir test invocation (ref #120) 
- Remove set_session_callback(_handle_session_event) from startup() — it was
  overwriting the Task 13 _on_session_event registration (which uses jwt.jti)
  with the legacy handler (which used secrets.token_hex(8)), breaking jti-keyed
  session lookup in _session_validator.
- Fix comment on module-load set_session_callback registration (line 167).
- Add pytest.ini at worktree root: documents the api/ vs api/ namespace collision
  that prevents combined cross-dir collection; enforces per-directory test runs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-05-13 09:18:47 +02:00