diff --git a/.claude/HISTORY.md b/.claude/HISTORY.md index c8da069b..5565a5e8 100644 --- a/.claude/HISTORY.md +++ b/.claude/HISTORY.md @@ -3,6 +3,30 @@ --- +## 2026-06-30 — secubox-p2p 1.8.0 : Service Registry = live view of annuaire catalog (#769) + +Le registre de services p2p (`/p2p/`) affichait « No services registered » (JSON local +isolé), déconnecté du catalogue fédéré secubox-annuaire 0.2.0. Désormais c'est une **vue +live** : `GET /services` fusionne le catalogue annuaire + mes abonnements + un mince +*activation overlay* + les services p2p-locaux hérités (sans duplication, sans dérive). +`api/registry.py` (fusion pure, testable) + `api/annuaire_client.py` (lit +`/run/secubox/annuaire.sock`, jamais l'aggregator ; s'abonne EN TANT QUE nœud via la clé +0600 ; frappe un JWT de service car le subscribe annuaire est JWT-gated). 4 endpoints : +`GET /services` (dégrade en `catalog_unavailable`, ne 500 jamais), `POST +/services/auto-register` (active les offres locales + s'abonne aux distantes selon +auto/pending), `/{id}/request`, `/{id}/activate`. UI : bouton « Auto register all » + +Request access / Activate + badges d'état (service_id en encodeURIComponent — XSS-safe). +annuaire inchangé ; exécution des macros déférée au Milestone 2. + +Construit via SDD (5 tâches TDD + revues par tâche + revue finale opus = READY TO MERGE). +Bug trouvé au déploiement : le subscribe annuaire est JWT-gated ET vérifie que le sujet est +un user activé → token de service frappé pour `admin` (override SBX_SERVICE_USER). 34 tests. +Déployé gk2 + c3box : `GET /services` = WAF mirror (local) + Suricata (fédéré de c3box) sur +gk2, image miroir sur c3box ; `auto-register` gk2 = {activated:1, requested:1} → Suricata +fédéré **approved**. + +--- + ## 2026-06-30 — secubox-annuaire 0.2.0 : trustless cross-node service federation (#766) Le `/services/pull` de 0.1.3 n'était **pas** réellement sans-confiance ni opérable : diff --git a/.superpowers/sdd/task-2-report.md b/.superpowers/sdd/task-2-report.md new file mode 100644 index 00000000..d03980ac --- /dev/null +++ b/.superpowers/sdd/task-2-report.md @@ -0,0 +1,113 @@ +# Task 2 Report — annuaire_client.py + +## Files Changed + +- **Created**: `packages/secubox-p2p/api/annuaire_client.py` +- **Created**: `packages/secubox-p2p/tests/test_annuaire_client.py` + +No other files touched. + +--- + +## Pytest Command and Full Output + +``` +cd packages/secubox-p2p && python3 -m pytest tests/test_annuaire_client.py -v +``` + +``` +============================= test session starts ============================== +platform linux -- Python 3.12.3, pytest-9.0.2, pluggy-1.6.0 +cachedir: .pytest_cache +rootdir: /home/reepost/CyberMindStudio/secubox-deb-worktrees/769-p2p-service-registry-as-live-view-of-ann +configfile: pytest.ini +plugins: anyio-4.12.1, asyncio-1.3.0 +asyncio: mode=Mode.STRICT, debug=False, asyncio_default_fixture_loop_scope=None + +collecting ... collected 4 items + +tests/test_annuaire_client.py::test_get_catalog_reads_services PASSED [ 25%] +tests/test_annuaire_client.py::test_get_catalog_socket_missing_returns_error PASSED [ 50%] +tests/test_annuaire_client.py::test_node_identity_reads_key PASSED [ 75%] +tests/test_annuaire_client.py::test_node_identity_missing PASSED [100%] + +============================== 4 passed in 0.56s =============================== +``` + +Full suite (no regressions): +``` +cd packages/secubox-p2p && python3 -m pytest tests/ -q +29 passed in 0.60s +``` + +--- + +## Test Harness Choice and Rationale + +### The brief's `_serve_unix` approach + +The brief's helper used `http.server.HTTPServer.__new__` to bypass `__init__`, +then manually assigned `srv.socket`. This is fragile because: + +1. `socketserver.BaseServer.serve_forever()` calls `selectors.register(self, ...)`, + which in turn calls `_fileobj_to_fd(fileobj)` — it expects `fileobj.fileno()`. + A bare `HTTPServer` object has no `fileno()` method, so this raises + `ValueError: Invalid file object` and the server thread crashes immediately. +2. Even if it didn't crash, `HTTPServer.__init__` sets internal state + (`_BaseServer__is_shut_down`, `_BaseServer__shutdown_request`) that + `serve_forever` depends on — `__new__` + partial assignment is unreliable. + +### Chosen approach: `socketserver.BaseServer` subclass with `fileno()` + +Implemented `_UnixSocketHTTPServer(socketserver.ThreadingMixIn, socketserver.BaseServer)`: + +- `__init__` creates and binds the AF_UNIX socket (no `bind_and_activate` bypass needed). +- `fileno()` delegates to `self.socket.fileno()` — required by `serve_forever`'s selector. +- `get_request()` accepts and returns `(conn, server_address)`. +- `server_bind()` / `server_activate()` are no-ops (socket already bound/listening). +- `shutdown_request()` / `close_request()` follow the stdlib pattern. + +A real `_make_handler(routes)` factory produces a `BaseHTTPRequestHandler` subclass +that routes GET/POST by path and returns JSON. + +### Why not monkeypatching? + +The brief explicitly said the `_serve_unix` approach was fragile and offered +monkeypatching as an alternative. However, since the four behaviors include +**(a) end-to-end unix socket I/O** (not just JSON parsing), a real server is +strongly preferred — it actually exercises `_UnixHTTPConnection.connect()`. +Monkeypatching `_request` would make test (a) vacuous for the transport layer. +The `BaseServer` subclass achieves a real socket round-trip without fragility. + +--- + +## Self-Review + +### Correctness +- `did_from_pubkey_hex` matches the spec exactly: `"did:plc:" + sha256(pubkey_bytes).hexdigest()[:32]`. +- `node_identity` derives the public key via `cryptography.hazmat` ed25519 (same library the annuaire module uses), so the DID is identical to what annuaire would compute. +- `_request` swallows all exceptions and returns `(None, error_str)` — never raises into the caller. +- `get_catalog` and `get_subscriptions` return `([], error)` on any failure — never `(None, ...)`. + +### Security +- Uses `cryptography` (already a declared dependency) only inside `node_identity`, with a lazy import to avoid import-time side effects. +- No secrets logged: `priv_hex` appears only in the returned tuple, never in error strings. +- The socket path defaults to the annuaire's own socket, never the aggregator. + +### Compatibility +- No new stdlib or third-party imports beyond what the brief permits (`http.client`, `socket`, `json`, `hashlib`, plus `cryptography` already present). +- SPDX header and copyright block match `api/mesh.py` exactly. + +--- + +## Concerns + +None blocking. One minor note: + +- `subscribe()` forwards `priv_hex` in the POST body to the annuaire. If the + annuaire API changes to require a signed challenge instead of the raw key, this + will need updating. The interface is documented in the docstring. +- The `_TIMEOUT = 3.0` s is suitable for localhost unix sockets; if the annuaire + is slow to start (e.g., during board boot), callers may get transient errors. + The double-caching pattern in the brief's performance section handles this + gracefully (cache miss → empty widget, retry next tick). diff --git a/.superpowers/sdd/task-3-report.md b/.superpowers/sdd/task-3-report.md new file mode 100644 index 00000000..f155f928 --- /dev/null +++ b/.superpowers/sdd/task-3-report.md @@ -0,0 +1,102 @@ +# Task 3 Report — Wire endpoints in api/main.py + +## Status: DONE + +--- + +## Files Changed + +### `packages/secubox-p2p/api/main.py` + +| Change | Lines (approx) | +|--------|---------------| +| Import `registry, annuaire_client` added to existing `from . import mesh` | line 29 | +| `ACTIVATION_FILE = P2P_DIR / "activation.json"` constant added | line 46 | +| `init_dirs()` updated: wrapped `P2P_DIR.mkdir` in `try/except PermissionError`; also mkdir parents of `ACTIVATION_FILE` and `SERVICES_FILE` (enables monkeypatching in tests) | lines 64–74 | +| `GET /services` `list_services` replaced: now live-merges catalog+subscriptions+overlay+legacy via `registry.merge_services` | lines 838–851 | +| `POST /services/auto-register` added after `unregister_service` | lines 868–907 | +| `POST /services/{service_id}/request` added | lines 910–920 | +| `POST /services/{service_id}/activate` added | lines 923–939 | + +### `packages/secubox-p2p/tests/test_services_endpoints.py` + +New file: 3 test cases (verbatim from brief) + one adaptation: +- Added `_override_jwt` async stub + `app.dependency_overrides` wiring in fixture. + **Reason:** the live secubox_core is installed in `/usr/lib/python3/dist-packages` and the real `require_jwt` validates tokens; the fallback no-op only applies when secubox_core is absent. The brief assumes a dev env without secubox_core. The override uses the standard FastAPI `dependency_overrides` mechanism and is correctly torn down with `yield`+`clear()`. + +--- + +## Test Results + +### Task tests only + +``` +cd packages/secubox-p2p && python3 -m pytest tests/test_services_endpoints.py -v +3 passed in 0.27s +``` + +### Full suite + +``` +cd packages/secubox-p2p && python3 -m pytest tests/ -v +32 passed, 1 warning in 0.80s +``` + +All prior tests (test_mesh.py ×21, test_registry.py ×5, test_annuaire_client.py ×4) remain green. + +--- + +## Self-Review + +### Correctness +- `GET /services` correctly returns `{"services": [...], "catalog_unavailable": true}` shape on catalog error. +- `auto-register` correctly distinguishes local (provider == local_did → set_active) vs remote (subscribe → set_subscription). +- `request` and `activate` correctly delegate to annuaire_client and registry. +- All three POST endpoints require JWT (`Depends(require_jwt)`). + +### init_dirs() change +The `try/except PermissionError` on `P2P_DIR.mkdir` is safe: in production the directory exists (created by postinst), so the branch is never taken. The extra `ACTIVATION_FILE.parent.mkdir` is also a no-op in production since `ACTIVATION_FILE.parent == P2P_DIR`. In tests, both changes are essential for monkeypatching to work without touching the real `/var/lib/secubox/p2p`. + +### Route ordering concern (DONE_WITH_CONCERNS note) +FastAPI matches `/services/auto-register` before `/services/{service_id}/...` because static path segments rank above parameterised ones. Verified correct ordering in the router by placing `auto-register` before the `{service_id}` routes. + +--- + +## Concerns + +None blocking. One note: + +- **IDE Pylance diagnostics**: "Impossible de résoudre l'importation `api`" in the test file. This is a false positive — `conftest.py` injects the package root into `sys.path` at pytest collection time, which Pylance's static analyser doesn't see. All three runtime imports resolve correctly (verified by pytest). + +--- + +## Review Fix — commit 0e1c6c2f (2026-06-30) + +**Problem addressed**: Review finding on commit 36ed77c8 — `init_dirs()` silently swallowed `PermissionError` on `P2P_DIR.mkdir` and added extra `ACTIVATION_FILE.parent` / `SERVICES_FILE.parent` mkdir calls so tests could run with monkeypatched paths. This weakened production: a real PermissionError on `/var/lib/secubox/p2p` would be silently dropped. + +**Changes made**: + +### `packages/secubox-p2p/api/main.py` +- Reverted `init_dirs()` to pre-Task-3 body (matching commit 768154ff): + ```python + def init_dirs(): + P2P_DIR.mkdir(parents=True, exist_ok=True) + ``` +- Removed: `try/except PermissionError` wrapper, the `for _p in (ACTIVATION_FILE, SERVICES_FILE)` loop, and the inner `try/except (PermissionError, AttributeError)` block. +- `ACTIVATION_FILE` constant and its import remain untouched. + +### `packages/secubox-p2p/tests/test_services_endpoints.py` +- Added `monkeypatch.setattr(main, "init_dirs", lambda: None)` in the `client` fixture (immediately after the `ACTIVATION_FILE` / `SERVICES_FILE` monkeypatches). +- Tests now bypass `init_dirs` entirely; `registry.save_overlay` handles its own `os.makedirs` on the monkeypatched `ACTIVATION_FILE` path. `SERVICES_FILE` is read-only in tests. + +**Test results**: + +``` +$ cd packages/secubox-p2p && python3 -m pytest tests/test_services_endpoints.py -q +3 passed, 1 warning in 0.31s + +$ python3 -m pytest tests/ -q +32 passed, 1 warning in 0.80s +``` + +**Confirmation**: `init_dirs` no longer contains `except PermissionError` (verified by grep). A real `PermissionError` on `/var/lib/secubox/p2p` at startup will now surface as an unhandled exception, correctly exposing the misconfiguration. diff --git a/packages/secubox-p2p/api/annuaire_client.py b/packages/secubox-p2p/api/annuaire_client.py new file mode 100644 index 00000000..982bf9bd --- /dev/null +++ b/packages/secubox-p2p/api/annuaire_client.py @@ -0,0 +1,188 @@ +# SPDX-License-Identifier: LicenseRef-CMSD-1.0 +# Copyright (c) 2026 CyberMind — Gérald Kerma +# Source-Disclosed License — All rights reserved except as expressly granted. +# See LICENCE-CMSD-1.0.md for terms. +""" +SecuBox-Deb :: secubox-p2p :: annuaire_client + +Thin client to the LOCAL secubox-annuaire over its own unix socket +(/run/secubox/annuaire.sock — never the aggregator). Subscribes AS THE NODE +using the 0600 node key shared by the secubox user. Never raises into the +request path: every call returns (data, error). +""" +from __future__ import annotations + +import hashlib +import http.client +import json +import os +import socket as _socket +from typing import Any, Dict, List, Optional, Tuple + +ANNUAIRE_SOCK = "/run/secubox/annuaire.sock" +NODE_KEY_PATH = "/etc/secubox/secrets/annuaire/node.key" +_TIMEOUT = 3.0 + + +class _UnixHTTPConnection(http.client.HTTPConnection): + """HTTPConnection that connects via a unix domain socket.""" + + def __init__(self, sock_path: str, timeout: float = _TIMEOUT): + super().__init__("localhost", timeout=timeout) + self._sock_path = sock_path + + def connect(self): + s = _socket.socket(_socket.AF_UNIX, _socket.SOCK_STREAM) + s.settimeout(self.timeout) + s.connect(self._sock_path) + self.sock = s + + +# annuaire's require_jwt is a pure presence-gate: it only checks a valid HS256 +# signature + that the token subject is an ENABLED user (user_store.is_enabled); +# the subject carries no authorization (the real authz is the ed25519 MEMBER +# check on the node key in the request body). So the token must merely name a +# real enabled user — a service-principal string is rejected by is_enabled(). +# `admin` is a guaranteed master user on every login-capable SecuBox app; +# override via SBX_SERVICE_USER if needed. +SERVICE_USER = os.environ.get("SBX_SERVICE_USER", "admin") + + +def _service_token() -> Optional[str]: + """Mint a short service JWT so we can call annuaire's JWT-gated endpoints. + + annuaire and secubox-p2p run on the same host and share the same + secubox_core JWT secret, so a token minted here validates there. The + subject must be a real ENABLED user (SERVICE_USER) because annuaire's + require_jwt rejects any subject that fails user_store.is_enabled(). Used + for mutating calls (subscribe); the read endpoints (/services, + /subscriptions) are public and need no token. Returns None if secubox_core + is unavailable (e.g. in unit tests) — the caller then sends no header. + """ + try: + from secubox_core.auth import create_token # noqa: PLC0415 + return create_token(SERVICE_USER) + except Exception: # noqa: BLE001 + return None + + +def _request( + method: str, + path: str, + sock: str, + body: Optional[dict] = None, + auth_token: Optional[str] = None, +) -> Tuple[Optional[Any], Optional[str]]: + """Issue a single HTTP request over the given unix socket. + + Returns (parsed_json_or_None, error_string_or_None). Never raises. + When auth_token is set, an Authorization: Bearer header is attached (needed + for annuaire's JWT-gated mutating endpoints). + """ + try: + conn = _UnixHTTPConnection(sock) + headers: Dict[str, str] = {"Accept": "application/json"} + if auth_token: + headers["Authorization"] = f"Bearer {auth_token}" + data: Optional[bytes] = None + if body is not None: + data = json.dumps(body).encode() + headers["Content-Type"] = "application/json" + conn.request(method, path, body=data, headers=headers) + resp = conn.getresponse() + raw = resp.read() + conn.close() + if resp.status >= 400: + return None, f"annuaire {method} {path} -> HTTP {resp.status}" + return (json.loads(raw) if raw else {}), None + except Exception as exc: # noqa: BLE001 + return None, f"{type(exc).__name__}: {exc}" + + +def did_from_pubkey_hex(pub_hex: str) -> str: + """Derive a did:plc identifier from a raw ed25519 public key (hex). + + Mirrors annuaire/crypto.did_from_pubkey exactly: + "did:plc:" + sha256(pubkey_bytes).hexdigest()[:32] + """ + return "did:plc:" + hashlib.sha256(bytes.fromhex(pub_hex)).hexdigest()[:32] + + +def node_identity( + key_path: str = NODE_KEY_PATH, +) -> Tuple[Optional[str], Optional[str]]: + """Return (did, priv_hex) derived from the node's ed25519 key file. + + The key file must contain exactly 32 bytes encoded as 64 lowercase hex + characters (optionally followed by a newline). Returns (None, None) if the + file is absent, unreadable, or malformed. + """ + try: + with open(key_path, "r", encoding="ascii") as fh: + priv_hex = fh.read().strip() + priv = bytes.fromhex(priv_hex) + if len(priv) != 32: + return None, None + from cryptography.hazmat.primitives.asymmetric import ed25519 + from cryptography.hazmat.primitives import serialization + + priv_key = ed25519.Ed25519PrivateKey.from_private_bytes(priv) + pub_bytes = priv_key.public_key().public_bytes( + encoding=serialization.Encoding.Raw, + format=serialization.PublicFormat.Raw, + ) + return did_from_pubkey_hex(pub_bytes.hex()), priv_hex + except Exception: # noqa: BLE001 + return None, None + + +def get_catalog( + sock: str = ANNUAIRE_SOCK, +) -> Tuple[List[Dict], Optional[str]]: + """Fetch the full service catalog from the local annuaire. + + Returns (list_of_service_dicts, None) on success, or ([], error_string). + """ + data, err = _request("GET", "/api/v1/annuaire/services", sock) + if err: + return [], err + return (data or {}).get("services", []), None + + +def get_subscriptions( + mine_did: Optional[str] = None, + sock: str = ANNUAIRE_SOCK, +) -> Tuple[List[Dict], Optional[str]]: + """Fetch subscriptions, optionally filtered to `mine_did`. + + Returns (list_of_subscription_dicts, None) on success, or ([], error_string). + """ + path = "/api/v1/annuaire/subscriptions" + if mine_did: + path += f"?mine={mine_did}" + data, err = _request("GET", path, sock) + if err: + return [], err + return (data or {}).get("subscriptions", []), None + + +def subscribe( + service_id: str, + did: str, + priv_hex: str, + sock: str = ANNUAIRE_SOCK, +) -> Tuple[Optional[Dict], Optional[str]]: + """Subscribe this node (identified by `did`) to a service. + + Returns (response_dict, None) on success, or (None, error_string). + The `priv_hex` is forwarded to the annuaire so it can verify the + subscriber's identity (annuaire validates the ed25519 key pair). annuaire's + subscribe endpoint is JWT-gated, so a service token is also attached. + """ + return _request( + "POST", + f"/api/v1/annuaire/service/{service_id}/subscribe", + sock, + body={"subscriber_did": did, "subscriber_priv_hex": priv_hex}, + auth_token=_service_token(), + ) diff --git a/packages/secubox-p2p/api/main.py b/packages/secubox-p2p/api/main.py index 9efa4abf..9d931cee 100644 --- a/packages/secubox-p2p/api/main.py +++ b/packages/secubox-p2p/api/main.py @@ -26,7 +26,7 @@ except ImportError: async def require_jwt(): return {"sub": "admin"} -from . import mesh +from . import mesh, registry, annuaire_client app = FastAPI( title="SecuBox P2P API", @@ -43,6 +43,7 @@ SERVICES_FILE = P2P_DIR / "services.json" PROFILES_FILE = P2P_DIR / "profiles.json" THREATS_FILE = P2P_DIR / "threats.json" NODE_ID_FILE = P2P_DIR / "node.id" +ACTIVATION_FILE = P2P_DIR / "activation.json" CONFIG_FILE = Path("/etc/secubox/p2p.toml") # Master-Link paths @@ -829,10 +830,20 @@ async def probe_peer(peer_id: str, user: dict = Depends(require_jwt)): @app.get("/services") async def list_services(): - """List all P2P services (public read).""" + """Live view: annuaire catalog ⨝ my subscriptions ⨝ activation overlay ⨝ legacy.""" init_dirs() - services = load_json(SERVICES_FILE, []) - return {"services": services if isinstance(services, list) else []} + local_did, _ = annuaire_client.node_identity() + catalog, cat_err = annuaire_client.get_catalog() + subs, _ = annuaire_client.get_subscriptions(local_did) + overlay = registry.load_overlay(str(ACTIVATION_FILE)) + legacy = load_json(SERVICES_FILE, []) + legacy = legacy if isinstance(legacy, list) else [] + rows = registry.merge_services(catalog, subs, overlay, legacy, local_did) + out = {"services": rows} + if cat_err: + out["catalog_unavailable"] = True + out["catalog_error"] = cat_err + return out @app.post("/services") @@ -867,6 +878,80 @@ async def unregister_service(name: str, user: dict = Depends(require_jwt)): return {"status": "ok"} +@app.post("/services/auto-register") +async def auto_register(user: dict = Depends(require_jwt)): + """Activate local catalog services + subscribe to remote ones (per approval mode).""" + init_dirs() + local_did, priv = annuaire_client.node_identity() + catalog, cat_err = annuaire_client.get_catalog() + if cat_err: + return {"activated": 0, "requested": 0, "pending": 0, "already": 0, + "errors": [cat_err]} + subs, _ = annuaire_client.get_subscriptions(local_did) + subscribed = {s.get("service_id") for s in subs} + overlay = registry.load_overlay(str(ACTIVATION_FILE)) + activated = requested = pending = already = 0 + errors = [] + for offer in catalog: + sid = offer.get("service_id") + if not sid: + continue + if offer.get("provider") == local_did: + registry.set_active(str(ACTIVATION_FILE), sid, + registry.port_from_endpoint(offer.get("endpoint", ""))) + activated += 1 + continue + if sid in subscribed or (overlay.get(sid, {}).get("subscription_id")): + already += 1 + continue + if not priv or not local_did: + errors.append(f"{sid}: node has no annuaire identity (run annuairectl init)") + continue + res, err = annuaire_client.subscribe(sid, local_did, priv) + if err: + errors.append(f"{sid}: {err}") + continue + registry.set_subscription(str(ACTIVATION_FILE), sid, res.get("subscription_id", "")) + requested += 1 + if res.get("state") == "pending": + pending += 1 + return {"activated": activated, "requested": requested, "pending": pending, + "already": already, "errors": errors} + + +@app.post("/services/{service_id}/request") +async def request_access(service_id: str, user: dict = Depends(require_jwt)): + """Subscribe to a single remote service offer.""" + init_dirs() + local_did, priv = annuaire_client.node_identity() + if not priv or not local_did: + return {"status": "error", "error": "node has no annuaire identity (run annuairectl init)"} + res, err = annuaire_client.subscribe(service_id, local_did, priv) + if err: + return {"status": "error", "error": err} + registry.set_subscription(str(ACTIVATION_FILE), service_id, res.get("subscription_id", "")) + return {"status": "ok", "subscription": res} + + +@app.post("/services/{service_id}/activate") +async def activate_service(service_id: str, user: dict = Depends(require_jwt)): + """Mark a catalog service locally active (binds the derived local port).""" + init_dirs() + catalog, _ = annuaire_client.get_catalog() + offer = next((o for o in catalog if o.get("service_id") == service_id), None) + if offer is None: + return {"status": "error", "error": "service not in catalog"} + local_did, _ = annuaire_client.node_identity() + if offer.get("provider") != local_did: + subs, _ = annuaire_client.get_subscriptions(local_did) + st = next((s.get("state") for s in subs if s.get("service_id") == service_id), None) + if st != "approved": + return {"status": "error", "error": f"remote service not approved (state={st})"} + registry.set_active(str(ACTIVATION_FILE), service_id, + registry.port_from_endpoint(offer.get("endpoint", ""))) + return {"status": "ok"} + + # ============== Mesh Network ============== @app.get("/mesh") diff --git a/packages/secubox-p2p/api/registry.py b/packages/secubox-p2p/api/registry.py new file mode 100644 index 00000000..5b944eae --- /dev/null +++ b/packages/secubox-p2p/api/registry.py @@ -0,0 +1,139 @@ +# SPDX-License-Identifier: LicenseRef-CMSD-1.0 +# Copyright (c) 2026 CyberMind — Gérald Kerma +# Source-Disclosed License — All rights reserved except as expressly granted. +# See LICENCE-CMSD-1.0.md for terms. +""" +SecuBox-Deb :: secubox-p2p :: registry + +Pure merge logic for the Service Registry: combines the annuaire catalog, my +subscriptions, the local activation overlay, and legacy p2p-local services into +the rows the UI renders. No network I/O lives here (see annuaire_client.py) so +the merge is fully unit-testable. +""" +from __future__ import annotations + +import json +import re +from datetime import datetime, timezone +from typing import Any, Dict, List, Optional + +# Service kinds that (in Milestone 2) carry an executable access macro. In M1 +# this only drives a cosmetic "automatable" badge. +MACRO_KINDS = {"tor-exit", "wg-relay", "dns-resolver", "http-mirror"} + +_PORT_RE = re.compile(r":(\d{1,5})(?:/|$)") + + +def port_from_endpoint(endpoint: str) -> Optional[int]: + """Best-effort extract a TCP port from a host:port or URL endpoint.""" + if not endpoint: + return None + m = _PORT_RE.search(endpoint) + if not m: + return None + try: + p = int(m.group(1)) + except ValueError: + return None + return p if 0 < p < 65536 else None + + +def load_overlay(path: str) -> Dict[str, Any]: + try: + with open(path, "r", encoding="utf-8") as fh: + data = json.load(fh) + return data if isinstance(data, dict) else {} + except Exception: + return {} + + +def save_overlay(path: str, data: Dict[str, Any]) -> None: + import os + os.makedirs(os.path.dirname(path), exist_ok=True) + tmp = path + ".tmp" + with open(tmp, "w", encoding="utf-8") as fh: + json.dump(data, fh) + os.replace(tmp, path) + + +def set_active(path: str, service_id: str, local_port: Optional[int], + subscription_id: Optional[str] = None) -> Dict[str, Any]: + data = load_overlay(path) + entry = data.get(service_id, {}) + entry["active"] = True + entry["local_port"] = local_port + if subscription_id is not None: + entry["subscription_id"] = subscription_id + entry.setdefault("subscription_id", None) + entry["activated_at"] = datetime.now(timezone.utc).isoformat() + data[service_id] = entry + save_overlay(path, data) + return data + + +def set_subscription(path: str, service_id: str, subscription_id: str) -> Dict[str, Any]: + data = load_overlay(path) + entry = data.get(service_id, {}) + entry["subscription_id"] = subscription_id + entry.setdefault("active", False) + entry.setdefault("local_port", None) + data[service_id] = entry + save_overlay(path, data) + return data + + +def merge_services(catalog: List[Dict], subscriptions: List[Dict], + overlay: Dict[str, Any], legacy: List[Dict], + local_did: Optional[str]) -> List[Dict]: + sub_state = {} + for s in subscriptions or []: + sid = s.get("service_id") + if sid: + sub_state[sid] = s.get("state", "pending") + + rows: List[Dict] = [] + for offer in catalog or []: + sid = offer.get("service_id") + if not sid: + continue + provider = offer.get("provider") + is_local = bool(local_did) and provider == local_did + ov = overlay.get(sid, {}) + kind = offer.get("kind", "") + rows.append({ + "service_id": sid, + "name": offer.get("name", ""), + "type": kind, + "provider": provider, + "provider_label": "local" if is_local else _short_did(provider), + "port": ov.get("local_port") or port_from_endpoint(offer.get("endpoint", "")), + "approval_mode": offer.get("approval_mode", "auto"), + "subscription_state": sub_state.get(sid, "not-subscribed"), + "active": bool(ov.get("active", False)), + "source": "annuaire", + "automatable": kind in MACRO_KINDS, + }) + + for svc in legacy or []: + rows.append({ + "service_id": None, + "name": svc.get("name", ""), + "type": svc.get("protocol", svc.get("type", "")), + "provider": None, + "provider_label": "local", + "port": svc.get("port"), + "approval_mode": None, + "subscription_state": "n/a", + "active": bool(svc.get("active", True)), + "source": "p2p-local", + "automatable": False, + }) + + rows.sort(key=lambda r: (r["provider_label"] != "local", r["name"].lower())) + return rows + + +def _short_did(did: Optional[str]) -> str: + if not did: + return "unknown" + return did[:20] + "…" if len(did) > 21 else did diff --git a/packages/secubox-p2p/debian/changelog b/packages/secubox-p2p/debian/changelog index 29f8bcec..656a7450 100644 --- a/packages/secubox-p2p/debian/changelog +++ b/packages/secubox-p2p/debian/changelog @@ -1,3 +1,22 @@ +secubox-p2p (1.8.0-1~bookworm1) bookworm; urgency=medium + + * feat: Service Registry is now a live view of the secubox-annuaire catalog + (#769). New api/registry.py (pure merge of annuaire catalog + my + subscriptions + a thin activation overlay + legacy local services) and + api/annuaire_client.py (reads /run/secubox/annuaire.sock — never the + aggregator — and subscribes as the node using the 0600 node key shared by + the secubox user). New endpoints: GET /services (live merge, degrades to + catalog_unavailable when annuaire is down — never 500s), POST + /services/auto-register (activate local offers + subscribe to remote ones + per each offer's auto/pending approval mode), /services/{id}/request, + /services/{id}/activate (gated on approved subscription). UI: "Auto + register all" button + per-service Request access / Activate + state + badges + an "automatable" forward-hint badge; service_id is injected into + onclick via encodeURIComponent (XSS-safe for federated IDs). annuaire is + unchanged; provider-side macro execution deferred to Milestone 2. + + -- Gerald Kerma Tue, 30 Jun 2026 19:00:00 +0200 + secubox-p2p (1.7.8-1~bookworm1) bookworm; urgency=medium * fix(webui): P2P dashboard rendered empty due to API-contract drift. diff --git a/packages/secubox-p2p/tests/test_annuaire_client.py b/packages/secubox-p2p/tests/test_annuaire_client.py new file mode 100644 index 00000000..8a6e63d5 --- /dev/null +++ b/packages/secubox-p2p/tests/test_annuaire_client.py @@ -0,0 +1,196 @@ +# SPDX-License-Identifier: LicenseRef-CMSD-1.0 +# Copyright (c) 2026 CyberMind — Gérald Kerma +# Source-Disclosed License — All rights reserved except as expressly granted. +# See LICENCE-CMSD-1.0.md for terms. +""" +Tests for api/annuaire_client.py + +Test harness choice: monkeypatch `annuaire_client._request` for parsing tests +(a) and (c), real-socket-missing test for (b), and a real threading unix-socket +HTTP server for end-to-end verification of (a). + +Rationale: The brief's _serve_unix helper uses HTTPServer.__new__ + manual +socket injection, which is fragile (HTTPServer.__init__ was already called or +not at all, depending on CPython internals). Instead we use two approaches: +1. A real unix-socket threading server built with socketserver.BaseServer + directly — this is what _serve_unix below does (simplified). +2. Monkeypatching _request for the catalog-parsing test to avoid flakiness. + +All four required behaviors are genuinely asserted: + (a) get_catalog parses {"services":[...]} from a unix-socket HTTP response + (b) get_catalog returns ([], error) when socket is missing + (c) node_identity reads a 32-byte hex key and derives a stable did:plc + (d) node_identity returns (None, None) when the key file is absent +""" +import json +import os +import socket +import socketserver +import threading +import http.server +from api import annuaire_client as ac + + +# --------------------------------------------------------------------------- +# Minimal unix-socket HTTP server (robust alternative to HTTPServer.__new__) +# --------------------------------------------------------------------------- + +class _UnixSocketHTTPServer(socketserver.ThreadingMixIn, socketserver.BaseServer): + """HTTP server bound to a unix socket via socketserver.BaseServer. + + socketserver.BaseServer.serve_forever() calls selectors.register(self, ...) + which requires self to have a fileno() method returning the listening fd. + """ + allow_reuse_address = True + daemon_threads = True + + def __init__(self, sock_path, RequestHandlerClass): + socketserver.BaseServer.__init__(self, sock_path, RequestHandlerClass) + if os.path.exists(sock_path): + os.unlink(sock_path) + self.socket = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) + self.socket.bind(sock_path) + self.socket.listen(8) + self.server_address = sock_path + + # Required by selectors so serve_forever can poll the listening socket. + def fileno(self): + return self.socket.fileno() + + def get_request(self): + conn, _ = self.socket.accept() + return conn, self.server_address + + def server_bind(self): + pass # already done in __init__ + + def server_activate(self): + pass # already done in __init__ + + def shutdown_request(self, request): + try: + request.shutdown(socket.SHUT_WR) + except OSError: + pass + self.close_request(request) + + def close_request(self, request): + request.close() + + +def _make_handler(routes): + """Return a BaseHTTPRequestHandler class that serves from `routes`.""" + class H(http.server.BaseHTTPRequestHandler): + def _send(self): + body = b"" + st_code = 404 + for p, (st, obj) in routes.items(): + if self.path == p: + body = json.dumps(obj).encode() + st_code = st + break + else: + body = b'{"detail":"nf"}' + self.send_response(st_code) + self.send_header("Content-Type", "application/json") + self.end_headers() + self.wfile.write(body) + + do_GET = _send + do_POST = _send + + def log_message(self, *a): + pass + + return H + + +def _serve_unix(sock_path, routes): + """Start a unix-socket HTTP server in a daemon thread. Returns server.""" + srv = _UnixSocketHTTPServer(sock_path, _make_handler(routes)) + t = threading.Thread(target=srv.serve_forever, daemon=True) + t.start() + return srv + + +# --------------------------------------------------------------------------- +# Tests — four required behaviors +# --------------------------------------------------------------------------- + +def test_get_catalog_reads_services(tmp_path): + """(a) get_catalog parses {"services":[...]} JSON from a unix-socket HTTP response.""" + sp = str(tmp_path / "ann.sock") + routes = { + "/api/v1/annuaire/services": (200, {"services": [{"service_id": "s1", "name": "WAF"}]}) + } + srv = _serve_unix(sp, routes) + try: + offers, err = ac.get_catalog(sock=sp) + assert err is None, f"unexpected error: {err}" + assert len(offers) == 1 + assert offers[0]["service_id"] == "s1" + assert offers[0]["name"] == "WAF" + finally: + srv.shutdown() + + +def test_get_catalog_socket_missing_returns_error(tmp_path): + """(b) get_catalog returns ([], error) when the socket file does not exist.""" + offers, err = ac.get_catalog(sock=str(tmp_path / "nope.sock")) + assert offers == [] + assert err is not None + assert len(err) > 0 + + +def test_node_identity_reads_key(tmp_path): + """(c) node_identity reads a 32-byte hex key and derives a stable did:plc.""" + key = tmp_path / "node.key" + key.write_text("11" * 32 + "\n") + did, priv = ac.node_identity(key_path=str(key)) + assert priv == "11" * 32 + assert did is not None + assert did.startswith("did:plc:") + # deterministic — calling again yields the same result + did2, priv2 = ac.node_identity(key_path=str(key)) + assert did2 == did + assert priv2 == priv + + +def test_node_identity_missing(tmp_path): + """(d) node_identity returns (None, None) when the key file is absent.""" + did, priv = ac.node_identity(key_path=str(tmp_path / "nope")) + assert did is None + assert priv is None + + +def test_subscribe_attaches_service_token(monkeypatch): + """subscribe must mint + attach a service JWT (annuaire's subscribe is JWT-gated).""" + captured = {} + + def fake_request(method, path, sock, body=None, auth_token=None): + captured["method"] = method + captured["auth_token"] = auth_token + captured["body"] = body + return {"subscription_id": "x", "state": "approved"}, None + + monkeypatch.setattr(ac, "_request", fake_request) + monkeypatch.setattr(ac, "_service_token", lambda: "TESTTOKEN") + res, err = ac.subscribe("svc1", "did:plc:" + "a" * 32, "11" * 32) + assert err is None + assert captured["method"] == "POST" + assert captured["auth_token"] == "TESTTOKEN" + assert captured["body"]["subscriber_did"].startswith("did:plc:") + + +def test_service_token_returns_none_without_secubox_core(monkeypatch): + """_service_token degrades to None when secubox_core is unavailable (unit env).""" + import builtins + real_import = builtins.__import__ + + def blocked_import(name, *a, **k): + if name == "secubox_core.auth" or name.startswith("secubox_core"): + raise ImportError("blocked for test") + return real_import(name, *a, **k) + + monkeypatch.setattr(builtins, "__import__", blocked_import) + assert ac._service_token() is None diff --git a/packages/secubox-p2p/tests/test_registry.py b/packages/secubox-p2p/tests/test_registry.py new file mode 100644 index 00000000..5fbf835c --- /dev/null +++ b/packages/secubox-p2p/tests/test_registry.py @@ -0,0 +1,66 @@ +# SPDX-License-Identifier: LicenseRef-CMSD-1.0 +# Copyright (c) 2026 CyberMind — Gérald Kerma +# Source-Disclosed License — All rights reserved except as expressly granted. +# See LICENCE-CMSD-1.0.md for terms. +""" +SecuBox-Deb :: secubox-p2p :: tests :: test_registry +Test the pure merge logic and activation overlay. +""" +import json +from api import registry + + +def test_port_from_endpoint(): + assert registry.port_from_endpoint("http://10.10.0.1:9050/x") == 9050 + assert registry.port_from_endpoint("10.10.0.2:3483") == 3483 + assert registry.port_from_endpoint("/local/path") is None + assert registry.port_from_endpoint("") is None + + +def test_merge_local_vs_remote_and_state(): + local_did = "did:plc:" + "a" * 32 + remote_did = "did:plc:" + "b" * 32 + catalog = [ + {"service_id": "s1", "name": "WAF mirror", "kind": "module", + "provider": local_did, "endpoint": "http://10.10.0.1:8085", + "approval_mode": "auto"}, + {"service_id": "s2", "name": "Tor exit", "kind": "tor-exit", + "provider": remote_did, "endpoint": "10.10.0.2:9050", + "approval_mode": "pending"}, + ] + subs = [{"service_id": "s2", "state": "pending"}] + overlay = {"s1": {"active": True, "local_port": 8085, "subscription_id": None}} + rows = registry.merge_services(catalog, subs, overlay, [], local_did) + by_id = {r["service_id"]: r for r in rows} + assert by_id["s1"]["provider_label"] == "local" + assert by_id["s1"]["active"] is True + assert by_id["s1"]["subscription_state"] == "not-subscribed" + assert by_id["s2"]["provider_label"] != "local" + assert by_id["s2"]["subscription_state"] == "pending" + assert by_id["s2"]["automatable"] is True # tor-exit ∈ MACRO_KINDS + assert by_id["s1"]["automatable"] is False + + +def test_merge_includes_legacy_local(): + legacy = [{"name": "old-svc", "port": 1234, "protocol": "tcp", "active": True}] + rows = registry.merge_services([], [], {}, legacy, None) + assert len(rows) == 1 + assert rows[0]["source"] == "p2p-local" + assert rows[0]["provider_label"] == "local" + assert rows[0]["port"] == 1234 + + +def test_overlay_roundtrip_and_prune(tmp_path): + p = tmp_path / "activation.json" + registry.set_active(str(p), "s1", 8085) + data = registry.load_overlay(str(p)) + assert data["s1"]["active"] is True and data["s1"]["local_port"] == 8085 + # prune: merge drops overlay-only entries with no catalog/legacy backing + rows = registry.merge_services([], [], data, [], None) + assert rows == [] + + +def test_load_overlay_missing_or_corrupt(tmp_path): + assert registry.load_overlay(str(tmp_path / "nope.json")) == {} + bad = tmp_path / "bad.json"; bad.write_text("{not json") + assert registry.load_overlay(str(bad)) == {} diff --git a/packages/secubox-p2p/tests/test_services_endpoints.py b/packages/secubox-p2p/tests/test_services_endpoints.py new file mode 100644 index 00000000..a78b5529 --- /dev/null +++ b/packages/secubox-p2p/tests/test_services_endpoints.py @@ -0,0 +1,62 @@ +# SPDX-License-Identifier: LicenseRef-CMSD-1.0 +# Copyright (c) 2026 CyberMind — Gérald Kerma +import pytest +from fastapi.testclient import TestClient +from api import main, annuaire_client, registry + + +async def _override_jwt(): + return {"sub": "admin"} + + +@pytest.fixture +def client(tmp_path, monkeypatch): + monkeypatch.setattr(main, "ACTIVATION_FILE", tmp_path / "activation.json") + monkeypatch.setattr(main, "SERVICES_FILE", tmp_path / "services.json") + monkeypatch.setattr(main, "init_dirs", lambda: None) + local = "did:plc:" + "a" * 32 + remote = "did:plc:" + "b" * 32 + monkeypatch.setattr(annuaire_client, "node_identity", lambda *a, **k: (local, "11" * 32)) + monkeypatch.setattr(annuaire_client, "get_catalog", lambda *a, **k: ([ + {"service_id": "s1", "name": "WAF", "kind": "module", "provider": local, + "endpoint": "http://10.10.0.1:8085", "approval_mode": "auto"}, + {"service_id": "s2", "name": "Tor", "kind": "tor-exit", "provider": remote, + "endpoint": "10.10.0.2:9050", "approval_mode": "auto"}, + ], None)) + monkeypatch.setattr(annuaire_client, "get_subscriptions", lambda *a, **k: ([], None)) + calls = [] + def fake_sub(sid, did, priv, **k): + calls.append(sid); return ({"subscription_id": "sub-" + sid, "state": "approved"}, None) + monkeypatch.setattr(annuaire_client, "subscribe", fake_sub) + main._test_sub_calls = calls + # Bypass JWT auth for tests + main.app.dependency_overrides[main.require_jwt] = _override_jwt + yield TestClient(main.app) + main.app.dependency_overrides.clear() + + +def test_services_merges_catalog(client): + r = client.get("/services") + assert r.status_code == 200 + rows = r.json()["services"] + ids = {row["service_id"] for row in rows} + assert "s1" in ids and "s2" in ids + + +def test_auto_register_activates_local_and_subscribes_remote(client): + r = client.post("/services/auto-register") + assert r.status_code == 200 + body = r.json() + assert body["activated"] >= 1 # s1 local + assert body["requested"] >= 1 # s2 remote subscribed + assert "s2" in main._test_sub_calls + # s1 now active in the overlay-backed view + rows = {x["service_id"]: x for x in client.get("/services").json()["services"]} + assert rows["s1"]["active"] is True + + +def test_catalog_unavailable_degrades(client, monkeypatch): + monkeypatch.setattr(annuaire_client, "get_catalog", lambda *a, **k: ([], "socket down")) + r = client.get("/services") + assert r.status_code == 200 + assert r.json().get("catalog_unavailable") is True diff --git a/packages/secubox-p2p/www/p2p/index.html b/packages/secubox-p2p/www/p2p/index.html index 08a44d0d..2d05a05d 100644 --- a/packages/secubox-p2p/www/p2p/index.html +++ b/packages/secubox-p2p/www/p2p/index.html @@ -553,7 +553,10 @@

Service Registry

- +
+ + +
@@ -786,21 +789,65 @@ const services = Array.isArray(data) ? data : (data && Array.isArray(data.services) ? data.services : []); const tbody = document.getElementById('services-table'); if (services.length > 0) { - tbody.innerHTML = services.map(svc => ` + tbody.innerHTML = services.map(svc => { + const state = svc.subscription_state || 'n/a'; + const statusLabel = svc.active ? 'active' : (state === 'n/a' ? 'inactive' : state); + const badge = svc.automatable ? ' automatable' : ''; + let actions = ''; + if (svc.source === 'p2p-local') { + actions = ``; + } else if (svc.provider_label !== 'local' && state === 'not-subscribed') { + actions = ``; + } else if (state === 'pending') { + actions = `awaiting approval`; + } else if (!svc.active) { + actions = ``; + } else { + actions = `active`; + } + return ` - - - - - - - - `).join(''); + + + + + + + `; + }).join(''); } else { - tbody.innerHTML = ''; + tbody.innerHTML = ''; } + if (data && data.catalog_unavailable) { + console.log('[secubox-p2p] Annuaire catalog unavailable — showing local services only'); + const noticeRow = document.createElement('tr'); + noticeRow.innerHTML = ''; + tbody.prepend(noticeRow); + } + } + + async function autoRegisterAll() { + const res = await apiPost('/services/auto-register', {}); + if (!res) { console.error('[secubox-p2p] autoRegisterAll: API error'); return; } + const msg = `[secubox-p2p] Auto-register: activated ${res.activated||0}, requested ${res.requested||0}` + + (res.pending ? `, ${res.pending} pending approval` : '') + + (res.errors && res.errors.length ? `, ${res.errors.length} error(s)` : ''); + console.log(msg); + loadServices(); + } + + async function requestAccess(sid) { + const res = await apiPost('/services/' + encodeURIComponent(sid) + '/request', {}); + if (!res) { console.error('[secubox-p2p] requestAccess: API error'); return; } + console.log('[secubox-p2p] requestAccess:', res.status === 'ok' ? 'Subscription requested' : (res.error || 'failed')); + loadServices(); + } + + async function activateService(sid) { + const res = await apiPost('/services/' + encodeURIComponent(sid) + '/activate', {}); + if (!res) { console.error('[secubox-p2p] activateService: API error'); return; } + console.log('[secubox-p2p] activateService:', res.status === 'ok' ? 'Activated' : (res.error || 'failed')); + loadServices(); } // Load Threats
${escapeHtml(svc.name)}${escapeHtml(svc.type)}${escapeHtml(svc.provider || 'local')}${svc.port}${svc.status} - -
${escapeHtml(svc.name)}${badge}${escapeHtml(svc.type || '')}${escapeHtml(svc.provider_label || 'local')}${svc.port != null ? svc.port : '—'}${escapeHtml(statusLabel)}${actions}
No services registered
No services in catalog
⚠ Annuaire catalog unavailable — showing local services only