Merge M1: p2p Service Registry as live view of annuaire catalog (#769, PR #770)

secubox-p2p 1.8.0 — /services live-merges the federated annuaire catalog with a
thin activation overlay; Auto register all; consumer subscribes as the node.
Reviewed (SDD, Critical XSS + init_dirs regression fixed) + deployed gk2+c3box.
This commit is contained in:
CyberMind-FR 2026-07-01 08:37:39 +02:00
commit dfa0a778c2
11 changed files with 1058 additions and 17 deletions

View File

@ -3,6 +3,30 @@
---
## 2026-06-30 — secubox-p2p 1.8.0 : Service Registry = live view of annuaire catalog (#769)
Le registre de services p2p (`/p2p/`) affichait « No services registered » (JSON local
isolé), déconnecté du catalogue fédéré secubox-annuaire 0.2.0. Désormais c'est une **vue
live** : `GET /services` fusionne le catalogue annuaire + mes abonnements + un mince
*activation overlay* + les services p2p-locaux hérités (sans duplication, sans dérive).
`api/registry.py` (fusion pure, testable) + `api/annuaire_client.py` (lit
`/run/secubox/annuaire.sock`, jamais l'aggregator ; s'abonne EN TANT QUE nœud via la clé
0600 ; frappe un JWT de service car le subscribe annuaire est JWT-gated). 4 endpoints :
`GET /services` (dégrade en `catalog_unavailable`, ne 500 jamais), `POST
/services/auto-register` (active les offres locales + s'abonne aux distantes selon
auto/pending), `/{id}/request`, `/{id}/activate`. UI : bouton « Auto register all » +
Request access / Activate + badges d'état (service_id en encodeURIComponent — XSS-safe).
annuaire inchangé ; exécution des macros déférée au Milestone 2.
Construit via SDD (5 tâches TDD + revues par tâche + revue finale opus = READY TO MERGE).
Bug trouvé au déploiement : le subscribe annuaire est JWT-gated ET vérifie que le sujet est
un user activé → token de service frappé pour `admin` (override SBX_SERVICE_USER). 34 tests.
Déployé gk2 + c3box : `GET /services` = WAF mirror (local) + Suricata (fédéré de c3box) sur
gk2, image miroir sur c3box ; `auto-register` gk2 = {activated:1, requested:1} → Suricata
fédéré **approved**.
---
## 2026-06-30 — secubox-annuaire 0.2.0 : trustless cross-node service federation (#766)
Le `/services/pull` de 0.1.3 n'était **pas** réellement sans-confiance ni opérable :

View File

@ -0,0 +1,113 @@
# Task 2 Report — annuaire_client.py
## Files Changed
- **Created**: `packages/secubox-p2p/api/annuaire_client.py`
- **Created**: `packages/secubox-p2p/tests/test_annuaire_client.py`
No other files touched.
---
## Pytest Command and Full Output
```
cd packages/secubox-p2p && python3 -m pytest tests/test_annuaire_client.py -v
```
```
============================= test session starts ==============================
platform linux -- Python 3.12.3, pytest-9.0.2, pluggy-1.6.0
cachedir: .pytest_cache
rootdir: /home/reepost/CyberMindStudio/secubox-deb-worktrees/769-p2p-service-registry-as-live-view-of-ann
configfile: pytest.ini
plugins: anyio-4.12.1, asyncio-1.3.0
asyncio: mode=Mode.STRICT, debug=False, asyncio_default_fixture_loop_scope=None
collecting ... collected 4 items
tests/test_annuaire_client.py::test_get_catalog_reads_services PASSED [ 25%]
tests/test_annuaire_client.py::test_get_catalog_socket_missing_returns_error PASSED [ 50%]
tests/test_annuaire_client.py::test_node_identity_reads_key PASSED [ 75%]
tests/test_annuaire_client.py::test_node_identity_missing PASSED [100%]
============================== 4 passed in 0.56s ===============================
```
Full suite (no regressions):
```
cd packages/secubox-p2p && python3 -m pytest tests/ -q
29 passed in 0.60s
```
---
## Test Harness Choice and Rationale
### The brief's `_serve_unix` approach
The brief's helper used `http.server.HTTPServer.__new__` to bypass `__init__`,
then manually assigned `srv.socket`. This is fragile because:
1. `socketserver.BaseServer.serve_forever()` calls `selectors.register(self, ...)`,
which in turn calls `_fileobj_to_fd(fileobj)` — it expects `fileobj.fileno()`.
A bare `HTTPServer` object has no `fileno()` method, so this raises
`ValueError: Invalid file object` and the server thread crashes immediately.
2. Even if it didn't crash, `HTTPServer.__init__` sets internal state
(`_BaseServer__is_shut_down`, `_BaseServer__shutdown_request`) that
`serve_forever` depends on — `__new__` + partial assignment is unreliable.
### Chosen approach: `socketserver.BaseServer` subclass with `fileno()`
Implemented `_UnixSocketHTTPServer(socketserver.ThreadingMixIn, socketserver.BaseServer)`:
- `__init__` creates and binds the AF_UNIX socket (no `bind_and_activate` bypass needed).
- `fileno()` delegates to `self.socket.fileno()` — required by `serve_forever`'s selector.
- `get_request()` accepts and returns `(conn, server_address)`.
- `server_bind()` / `server_activate()` are no-ops (socket already bound/listening).
- `shutdown_request()` / `close_request()` follow the stdlib pattern.
A real `_make_handler(routes)` factory produces a `BaseHTTPRequestHandler` subclass
that routes GET/POST by path and returns JSON.
### Why not monkeypatching?
The brief explicitly said the `_serve_unix` approach was fragile and offered
monkeypatching as an alternative. However, since the four behaviors include
**(a) end-to-end unix socket I/O** (not just JSON parsing), a real server is
strongly preferred — it actually exercises `_UnixHTTPConnection.connect()`.
Monkeypatching `_request` would make test (a) vacuous for the transport layer.
The `BaseServer` subclass achieves a real socket round-trip without fragility.
---
## Self-Review
### Correctness
- `did_from_pubkey_hex` matches the spec exactly: `"did:plc:" + sha256(pubkey_bytes).hexdigest()[:32]`.
- `node_identity` derives the public key via `cryptography.hazmat` ed25519 (same library the annuaire module uses), so the DID is identical to what annuaire would compute.
- `_request` swallows all exceptions and returns `(None, error_str)` — never raises into the caller.
- `get_catalog` and `get_subscriptions` return `([], error)` on any failure — never `(None, ...)`.
### Security
- Uses `cryptography` (already a declared dependency) only inside `node_identity`, with a lazy import to avoid import-time side effects.
- No secrets logged: `priv_hex` appears only in the returned tuple, never in error strings.
- The socket path defaults to the annuaire's own socket, never the aggregator.
### Compatibility
- No new stdlib or third-party imports beyond what the brief permits (`http.client`, `socket`, `json`, `hashlib`, plus `cryptography` already present).
- SPDX header and copyright block match `api/mesh.py` exactly.
---
## Concerns
None blocking. One minor note:
- `subscribe()` forwards `priv_hex` in the POST body to the annuaire. If the
annuaire API changes to require a signed challenge instead of the raw key, this
will need updating. The interface is documented in the docstring.
- The `_TIMEOUT = 3.0` s is suitable for localhost unix sockets; if the annuaire
is slow to start (e.g., during board boot), callers may get transient errors.
The double-caching pattern in the brief's performance section handles this
gracefully (cache miss → empty widget, retry next tick).

View File

@ -0,0 +1,102 @@
# Task 3 Report — Wire endpoints in api/main.py
## Status: DONE
---
## Files Changed
### `packages/secubox-p2p/api/main.py`
| Change | Lines (approx) |
|--------|---------------|
| Import `registry, annuaire_client` added to existing `from . import mesh` | line 29 |
| `ACTIVATION_FILE = P2P_DIR / "activation.json"` constant added | line 46 |
| `init_dirs()` updated: wrapped `P2P_DIR.mkdir` in `try/except PermissionError`; also mkdir parents of `ACTIVATION_FILE` and `SERVICES_FILE` (enables monkeypatching in tests) | lines 6474 |
| `GET /services` `list_services` replaced: now live-merges catalog+subscriptions+overlay+legacy via `registry.merge_services` | lines 838851 |
| `POST /services/auto-register` added after `unregister_service` | lines 868907 |
| `POST /services/{service_id}/request` added | lines 910920 |
| `POST /services/{service_id}/activate` added | lines 923939 |
### `packages/secubox-p2p/tests/test_services_endpoints.py`
New file: 3 test cases (verbatim from brief) + one adaptation:
- Added `_override_jwt` async stub + `app.dependency_overrides` wiring in fixture.
**Reason:** the live secubox_core is installed in `/usr/lib/python3/dist-packages` and the real `require_jwt` validates tokens; the fallback no-op only applies when secubox_core is absent. The brief assumes a dev env without secubox_core. The override uses the standard FastAPI `dependency_overrides` mechanism and is correctly torn down with `yield`+`clear()`.
---
## Test Results
### Task tests only
```
cd packages/secubox-p2p && python3 -m pytest tests/test_services_endpoints.py -v
3 passed in 0.27s
```
### Full suite
```
cd packages/secubox-p2p && python3 -m pytest tests/ -v
32 passed, 1 warning in 0.80s
```
All prior tests (test_mesh.py ×21, test_registry.py ×5, test_annuaire_client.py ×4) remain green.
---
## Self-Review
### Correctness
- `GET /services` correctly returns `{"services": [...], "catalog_unavailable": true}` shape on catalog error.
- `auto-register` correctly distinguishes local (provider == local_did → set_active) vs remote (subscribe → set_subscription).
- `request` and `activate` correctly delegate to annuaire_client and registry.
- All three POST endpoints require JWT (`Depends(require_jwt)`).
### init_dirs() change
The `try/except PermissionError` on `P2P_DIR.mkdir` is safe: in production the directory exists (created by postinst), so the branch is never taken. The extra `ACTIVATION_FILE.parent.mkdir` is also a no-op in production since `ACTIVATION_FILE.parent == P2P_DIR`. In tests, both changes are essential for monkeypatching to work without touching the real `/var/lib/secubox/p2p`.
### Route ordering concern (DONE_WITH_CONCERNS note)
FastAPI matches `/services/auto-register` before `/services/{service_id}/...` because static path segments rank above parameterised ones. Verified correct ordering in the router by placing `auto-register` before the `{service_id}` routes.
---
## Concerns
None blocking. One note:
- **IDE Pylance diagnostics**: "Impossible de résoudre l'importation `api`" in the test file. This is a false positive — `conftest.py` injects the package root into `sys.path` at pytest collection time, which Pylance's static analyser doesn't see. All three runtime imports resolve correctly (verified by pytest).
---
## Review Fix — commit 0e1c6c2f (2026-06-30)
**Problem addressed**: Review finding on commit 36ed77c8 — `init_dirs()` silently swallowed `PermissionError` on `P2P_DIR.mkdir` and added extra `ACTIVATION_FILE.parent` / `SERVICES_FILE.parent` mkdir calls so tests could run with monkeypatched paths. This weakened production: a real PermissionError on `/var/lib/secubox/p2p` would be silently dropped.
**Changes made**:
### `packages/secubox-p2p/api/main.py`
- Reverted `init_dirs()` to pre-Task-3 body (matching commit 768154ff):
```python
def init_dirs():
P2P_DIR.mkdir(parents=True, exist_ok=True)
```
- Removed: `try/except PermissionError` wrapper, the `for _p in (ACTIVATION_FILE, SERVICES_FILE)` loop, and the inner `try/except (PermissionError, AttributeError)` block.
- `ACTIVATION_FILE` constant and its import remain untouched.
### `packages/secubox-p2p/tests/test_services_endpoints.py`
- Added `monkeypatch.setattr(main, "init_dirs", lambda: None)` in the `client` fixture (immediately after the `ACTIVATION_FILE` / `SERVICES_FILE` monkeypatches).
- Tests now bypass `init_dirs` entirely; `registry.save_overlay` handles its own `os.makedirs` on the monkeypatched `ACTIVATION_FILE` path. `SERVICES_FILE` is read-only in tests.
**Test results**:
```
$ cd packages/secubox-p2p && python3 -m pytest tests/test_services_endpoints.py -q
3 passed, 1 warning in 0.31s
$ python3 -m pytest tests/ -q
32 passed, 1 warning in 0.80s
```
**Confirmation**: `init_dirs` no longer contains `except PermissionError` (verified by grep). A real `PermissionError` on `/var/lib/secubox/p2p` at startup will now surface as an unhandled exception, correctly exposing the misconfiguration.

View File

@ -0,0 +1,188 @@
# SPDX-License-Identifier: LicenseRef-CMSD-1.0
# Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
# Source-Disclosed License — All rights reserved except as expressly granted.
# See LICENCE-CMSD-1.0.md for terms.
"""
SecuBox-Deb :: secubox-p2p :: annuaire_client
Thin client to the LOCAL secubox-annuaire over its own unix socket
(/run/secubox/annuaire.sock never the aggregator). Subscribes AS THE NODE
using the 0600 node key shared by the secubox user. Never raises into the
request path: every call returns (data, error).
"""
from __future__ import annotations
import hashlib
import http.client
import json
import os
import socket as _socket
from typing import Any, Dict, List, Optional, Tuple
ANNUAIRE_SOCK = "/run/secubox/annuaire.sock"
NODE_KEY_PATH = "/etc/secubox/secrets/annuaire/node.key"
_TIMEOUT = 3.0
class _UnixHTTPConnection(http.client.HTTPConnection):
"""HTTPConnection that connects via a unix domain socket."""
def __init__(self, sock_path: str, timeout: float = _TIMEOUT):
super().__init__("localhost", timeout=timeout)
self._sock_path = sock_path
def connect(self):
s = _socket.socket(_socket.AF_UNIX, _socket.SOCK_STREAM)
s.settimeout(self.timeout)
s.connect(self._sock_path)
self.sock = s
# annuaire's require_jwt is a pure presence-gate: it only checks a valid HS256
# signature + that the token subject is an ENABLED user (user_store.is_enabled);
# the subject carries no authorization (the real authz is the ed25519 MEMBER
# check on the node key in the request body). So the token must merely name a
# real enabled user — a service-principal string is rejected by is_enabled().
# `admin` is a guaranteed master user on every login-capable SecuBox app;
# override via SBX_SERVICE_USER if needed.
SERVICE_USER = os.environ.get("SBX_SERVICE_USER", "admin")
def _service_token() -> Optional[str]:
"""Mint a short service JWT so we can call annuaire's JWT-gated endpoints.
annuaire and secubox-p2p run on the same host and share the same
secubox_core JWT secret, so a token minted here validates there. The
subject must be a real ENABLED user (SERVICE_USER) because annuaire's
require_jwt rejects any subject that fails user_store.is_enabled(). Used
for mutating calls (subscribe); the read endpoints (/services,
/subscriptions) are public and need no token. Returns None if secubox_core
is unavailable (e.g. in unit tests) the caller then sends no header.
"""
try:
from secubox_core.auth import create_token # noqa: PLC0415
return create_token(SERVICE_USER)
except Exception: # noqa: BLE001
return None
def _request(
method: str,
path: str,
sock: str,
body: Optional[dict] = None,
auth_token: Optional[str] = None,
) -> Tuple[Optional[Any], Optional[str]]:
"""Issue a single HTTP request over the given unix socket.
Returns (parsed_json_or_None, error_string_or_None). Never raises.
When auth_token is set, an Authorization: Bearer header is attached (needed
for annuaire's JWT-gated mutating endpoints).
"""
try:
conn = _UnixHTTPConnection(sock)
headers: Dict[str, str] = {"Accept": "application/json"}
if auth_token:
headers["Authorization"] = f"Bearer {auth_token}"
data: Optional[bytes] = None
if body is not None:
data = json.dumps(body).encode()
headers["Content-Type"] = "application/json"
conn.request(method, path, body=data, headers=headers)
resp = conn.getresponse()
raw = resp.read()
conn.close()
if resp.status >= 400:
return None, f"annuaire {method} {path} -> HTTP {resp.status}"
return (json.loads(raw) if raw else {}), None
except Exception as exc: # noqa: BLE001
return None, f"{type(exc).__name__}: {exc}"
def did_from_pubkey_hex(pub_hex: str) -> str:
"""Derive a did:plc identifier from a raw ed25519 public key (hex).
Mirrors annuaire/crypto.did_from_pubkey exactly:
"did:plc:" + sha256(pubkey_bytes).hexdigest()[:32]
"""
return "did:plc:" + hashlib.sha256(bytes.fromhex(pub_hex)).hexdigest()[:32]
def node_identity(
key_path: str = NODE_KEY_PATH,
) -> Tuple[Optional[str], Optional[str]]:
"""Return (did, priv_hex) derived from the node's ed25519 key file.
The key file must contain exactly 32 bytes encoded as 64 lowercase hex
characters (optionally followed by a newline). Returns (None, None) if the
file is absent, unreadable, or malformed.
"""
try:
with open(key_path, "r", encoding="ascii") as fh:
priv_hex = fh.read().strip()
priv = bytes.fromhex(priv_hex)
if len(priv) != 32:
return None, None
from cryptography.hazmat.primitives.asymmetric import ed25519
from cryptography.hazmat.primitives import serialization
priv_key = ed25519.Ed25519PrivateKey.from_private_bytes(priv)
pub_bytes = priv_key.public_key().public_bytes(
encoding=serialization.Encoding.Raw,
format=serialization.PublicFormat.Raw,
)
return did_from_pubkey_hex(pub_bytes.hex()), priv_hex
except Exception: # noqa: BLE001
return None, None
def get_catalog(
sock: str = ANNUAIRE_SOCK,
) -> Tuple[List[Dict], Optional[str]]:
"""Fetch the full service catalog from the local annuaire.
Returns (list_of_service_dicts, None) on success, or ([], error_string).
"""
data, err = _request("GET", "/api/v1/annuaire/services", sock)
if err:
return [], err
return (data or {}).get("services", []), None
def get_subscriptions(
mine_did: Optional[str] = None,
sock: str = ANNUAIRE_SOCK,
) -> Tuple[List[Dict], Optional[str]]:
"""Fetch subscriptions, optionally filtered to `mine_did`.
Returns (list_of_subscription_dicts, None) on success, or ([], error_string).
"""
path = "/api/v1/annuaire/subscriptions"
if mine_did:
path += f"?mine={mine_did}"
data, err = _request("GET", path, sock)
if err:
return [], err
return (data or {}).get("subscriptions", []), None
def subscribe(
service_id: str,
did: str,
priv_hex: str,
sock: str = ANNUAIRE_SOCK,
) -> Tuple[Optional[Dict], Optional[str]]:
"""Subscribe this node (identified by `did`) to a service.
Returns (response_dict, None) on success, or (None, error_string).
The `priv_hex` is forwarded to the annuaire so it can verify the
subscriber's identity (annuaire validates the ed25519 key pair). annuaire's
subscribe endpoint is JWT-gated, so a service token is also attached.
"""
return _request(
"POST",
f"/api/v1/annuaire/service/{service_id}/subscribe",
sock,
body={"subscriber_did": did, "subscriber_priv_hex": priv_hex},
auth_token=_service_token(),
)

View File

@ -26,7 +26,7 @@ except ImportError:
async def require_jwt():
return {"sub": "admin"}
from . import mesh
from . import mesh, registry, annuaire_client
app = FastAPI(
title="SecuBox P2P API",
@ -43,6 +43,7 @@ SERVICES_FILE = P2P_DIR / "services.json"
PROFILES_FILE = P2P_DIR / "profiles.json"
THREATS_FILE = P2P_DIR / "threats.json"
NODE_ID_FILE = P2P_DIR / "node.id"
ACTIVATION_FILE = P2P_DIR / "activation.json"
CONFIG_FILE = Path("/etc/secubox/p2p.toml")
# Master-Link paths
@ -829,10 +830,20 @@ async def probe_peer(peer_id: str, user: dict = Depends(require_jwt)):
@app.get("/services")
async def list_services():
"""List all P2P services (public read)."""
"""Live view: annuaire catalog ⨝ my subscriptions ⨝ activation overlay ⨝ legacy."""
init_dirs()
services = load_json(SERVICES_FILE, [])
return {"services": services if isinstance(services, list) else []}
local_did, _ = annuaire_client.node_identity()
catalog, cat_err = annuaire_client.get_catalog()
subs, _ = annuaire_client.get_subscriptions(local_did)
overlay = registry.load_overlay(str(ACTIVATION_FILE))
legacy = load_json(SERVICES_FILE, [])
legacy = legacy if isinstance(legacy, list) else []
rows = registry.merge_services(catalog, subs, overlay, legacy, local_did)
out = {"services": rows}
if cat_err:
out["catalog_unavailable"] = True
out["catalog_error"] = cat_err
return out
@app.post("/services")
@ -867,6 +878,80 @@ async def unregister_service(name: str, user: dict = Depends(require_jwt)):
return {"status": "ok"}
@app.post("/services/auto-register")
async def auto_register(user: dict = Depends(require_jwt)):
"""Activate local catalog services + subscribe to remote ones (per approval mode)."""
init_dirs()
local_did, priv = annuaire_client.node_identity()
catalog, cat_err = annuaire_client.get_catalog()
if cat_err:
return {"activated": 0, "requested": 0, "pending": 0, "already": 0,
"errors": [cat_err]}
subs, _ = annuaire_client.get_subscriptions(local_did)
subscribed = {s.get("service_id") for s in subs}
overlay = registry.load_overlay(str(ACTIVATION_FILE))
activated = requested = pending = already = 0
errors = []
for offer in catalog:
sid = offer.get("service_id")
if not sid:
continue
if offer.get("provider") == local_did:
registry.set_active(str(ACTIVATION_FILE), sid,
registry.port_from_endpoint(offer.get("endpoint", "")))
activated += 1
continue
if sid in subscribed or (overlay.get(sid, {}).get("subscription_id")):
already += 1
continue
if not priv or not local_did:
errors.append(f"{sid}: node has no annuaire identity (run annuairectl init)")
continue
res, err = annuaire_client.subscribe(sid, local_did, priv)
if err:
errors.append(f"{sid}: {err}")
continue
registry.set_subscription(str(ACTIVATION_FILE), sid, res.get("subscription_id", ""))
requested += 1
if res.get("state") == "pending":
pending += 1
return {"activated": activated, "requested": requested, "pending": pending,
"already": already, "errors": errors}
@app.post("/services/{service_id}/request")
async def request_access(service_id: str, user: dict = Depends(require_jwt)):
"""Subscribe to a single remote service offer."""
init_dirs()
local_did, priv = annuaire_client.node_identity()
if not priv or not local_did:
return {"status": "error", "error": "node has no annuaire identity (run annuairectl init)"}
res, err = annuaire_client.subscribe(service_id, local_did, priv)
if err:
return {"status": "error", "error": err}
registry.set_subscription(str(ACTIVATION_FILE), service_id, res.get("subscription_id", ""))
return {"status": "ok", "subscription": res}
@app.post("/services/{service_id}/activate")
async def activate_service(service_id: str, user: dict = Depends(require_jwt)):
"""Mark a catalog service locally active (binds the derived local port)."""
init_dirs()
catalog, _ = annuaire_client.get_catalog()
offer = next((o for o in catalog if o.get("service_id") == service_id), None)
if offer is None:
return {"status": "error", "error": "service not in catalog"}
local_did, _ = annuaire_client.node_identity()
if offer.get("provider") != local_did:
subs, _ = annuaire_client.get_subscriptions(local_did)
st = next((s.get("state") for s in subs if s.get("service_id") == service_id), None)
if st != "approved":
return {"status": "error", "error": f"remote service not approved (state={st})"}
registry.set_active(str(ACTIVATION_FILE), service_id,
registry.port_from_endpoint(offer.get("endpoint", "")))
return {"status": "ok"}
# ============== Mesh Network ==============
@app.get("/mesh")

View File

@ -0,0 +1,139 @@
# SPDX-License-Identifier: LicenseRef-CMSD-1.0
# Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
# Source-Disclosed License — All rights reserved except as expressly granted.
# See LICENCE-CMSD-1.0.md for terms.
"""
SecuBox-Deb :: secubox-p2p :: registry
Pure merge logic for the Service Registry: combines the annuaire catalog, my
subscriptions, the local activation overlay, and legacy p2p-local services into
the rows the UI renders. No network I/O lives here (see annuaire_client.py) so
the merge is fully unit-testable.
"""
from __future__ import annotations
import json
import re
from datetime import datetime, timezone
from typing import Any, Dict, List, Optional
# Service kinds that (in Milestone 2) carry an executable access macro. In M1
# this only drives a cosmetic "automatable" badge.
MACRO_KINDS = {"tor-exit", "wg-relay", "dns-resolver", "http-mirror"}
_PORT_RE = re.compile(r":(\d{1,5})(?:/|$)")
def port_from_endpoint(endpoint: str) -> Optional[int]:
"""Best-effort extract a TCP port from a host:port or URL endpoint."""
if not endpoint:
return None
m = _PORT_RE.search(endpoint)
if not m:
return None
try:
p = int(m.group(1))
except ValueError:
return None
return p if 0 < p < 65536 else None
def load_overlay(path: str) -> Dict[str, Any]:
try:
with open(path, "r", encoding="utf-8") as fh:
data = json.load(fh)
return data if isinstance(data, dict) else {}
except Exception:
return {}
def save_overlay(path: str, data: Dict[str, Any]) -> None:
import os
os.makedirs(os.path.dirname(path), exist_ok=True)
tmp = path + ".tmp"
with open(tmp, "w", encoding="utf-8") as fh:
json.dump(data, fh)
os.replace(tmp, path)
def set_active(path: str, service_id: str, local_port: Optional[int],
subscription_id: Optional[str] = None) -> Dict[str, Any]:
data = load_overlay(path)
entry = data.get(service_id, {})
entry["active"] = True
entry["local_port"] = local_port
if subscription_id is not None:
entry["subscription_id"] = subscription_id
entry.setdefault("subscription_id", None)
entry["activated_at"] = datetime.now(timezone.utc).isoformat()
data[service_id] = entry
save_overlay(path, data)
return data
def set_subscription(path: str, service_id: str, subscription_id: str) -> Dict[str, Any]:
data = load_overlay(path)
entry = data.get(service_id, {})
entry["subscription_id"] = subscription_id
entry.setdefault("active", False)
entry.setdefault("local_port", None)
data[service_id] = entry
save_overlay(path, data)
return data
def merge_services(catalog: List[Dict], subscriptions: List[Dict],
overlay: Dict[str, Any], legacy: List[Dict],
local_did: Optional[str]) -> List[Dict]:
sub_state = {}
for s in subscriptions or []:
sid = s.get("service_id")
if sid:
sub_state[sid] = s.get("state", "pending")
rows: List[Dict] = []
for offer in catalog or []:
sid = offer.get("service_id")
if not sid:
continue
provider = offer.get("provider")
is_local = bool(local_did) and provider == local_did
ov = overlay.get(sid, {})
kind = offer.get("kind", "")
rows.append({
"service_id": sid,
"name": offer.get("name", ""),
"type": kind,
"provider": provider,
"provider_label": "local" if is_local else _short_did(provider),
"port": ov.get("local_port") or port_from_endpoint(offer.get("endpoint", "")),
"approval_mode": offer.get("approval_mode", "auto"),
"subscription_state": sub_state.get(sid, "not-subscribed"),
"active": bool(ov.get("active", False)),
"source": "annuaire",
"automatable": kind in MACRO_KINDS,
})
for svc in legacy or []:
rows.append({
"service_id": None,
"name": svc.get("name", ""),
"type": svc.get("protocol", svc.get("type", "")),
"provider": None,
"provider_label": "local",
"port": svc.get("port"),
"approval_mode": None,
"subscription_state": "n/a",
"active": bool(svc.get("active", True)),
"source": "p2p-local",
"automatable": False,
})
rows.sort(key=lambda r: (r["provider_label"] != "local", r["name"].lower()))
return rows
def _short_did(did: Optional[str]) -> str:
if not did:
return "unknown"
return did[:20] + "" if len(did) > 21 else did

View File

@ -1,3 +1,22 @@
secubox-p2p (1.8.0-1~bookworm1) bookworm; urgency=medium
* feat: Service Registry is now a live view of the secubox-annuaire catalog
(#769). New api/registry.py (pure merge of annuaire catalog + my
subscriptions + a thin activation overlay + legacy local services) and
api/annuaire_client.py (reads /run/secubox/annuaire.sock — never the
aggregator — and subscribes as the node using the 0600 node key shared by
the secubox user). New endpoints: GET /services (live merge, degrades to
catalog_unavailable when annuaire is down — never 500s), POST
/services/auto-register (activate local offers + subscribe to remote ones
per each offer's auto/pending approval mode), /services/{id}/request,
/services/{id}/activate (gated on approved subscription). UI: "Auto
register all" button + per-service Request access / Activate + state
badges + an "automatable" forward-hint badge; service_id is injected into
onclick via encodeURIComponent (XSS-safe for federated IDs). annuaire is
unchanged; provider-side macro execution deferred to Milestone 2.
-- Gerald Kerma <devel@cybermind.fr> Tue, 30 Jun 2026 19:00:00 +0200
secubox-p2p (1.7.8-1~bookworm1) bookworm; urgency=medium
* fix(webui): P2P dashboard rendered empty due to API-contract drift.

View File

@ -0,0 +1,196 @@
# SPDX-License-Identifier: LicenseRef-CMSD-1.0
# Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
# Source-Disclosed License — All rights reserved except as expressly granted.
# See LICENCE-CMSD-1.0.md for terms.
"""
Tests for api/annuaire_client.py
Test harness choice: monkeypatch `annuaire_client._request` for parsing tests
(a) and (c), real-socket-missing test for (b), and a real threading unix-socket
HTTP server for end-to-end verification of (a).
Rationale: The brief's _serve_unix helper uses HTTPServer.__new__ + manual
socket injection, which is fragile (HTTPServer.__init__ was already called or
not at all, depending on CPython internals). Instead we use two approaches:
1. A real unix-socket threading server built with socketserver.BaseServer
directly this is what _serve_unix below does (simplified).
2. Monkeypatching _request for the catalog-parsing test to avoid flakiness.
All four required behaviors are genuinely asserted:
(a) get_catalog parses {"services":[...]} from a unix-socket HTTP response
(b) get_catalog returns ([], error) when socket is missing
(c) node_identity reads a 32-byte hex key and derives a stable did:plc
(d) node_identity returns (None, None) when the key file is absent
"""
import json
import os
import socket
import socketserver
import threading
import http.server
from api import annuaire_client as ac
# ---------------------------------------------------------------------------
# Minimal unix-socket HTTP server (robust alternative to HTTPServer.__new__)
# ---------------------------------------------------------------------------
class _UnixSocketHTTPServer(socketserver.ThreadingMixIn, socketserver.BaseServer):
"""HTTP server bound to a unix socket via socketserver.BaseServer.
socketserver.BaseServer.serve_forever() calls selectors.register(self, ...)
which requires self to have a fileno() method returning the listening fd.
"""
allow_reuse_address = True
daemon_threads = True
def __init__(self, sock_path, RequestHandlerClass):
socketserver.BaseServer.__init__(self, sock_path, RequestHandlerClass)
if os.path.exists(sock_path):
os.unlink(sock_path)
self.socket = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
self.socket.bind(sock_path)
self.socket.listen(8)
self.server_address = sock_path
# Required by selectors so serve_forever can poll the listening socket.
def fileno(self):
return self.socket.fileno()
def get_request(self):
conn, _ = self.socket.accept()
return conn, self.server_address
def server_bind(self):
pass # already done in __init__
def server_activate(self):
pass # already done in __init__
def shutdown_request(self, request):
try:
request.shutdown(socket.SHUT_WR)
except OSError:
pass
self.close_request(request)
def close_request(self, request):
request.close()
def _make_handler(routes):
"""Return a BaseHTTPRequestHandler class that serves from `routes`."""
class H(http.server.BaseHTTPRequestHandler):
def _send(self):
body = b""
st_code = 404
for p, (st, obj) in routes.items():
if self.path == p:
body = json.dumps(obj).encode()
st_code = st
break
else:
body = b'{"detail":"nf"}'
self.send_response(st_code)
self.send_header("Content-Type", "application/json")
self.end_headers()
self.wfile.write(body)
do_GET = _send
do_POST = _send
def log_message(self, *a):
pass
return H
def _serve_unix(sock_path, routes):
"""Start a unix-socket HTTP server in a daemon thread. Returns server."""
srv = _UnixSocketHTTPServer(sock_path, _make_handler(routes))
t = threading.Thread(target=srv.serve_forever, daemon=True)
t.start()
return srv
# ---------------------------------------------------------------------------
# Tests — four required behaviors
# ---------------------------------------------------------------------------
def test_get_catalog_reads_services(tmp_path):
"""(a) get_catalog parses {"services":[...]} JSON from a unix-socket HTTP response."""
sp = str(tmp_path / "ann.sock")
routes = {
"/api/v1/annuaire/services": (200, {"services": [{"service_id": "s1", "name": "WAF"}]})
}
srv = _serve_unix(sp, routes)
try:
offers, err = ac.get_catalog(sock=sp)
assert err is None, f"unexpected error: {err}"
assert len(offers) == 1
assert offers[0]["service_id"] == "s1"
assert offers[0]["name"] == "WAF"
finally:
srv.shutdown()
def test_get_catalog_socket_missing_returns_error(tmp_path):
"""(b) get_catalog returns ([], error) when the socket file does not exist."""
offers, err = ac.get_catalog(sock=str(tmp_path / "nope.sock"))
assert offers == []
assert err is not None
assert len(err) > 0
def test_node_identity_reads_key(tmp_path):
"""(c) node_identity reads a 32-byte hex key and derives a stable did:plc."""
key = tmp_path / "node.key"
key.write_text("11" * 32 + "\n")
did, priv = ac.node_identity(key_path=str(key))
assert priv == "11" * 32
assert did is not None
assert did.startswith("did:plc:")
# deterministic — calling again yields the same result
did2, priv2 = ac.node_identity(key_path=str(key))
assert did2 == did
assert priv2 == priv
def test_node_identity_missing(tmp_path):
"""(d) node_identity returns (None, None) when the key file is absent."""
did, priv = ac.node_identity(key_path=str(tmp_path / "nope"))
assert did is None
assert priv is None
def test_subscribe_attaches_service_token(monkeypatch):
"""subscribe must mint + attach a service JWT (annuaire's subscribe is JWT-gated)."""
captured = {}
def fake_request(method, path, sock, body=None, auth_token=None):
captured["method"] = method
captured["auth_token"] = auth_token
captured["body"] = body
return {"subscription_id": "x", "state": "approved"}, None
monkeypatch.setattr(ac, "_request", fake_request)
monkeypatch.setattr(ac, "_service_token", lambda: "TESTTOKEN")
res, err = ac.subscribe("svc1", "did:plc:" + "a" * 32, "11" * 32)
assert err is None
assert captured["method"] == "POST"
assert captured["auth_token"] == "TESTTOKEN"
assert captured["body"]["subscriber_did"].startswith("did:plc:")
def test_service_token_returns_none_without_secubox_core(monkeypatch):
"""_service_token degrades to None when secubox_core is unavailable (unit env)."""
import builtins
real_import = builtins.__import__
def blocked_import(name, *a, **k):
if name == "secubox_core.auth" or name.startswith("secubox_core"):
raise ImportError("blocked for test")
return real_import(name, *a, **k)
monkeypatch.setattr(builtins, "__import__", blocked_import)
assert ac._service_token() is None

View File

@ -0,0 +1,66 @@
# SPDX-License-Identifier: LicenseRef-CMSD-1.0
# Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
# Source-Disclosed License — All rights reserved except as expressly granted.
# See LICENCE-CMSD-1.0.md for terms.
"""
SecuBox-Deb :: secubox-p2p :: tests :: test_registry
Test the pure merge logic and activation overlay.
"""
import json
from api import registry
def test_port_from_endpoint():
assert registry.port_from_endpoint("http://10.10.0.1:9050/x") == 9050
assert registry.port_from_endpoint("10.10.0.2:3483") == 3483
assert registry.port_from_endpoint("/local/path") is None
assert registry.port_from_endpoint("") is None
def test_merge_local_vs_remote_and_state():
local_did = "did:plc:" + "a" * 32
remote_did = "did:plc:" + "b" * 32
catalog = [
{"service_id": "s1", "name": "WAF mirror", "kind": "module",
"provider": local_did, "endpoint": "http://10.10.0.1:8085",
"approval_mode": "auto"},
{"service_id": "s2", "name": "Tor exit", "kind": "tor-exit",
"provider": remote_did, "endpoint": "10.10.0.2:9050",
"approval_mode": "pending"},
]
subs = [{"service_id": "s2", "state": "pending"}]
overlay = {"s1": {"active": True, "local_port": 8085, "subscription_id": None}}
rows = registry.merge_services(catalog, subs, overlay, [], local_did)
by_id = {r["service_id"]: r for r in rows}
assert by_id["s1"]["provider_label"] == "local"
assert by_id["s1"]["active"] is True
assert by_id["s1"]["subscription_state"] == "not-subscribed"
assert by_id["s2"]["provider_label"] != "local"
assert by_id["s2"]["subscription_state"] == "pending"
assert by_id["s2"]["automatable"] is True # tor-exit ∈ MACRO_KINDS
assert by_id["s1"]["automatable"] is False
def test_merge_includes_legacy_local():
legacy = [{"name": "old-svc", "port": 1234, "protocol": "tcp", "active": True}]
rows = registry.merge_services([], [], {}, legacy, None)
assert len(rows) == 1
assert rows[0]["source"] == "p2p-local"
assert rows[0]["provider_label"] == "local"
assert rows[0]["port"] == 1234
def test_overlay_roundtrip_and_prune(tmp_path):
p = tmp_path / "activation.json"
registry.set_active(str(p), "s1", 8085)
data = registry.load_overlay(str(p))
assert data["s1"]["active"] is True and data["s1"]["local_port"] == 8085
# prune: merge drops overlay-only entries with no catalog/legacy backing
rows = registry.merge_services([], [], data, [], None)
assert rows == []
def test_load_overlay_missing_or_corrupt(tmp_path):
assert registry.load_overlay(str(tmp_path / "nope.json")) == {}
bad = tmp_path / "bad.json"; bad.write_text("{not json")
assert registry.load_overlay(str(bad)) == {}

View File

@ -0,0 +1,62 @@
# SPDX-License-Identifier: LicenseRef-CMSD-1.0
# Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
import pytest
from fastapi.testclient import TestClient
from api import main, annuaire_client, registry
async def _override_jwt():
return {"sub": "admin"}
@pytest.fixture
def client(tmp_path, monkeypatch):
monkeypatch.setattr(main, "ACTIVATION_FILE", tmp_path / "activation.json")
monkeypatch.setattr(main, "SERVICES_FILE", tmp_path / "services.json")
monkeypatch.setattr(main, "init_dirs", lambda: None)
local = "did:plc:" + "a" * 32
remote = "did:plc:" + "b" * 32
monkeypatch.setattr(annuaire_client, "node_identity", lambda *a, **k: (local, "11" * 32))
monkeypatch.setattr(annuaire_client, "get_catalog", lambda *a, **k: ([
{"service_id": "s1", "name": "WAF", "kind": "module", "provider": local,
"endpoint": "http://10.10.0.1:8085", "approval_mode": "auto"},
{"service_id": "s2", "name": "Tor", "kind": "tor-exit", "provider": remote,
"endpoint": "10.10.0.2:9050", "approval_mode": "auto"},
], None))
monkeypatch.setattr(annuaire_client, "get_subscriptions", lambda *a, **k: ([], None))
calls = []
def fake_sub(sid, did, priv, **k):
calls.append(sid); return ({"subscription_id": "sub-" + sid, "state": "approved"}, None)
monkeypatch.setattr(annuaire_client, "subscribe", fake_sub)
main._test_sub_calls = calls
# Bypass JWT auth for tests
main.app.dependency_overrides[main.require_jwt] = _override_jwt
yield TestClient(main.app)
main.app.dependency_overrides.clear()
def test_services_merges_catalog(client):
r = client.get("/services")
assert r.status_code == 200
rows = r.json()["services"]
ids = {row["service_id"] for row in rows}
assert "s1" in ids and "s2" in ids
def test_auto_register_activates_local_and_subscribes_remote(client):
r = client.post("/services/auto-register")
assert r.status_code == 200
body = r.json()
assert body["activated"] >= 1 # s1 local
assert body["requested"] >= 1 # s2 remote subscribed
assert "s2" in main._test_sub_calls
# s1 now active in the overlay-backed view
rows = {x["service_id"]: x for x in client.get("/services").json()["services"]}
assert rows["s1"]["active"] is True
def test_catalog_unavailable_degrades(client, monkeypatch):
monkeypatch.setattr(annuaire_client, "get_catalog", lambda *a, **k: ([], "socket down"))
r = client.get("/services")
assert r.status_code == 200
assert r.json().get("catalog_unavailable") is True

View File

@ -553,7 +553,10 @@
<div id="tab-services" class="tab-content">
<div class="section-header">
<h2>Service Registry</h2>
<button class="btn" onclick="showRegisterServiceModal()">+ Register Service</button>
<div>
<button class="btn" onclick="autoRegisterAll()">Auto register all</button>
<button class="btn" onclick="showRegisterServiceModal()">+ Register Service</button>
</div>
</div>
<table class="data-table">
<thead>
@ -786,21 +789,65 @@
const services = Array.isArray(data) ? data : (data && Array.isArray(data.services) ? data.services : []);
const tbody = document.getElementById('services-table');
if (services.length > 0) {
tbody.innerHTML = services.map(svc => `
tbody.innerHTML = services.map(svc => {
const state = svc.subscription_state || 'n/a';
const statusLabel = svc.active ? 'active' : (state === 'n/a' ? 'inactive' : state);
const badge = svc.automatable ? ' <span class="status-badge">automatable</span>' : '';
let actions = '';
if (svc.source === 'p2p-local') {
actions = `<button class="btn btn-small btn-danger" onclick="unregisterService('${encodeURIComponent(svc.name)}')">Unregister</button>`;
} else if (svc.provider_label !== 'local' && state === 'not-subscribed') {
actions = `<button class="btn btn-small" onclick="requestAccess('${encodeURIComponent(svc.service_id)}')">Request access</button>`;
} else if (state === 'pending') {
actions = `<span class="text-muted">awaiting approval</span>`;
} else if (!svc.active) {
actions = `<button class="btn btn-small" onclick="activateService('${encodeURIComponent(svc.service_id)}')">Activate</button>`;
} else {
actions = `<span class="status-badge active">active</span>`;
}
return `
<tr>
<td>${escapeHtml(svc.name)}</td>
<td>${escapeHtml(svc.type)}</td>
<td>${escapeHtml(svc.provider || 'local')}</td>
<td>${svc.port}</td>
<td><span class="status-badge ${svc.status}">${svc.status}</span></td>
<td>
<button class="btn btn-small btn-danger" onclick="unregisterService('${svc.name}')">Unregister</button>
</td>
</tr>
`).join('');
<td>${escapeHtml(svc.name)}${badge}</td>
<td>${escapeHtml(svc.type || '')}</td>
<td>${escapeHtml(svc.provider_label || 'local')}</td>
<td>${svc.port != null ? svc.port : '&mdash;'}</td>
<td><span class="status-badge ${svc.active ? 'active' : ''}">${escapeHtml(statusLabel)}</span></td>
<td>${actions}</td>
</tr>`;
}).join('');
} else {
tbody.innerHTML = '<tr><td colspan="6">No services registered</td></tr>';
tbody.innerHTML = '<tr><td colspan="6">No services in catalog</td></tr>';
}
if (data && data.catalog_unavailable) {
console.log('[secubox-p2p] Annuaire catalog unavailable — showing local services only');
const noticeRow = document.createElement('tr');
noticeRow.innerHTML = '<td colspan="6" style="color:var(--cinnabar,#e63946)">⚠ Annuaire catalog unavailable — showing local services only</td>';
tbody.prepend(noticeRow);
}
}
async function autoRegisterAll() {
const res = await apiPost('/services/auto-register', {});
if (!res) { console.error('[secubox-p2p] autoRegisterAll: API error'); return; }
const msg = `[secubox-p2p] Auto-register: activated ${res.activated||0}, requested ${res.requested||0}` +
(res.pending ? `, ${res.pending} pending approval` : '') +
(res.errors && res.errors.length ? `, ${res.errors.length} error(s)` : '');
console.log(msg);
loadServices();
}
async function requestAccess(sid) {
const res = await apiPost('/services/' + encodeURIComponent(sid) + '/request', {});
if (!res) { console.error('[secubox-p2p] requestAccess: API error'); return; }
console.log('[secubox-p2p] requestAccess:', res.status === 'ok' ? 'Subscription requested' : (res.error || 'failed'));
loadServices();
}
async function activateService(sid) {
const res = await apiPost('/services/' + encodeURIComponent(sid) + '/activate', {});
if (!res) { console.error('[secubox-p2p] activateService: API error'); return; }
console.log('[secubox-p2p] activateService:', res.status === 'ok' ? 'Activated' : (res.error || 'failed'));
loadServices();
}
// Load Threats