mirror of
https://github.com/CyberMind-FR/secubox-deb.git
synced 2026-07-29 15:37:03 +00:00
0ca16778cc
148 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
CyberMind
|
0ca16778cc
|
feat(secubox-sentinelle-gsm): v0.3.1 — L3 decode + scoring engine + baseline + qualified alerts (closes #349) (#350)
* docs(plan): #349 sentinelle-gsm v0.3.1 L3 + scoring + baseline (ref #349) * feat(sentinelle-gsm): GsmtapListener exposes raw_l3 payload bytes (ref #349) v0.3.0's Observation carried only header metadata (arfcn, frame_nr, channel, sub_type), which was enough to prove the GSMTAP pipe but not to drive L3-aware scoring. v0.3.1 needs the post-header payload to decode BCCH System Information (LAC/CI/MCC/MNC) and CCCH paging requests (IMSI/TMSI for the privacy-hashed subscriber digest). Slice the L3 payload from the datagram via data[hdr["hdr_len"]:] in _on_datagram() and surface it as a new Observation field `raw_l3: bytes = b""`. Default empty bytes keeps every existing caller backward compatible — only downstream decoders that opt in will consume it (wired in Task 5). Test: existing test_listener_receives_a_datagram now appends a known 18-byte payload and asserts obs.raw_l3 matches verbatim. * feat(sentinelle-gsm): l3_decode — BCCH SI + CCCH paging request parsing (ref #349) Pure-Python TLV decoder for the minimal L3 subset needed by the IMSI-catcher scoring engine: BCCH System Information Types 3/4/6 and CCCH Paging Request Types 1/2/3. No scapy dependency — the parser operates directly on the post-GSMTAP-header bytes exposed by Observation.raw_l3 (Task 1). Privacy invariant : the public dataclasses (CellInfo, PagedIdentity, ParsedPagingRequest, ParsedFrame) NEVER carry plaintext IMSI/TMSI/IMEI. The internal _try_extract_mobile_id() returns plaintext bytes; _parse_paging() immediately hands them to Anonymizer.anonymize() and only the HMAC-truncated subscriber_hash escapes into PagedIdentity. The plaintext goes out of scope at function return. Permissive parsing : every helper returns an empty CellInfo() / None on unexpected layouts rather than raising — real GSMTAP frames have surprising structural variants and exceptions in the consume loop would crash livemon ingestion. Tests : 9 cases (6 from the plan + 3 added for coverage). BCD encoder/decoder round-trip helpers build synthetic frames programmatically rather than hand-crafting bytes, which also validates the TS 24.008 §10.5.1.3/4 nibble ordering. * feat(sentinelle-gsm): baseline.py — operator-baseline learning + cell_baseline table (ref #349) Adds CellBaseline wrapper + cell_baseline table colocated in observations.db. Cells graduate to baseline once observed >= LEARN_THRESHOLD (default 3) times. Key design choices: * LEARN_THRESHOLD = 3 by default — three sightings before a cell is considered part of the operator's legitimate baseline. * set_learn_mode(seconds) skips the count entirely — every cell observed inside that explicit-learn window is INSERTed with learn_count = 3 and graduates on first sighting (used by the operator's calibration sweep). * CellBaseline accepts a raw sqlite3.Connection (NOT an ObservationsDB instance) so it stays decoupled from observations.py's public surface while still sharing the same .db file. * cell_baseline table is created idempotently (IF NOT EXISTS) in ObservationsDB.__init__, so pre-existing observations.db files keep working after upgrade. Privacy invariant: BaselineCell + cell_baseline have NO subscriber-id columns — paged identities never enter the baseline path. Tests (5/5 passing): - test_first_consider_starts_at_1_not_baseline_yet - test_three_considers_graduate_to_baseline - test_learn_mode_graduates_first_consider - test_consider_updates_metadata_via_coalesce - test_list_orders_by_last_learned_desc Full pkg sweep: 62 passing (57 prior + 5 new). * feat(sentinelle-gsm): scoring_engine — 8 heuristics + thresholds (ref #349) Replaces the v0.1 shape-only scoring.py stub with the full v0.3.1 detector brain. ScoringEngine wires against CellBaseline + L3Decode (Tasks 2-3) and runs every parsed frame through 8 heuristics, summing the triggered contributions into a CellScore clamped to [0, 100]. Heuristics + default scores (sum-clamped to 100): 1. cipher_downgrade 40 A5/X observed < baseline.cipher_a5 2. ghost_bts 35 cell unknown to baseline + paging seen 3. identity_mismatch 30 observed (mcc, mnc) != baseline pair 4. relocalization_storm 25 > 3 distinct LACs in 60s on cell 5. identity_request_abuse 35 > 2 paging reqs in 300s per hash 6. anomalous_neighbours 15 neighbour list went non-empty -> empty 7. t3212_out_of_band 15 T3212 outside [1, 60] min 8. orphan_arfcn 20 FR carrier announces ARFCN outside plan Rolling-window state (in-memory only, no DB persistence in v0.3.1): _lac_window cell_id -> deque[(ts, lac)] _idreq_window subscriber_hash -> deque[ts] _neighbour_baseline cell_id -> set[int] (cumulative ARFCNs) Thresholds: * DEFAULT_THRESHOLDS class constant carries per-heuristic config dicts. * update_thresholds() does a per-heuristic deep-merge so callers can pass partial updates like {"cipher_downgrade": {"score": 55}} without clobbering "enabled" or any other unspecified field. * thresholds() returns a deep-copy so callers can't mutate engine state. Privacy invariant kept: identity_request_abuse keys windows on subscriber_hash strings (HMAC-trunc, supplied by l3_decode.Anonymizer), never plaintext IMSI/TMSI. Reasons truncate the hash to 8 chars defensively. France GSM-900 ARFCN plan is approximate and permissive: unknown (mcc, mnc) pairs return triggered=False to avoid false-positives. Dropped lib/sentinelle_gsm/scoring.py — the v0.1.0 shape-only stub (empty Baseline + score()=0) is replaced. No other module imported it. Tests: * api/tests/test_scoring_engine.py — 11 tests, one per heuristic (plus permissive-unknown-carrier guard), one for evaluate() aggregation/clamping, one for update_thresholds() deep-merge. * Full sweep: 73 passed (was 62; +11 new). * feat(sentinelle-gsm): consume loop wires L3 + scoring + trusted match → Alert (ref #349) - Extracted `_process_observation(obs)` helper for testability — the consume coroutine now delegates per-frame work so tests can drive synthesized Observations through the full pipeline without binding UDP. - New endpoints: GET /baseline, POST /baseline/learn (sweep window), GET /scoring/thresholds, POST /scoring/thresholds (deep-merge). - Audit log entry on POST /scoring/thresholds via the existing `secubox.sentinelle-gsm.api` logger — surfaces in /journal/stream. - Alert emission decision tree: score < threshold → drop (no write) score >= threshold + match → one labeled alert per matched trusted phone score >= threshold + no match → single anomaly-only alert - ALERT_THRESHOLD configurable via SENTINELLE_ALERT_THRESHOLD env (default 60). - TrustedRegistry now hashes the IMSI as `imsi.encode("ascii").hex()` so the persisted token matches the canonical form produced by the L3 decoder's paging-request path. Existing trusted tests don't pin the hash value, so behaviour is unchanged for read-back / lookup-by-id. - 5 new integration tests at api/tests/test_alert_emission.py cover the below-threshold drop, anomaly-only emission, trusted-match labeling, paging-event hash persistence, and audit-log emission on threshold update. Full sweep 78/78 passing. * feat(sentinelle-gsm): webui baseline + scoring panels + trusted_label chip (ref #349) Adds two new webui panels and a trusted-label chip on the alerts row: - #baseline panel: Cell ID / MCC / MNC / LAC / ARFCN / cipher / learn count / last learned table backed by GET /api/v1/sensor/gsm/baseline. "Start learn (5 min)" button arms POST /baseline/learn?sweep_seconds=300 and re-polls the table every 15s for the 5-minute window so cells appear as they graduate. - #scoring panel: per-heuristic enable + score input (0..100, clamped client-side) backed by GET/POST /scoring/thresholds. Form values are cached in _thresholdState so any backend-only fields (heuristic-specific knobs) survive the POST round trip. - Alerts table: trusted_label is now rendered as a violet pill via the new .trusted-chip class (replaces the older .target-pill markup for the same data path). Local navigation gets two new anchors (Baseline, Scoring) between Observations and Actions. The 10s scan poll now also refreshes baseline while a scan is running. Wires 4 new event listeners (refresh-baseline, baseline-learn, refresh-scoring, save-scoring). No backend changes; all endpoints already exist (Tasks 1..5). Tests: 78/78 passing. * chore(sentinelle-gsm): bump 0.3.1 + extend privacy invariant tests (closes #349) Append 4 new structural privacy tests under tests/test_privacy_invariant.py to lock the v0.3.1 invariants in place: * test_l3_decode_returns_no_plaintext_fields — CellInfo, PagedIdentity, and ParsedPagingRequest dataclasses must not carry any plaintext subscriber-id field (imsi/tmsi/imei/msisdn/iccid/subscriber_id). * test_paging_request_hashes_paged_identities — feeds a synthetic Paging Request Type 1 with a known TMSI through L3Decode and asserts the plaintext TMSI bytes (as hex) NEVER appear in the resulting PagedIdentity.subscriber_hash. * test_cell_baseline_has_no_subscriber_fields — the BaselineCell dataclass must remain operator-side metadata only: no subscriber_hash, no plaintext identifier. * test_scoring_engine_reasons_contain_no_plaintext_imsi — the free-text reason strings produced by ScoringEngine.evaluate() must never echo a 15-digit token (plaintext-IMSI shape) for typical observations. These four tests pass against the v0.3.1 modules (Tasks 1-6) with no code change — the modules were designed for these invariants from the start, this commit just locks them in regression-test form. Full sweep: 78 -> 82 passing. Changelog: 0.3.0 -> 0.3.1 with the v0.3.1 entry summarising the new l3_decode, baseline, scoring_engine modules, the consume-loop wiring that emits qualified Alerts with trusted_label, the webui panels, and the updated test breakdown (l3_decode x9, baseline x5, scoring_engine x11, alert_emission x5, privacy x4 = 34 new, full sweep 82/82). --------- Co-authored-by: CyberMind-FR <gandalf@Gk2.net> |
||
|
CyberMind
|
5f3d268c05
|
feat(secubox-sentinelle-gsm): v0.3.0 — passive observation pipeline (closes #344 #347) (#348)
* docs(plan): #347 sentinelle-gsm v0.3.0 observation pipeline (ref #344 #347) * feat(sentinelle-gsm): livemon_runner — managed grgsm subprocess (closes #344 ref #347) Introduces LivemonRunner, the asyncio subprocess manager that owns the single grgsm_livemon_headless child for v0.3.0. Device-claim invariant (the load-bearing point that closes #344): the parent service NEVER opens the RTL-SDR itself. The RTL-SDR is claimed ONLY by the spawned grgsm_livemon_headless child, and ONLY for the lifetime of an active scan. start() spawns with `--args=rtl=0 -f <freq>` so gr-osmosdr binds RTL-SDR instead of its UHD default; stop() sends SIGTERM and falls back to SIGKILL after a 5 s timeout, releasing the device cleanly. Concretely this means ad-hoc tooling (rtl_test, rtl_adsb, kalibrate) keeps working alongside an idle secubox-sentinelle-gsm service — the race the v0.1 livemon.py USB-detect stub created is gone, because the parent has no reason to touch the dongle outside an explicit scan. Stderr from the child is drained into a 16 KiB-capped ring buffer so status() can surface the last 2 KiB without unbounded memory growth, and double-start() raises RuntimeError instead of silently leaking processes. Tests cover: initial-state, RTL args propagation to create_subprocess_exec, double-start refusal, SIGTERM-on-stop + state clear, and stop-as-noop when nothing is running. asyncio.create_subprocess_exec is fully mocked — no real grgsm process is spawned by the test suite. Test sweep: 5 new + 29 existing = 34 passing. * feat(sentinelle-gsm): gsmtap_listener — async UDP 4729 + GSMTAP v2 parse (ref #347) Task 2 of v0.3.0 observation pipeline. - `lib/sentinelle_gsm/gsmtap_listener.py`: asyncio DatagramProtocol bound to 127.0.0.1:4729 (configurable). Tiny manual 16-byte GSMTAP v2 header parser — no scapy runtime dep on the hot path. scapy stays reserved for the deeper L3 decode (BCCH/CCCH paging) landing in v0.3.1. - `Observation` dataclass exposes ts/arfcn/frame_nr/channel/sub_type plus optional lac/ci/mcc/mnc/cell_id/subscriber_hash. No plaintext-id field; `subscriber_hash` is HMAC-only (populated when L3 decode lands). - Bounded queue (maxsize=2048); on QueueFull we drop silently — observations are stochastic, backpressure to the radio would be worse. - 4 tests: minimum-valid parse, too-short reject, wrong-version reject, end-to-end UDP roundtrip on port 47291 (avoids colliding with a running grgsm on 4729). All tests pass: 4/4 new, 38/38 sweep. * feat(sentinelle-gsm): observations.py — SQLite sightings + paging_events (ref #347) Adds ObservationsDB persistence layer for the v0.3 observation pipeline. * Sighting (cell_id PK, mcc/mnc/lac/ci/arfcn, first_seen/last_seen, sighting_count) — UPSERT semantics: INSERT on new cell_id, else bump sighting_count + refresh last_seen with COALESCE on optional fields so partial updates don't clobber previously-observed values. * PagingEvent (ts, cell_id, subscriber_hash, request_type) — seeds the paging history table that v0.3.1 will hash-match against trusted_phones. Schema deliberately omits plaintext IMSI/TMSI/IMEI fields. * _guard_plaintext() mirrors the regex guard in alert_sink.AlertSink.write (same \b\d{15}\b pattern, same "plaintext-IMSI shape detected — refusing write" error contract). Applied on every write path: upsert_sighting guards cell_id; record_paging guards both cell_id and subscriber_hash. * Two indexes on paging_events (ts, cell_id) for the v0.3.1 lookup path. * 4 tests cover the contract: new-insert, bump-count, hash accept, and the plaintext-IMSI refusal. Full sweep: 42 passing (38 prior + 4 new). * feat(sentinelle-gsm): API — /scan/start /scan/stop /scan/status /observations (ref #347) Task 4 of the v0.3.0 observation-pipeline plan. Wires the LivemonRunner + GsmtapListener + ObservationsDB introduced in Tasks 1–3 to four new HTTP endpoints behind the existing nginx + Authelia JWT gate. - POST /scan/start {freq} → bind GSMTAP UDP listener, spawn grgsm_livemon_headless on freq, start a background consume task that drains Observations into the SQLite store - POST /scan/stop → cancel the consume task FIRST so no orphan subprocess is left behind, then SIGTERM the runner, then close the listener socket (the CHILD owns the RTL-SDR, never the parent) - GET /scan/status → current runner state (running / pid / freq / started_at / stderr_tail) - GET /observations?limit → recent cell sightings, newest first; limit clamped to [1, 1000], default 200 Synthetic cell_id formation until v0.3.1's L3 BCCH decode lands: `f"arfcn-{obs.arfcn}-ch-{obs.channel}"`. The ObservationsDB schema already accepts NULL for the operator quadruple (mcc/mnc/lac/ci) so nothing downstream has to change when the real cell_id arrives. Three new tests in api/tests/test_scan_api.py exercise the happy path with mocked LivemonRunner + mocked GsmtapListener (so no gr-gsm binary is spawned and no UDP socket is bound) plus a real ObservationsDB under tmp_path so GET /observations exercises the real SQLite path. Fixture clears `_consume_task` between tests to keep them order-independent. Test sweep: 42 (existing) + 3 (this) = 45 passing. * feat(sentinelle-gsm): webui scan control + observations panels (ref #347) Adds two new panels to the standalone MIND webui: * Scan control — Start/Stop buttons, freq input (default 925.4M = E-GSM 900 ARFCN 975), 4-cell status row (state dot, freq, pid, started_at), and a collapsible <details> showing the last 2 KB of stderr from grgsm_livemon_headless. Start sends {freq: <value>} to /scan/start. * Observations — live table of cell sightings with columns Cell ID, ARFCN, MCC, MNC, LAC, CI, First seen, Last seen, Sightings. Badge in the panel header tracks the current count. JS additions (sentinelle.js): * loadScanStatus / startScan / stopScan / loadObservations helpers * 3 new event listeners (btnScanStart, btnScanStop, btnRefreshObs) * Background interval (10s) that polls /scan/status always and /observations only while a scan is running — keeps the post-scan terminal state on screen without hammering sqlite. CSS additions (sentinelle.css): * .scan-status-row (4-col grid inside a panel, matches .status-strip) * .scan-stderr / .scan-stderr-details (monospace tail block, MIND palette) * Responsive collapse to 2 cols then 1 col matching the existing breakpoints. API surface untouched — Tasks 1-4 already shipped /scan/start, /scan/stop, /scan/status and /observations under /api/v1/sensor/gsm/. Test sweep still 45 passing. * build(sentinelle-gsm): merge detect_rtlsdr_usb into livemon_runner; drop v0.1 stub; add scapy + gr-gsm deps (ref #344 #347) Co-locate RTL-SDR USB detection with the grgsm_livemon orchestrator. - Move RTLSDR_USB_IDS + detect_rtlsdr_usb() from livemon.py into livemon_runner.py. The function is pure sysfs (reads /sys/bus/usb/devices/*/idVendor + idProduct) — it does NOT claim the USB device, so it was never the source of #344. The #344 bug was fixed by Task 1's LivemonRunner model (child-only device claim via subprocess.create_subprocess_exec). - Delete lib/sentinelle_gsm/livemon.py — the v0.1 stub also carried a useless NotImplementedError docstring for listen_gsmtap_udp, fully superseded by gsmtap_listener.py (Task 1). - api/main.py: collapse the two-line "from sentinelle_gsm.livemon ... / from sentinelle_gsm.livemon_runner ..." block into a single import pulling LivemonRunner + detect_rtlsdr_usb from livemon_runner. - debian/control: promote python3-scapy from Recommends to Depends (runtime requirement once the v0.3.1 L3 decode path ships); keep gr-gsm in Recommends (subprocess that runs scans). dpkg-buildpackage: ok (secubox-sentinelle-gsm_0.2.3-1~bookworm1_all.deb). dpkg-deb -c | grep livemon: only livemon_runner.py + test_livemon_runner.py, no v0.1 stub. pytest api/tests/ tests/: 45 passed. * chore(sentinelle-gsm): bump 0.3.0 + extend privacy invariant tests (closes #344 #347) Privacy invariant coverage extended for the v0.3.0 observation pipeline: * test_observation_has_no_plaintext_fields — Observation dataclass (gsmtap_listener) exposes ONLY subscriber_hash, never plaintext IMSI/TMSI/IMEI/MSISDN/ICCID/subscriber_id. * test_paging_event_only_stores_hash — PagingEvent dataclass (observations) has no plaintext-identifier fields and exposes subscriber_hash. * test_observations_db_refuses_plaintext_imsi — ObservationsDB.record_paging raises ValueError("plaintext-IMSI ...") when handed a 15-digit subscriber_hash (write-time shape guard). Version bump 0.2.3 → 0.3.0 with the full v0.3.0 changelog: livemon_runner.py (RTL-SDR subprocess wrapper, closes #344) + gsmtap_listener.py (asyncio UDP 4729) + observations.py (SQLite sightings/paging_events) + /scan API + webui panels + scapy promoted to Depends + dropped v0.1 livemon.py stub. 48/48 tests passing. --------- Co-authored-by: CyberMind-FR <gandalf@Gk2.net> |
||
|
CyberMind
|
95550edc4e
|
feat(secubox-sentinelle-gsm): v0.2.0 — standalone WebUI + local alerts (SSE + browser notification) (closes #340) (#341)
* docs(plan): #340 secubox-sentinelle-gsm v0.2 standalone webui + local alerts (ref #340) * feat(sentinelle-gsm): alert_sink — SQLite history + SSE broadcast hub (ref #340) Introduces lib/sentinelle_gsm/alert_sink.py: a SQLite-backed alert history with an in-memory asyncio pub/sub hub that fans new alerts out to live Server-Sent Events subscribers. The Alert dataclass carries only privacy-safe fields — `subscriber_hash` is HMAC-truncated (Anonymizer), never plaintext. AlertSink.write() enforces this with a load-bearing privacy guard: any string field matching `\b\d{15}\b` (plaintext IMSI shape) triggers ValueError and the row is refused. SQLite is opened with check_same_thread=False so the FastAPI app can read/write from multiple coroutines safely. Adds api/tests/test_alert_sink.py with 5 tests covering write/list roundtrip, the plaintext-IMSI privacy guard, HMAC acceptance, live subscriber fan-out, and SSE chunk format. 5 passed locally. * feat(sentinelle-gsm): trusted phones registry — HMAC-hashed IMSI + label (ref #340) Privacy contract: - Plaintext IMSI is accepted by add() ONCE, immediately HMAC-hashed via the existing sentinelle_gsm.observer.Anonymizer.anonymize() helper. - The plaintext never persisted, never logged, never returned by any method. After add() returns, plaintext_imsi is out of scope. - The on-disk store contains only {id, imsi_hash, label, added_at}. Storage format: JSON via the stdlib `json` module instead of TOML. `python3-tomli-w` (the only writer for the new tomllib reader) is not in Debian bookworm — it first ships in trixie. python3-toml (legacy) writes TOML but is deprecated. JSON is stdlib, atomically renderable, and human-grep-able, which is what a trusted-phones registry needs. Anonymizer integration: uses .anonymize() (HMAC-SHA256, 16 hex chars truncated). The plan referenced .hash_subscriber_id() — that method does not exist on Anonymizer in observer.py; .anonymize() is the canonical hashing entry point. Tests: 6/6 passing - test_add_hashes_and_does_not_store_plaintext (privacy invariant enforced: plaintext IMSI never appears on disk) - test_lookup_by_hash_finds_added_phone - test_lookup_by_hash_returns_none_for_unknown - test_delete_removes_entry - test_invalid_imsi_rejected - test_persistence_across_instances * feat(sentinelle-gsm): API — /alerts (history + SSE + test) + /trusted CRUD (ref #340) Adds 6 endpoints under the existing /api/v1/sensor/gsm/ prefix (nginx-added; mounted at root of the FastAPI app): GET /alerts — paginated SQLite-backed history (query: ?limit=100&since=<epoch>) GET /alerts/stream — SSE live feed; Cache-Control: no-cache, X-Accel-Buffering: no, Connection: keep-alive so nginx and HTTP caches don't buffer it POST /alerts/test — operator manual trigger; ValueError from the AlertSink privacy guard (plaintext-IMSI shape) surfaces as HTTP 500 GET /trusted — list trusted phones (HMAC + label, no plaintext) POST /trusted — body {imsi, label}; ValueError → 400 DELETE /trusted/{id} — unknown id → 404 Wiring: * AlertSink + TrustedRegistry are module-level singletons initialised on FastAPI startup so tests can override via monkeypatch. * `require_jwt` is a no-op dependency stub — real JWT enforcement is done by nginx + Authelia in front of the Unix socket. The stub exists so test code (and future host-direct callers) can swap auth via `app.dependency_overrides[require_jwt]`. * Anonymizer is loaded via a new `_get_anonymizer()` helper that reads /etc/secubox/secrets/sentinelle-gsm-hmac when present and falls back to an ephemeral key otherwise. * trusted.json path is /etc/secubox/sentinelle-gsm/trusted.json (the Task-2 deviation from the plan's TOML; bookworm lacks python3-tomli-w). * version bumped 0.1.0 → 0.2.0 in the FastAPI metadata only; the debian/changelog bump lands in Task 6. Tests (7 new in api/tests/): * test_alerts_api.py — POST /alerts/test happy path, GET history roundtrip, plaintext-IMSI guard → 500, SSE route registration (via app.routes + /openapi.json, not by opening the stream — the async generator awaits forever on q.get(), incompatible with sync TestClient iteration; SSE chunk format is covered by test_alert_sink.py::test_stream_emits_sse_format). * test_trusted_api.py — add/list/delete roundtrip with hash-not- plaintext invariant, invalid IMSI → 400, unknown id → 404. Run: pytest api/tests/test_alerts_api.py api/tests/test_trusted_api.py Result: 7 passed. * feat(sentinelle-gsm): standalone webui — live alerts + browser Notification API (ref #340) Adds the Task 4 standalone WebUI for sentinelle-gsm at packages/secubox-sentinelle-gsm/www/sentinelle/: - index.html — canonical SecuBox scaffold (body flex, aside.sidebar 220px fixed with section anchors, main reserves the global menu bar with padding-top: calc(48px + 1.5rem)). Four panels: status strip, alerts table (aria-live=polite), trusted phones table, actions. - sentinelle.css — MIND palette (--mind-violet: #3D35A0) per Charte SecuBox, Space Grotesk titles + JetBrains Mono data/code, modal with z-index 1500 (above the global-menu-bar z-index 998), toast banner, score chip colour grading (low/med/high), responsive breakpoints. - sentinelle.js — vanilla single-IIFE module, no framework, no CDN. EventSource consumer of /api/v1/sensor/gsm/alerts/stream (auto-reconnect surface), CRUD wrappers for /alerts /alerts/test /trusted /trusted/{id}, Browser Notification API (requested from a user-gesture handler), beep synthesised on-the-fly via Web Audio API (no asset), localStorage mute toggle, modal-based add flow, ESC/backdrop close, accessible aria-live region for the alerts list. All API calls hit root-relative /api/v1/sensor/gsm/* — Task 5 will mount the nginx route. Local smoke test booted uvicorn against the API and verified the HTML/CSS/JS serve and all four CRUD/test routes respond as the UI expects. * feat(sentinelle-gsm): nginx route /sentinelle/ + MIND menu entry + .install (ref #340) * nginx/sentinelle-webui.conf: alias for /sentinelle/ (static webui) + SSE-tuned override for /api/v1/sensor/gsm/alerts/stream. Uses `^~` literal-prefix match so it wins precedence over the shorter /api/v1/sensor/gsm/ prefix in sentinelle-gsm.conf and against any future regex locations in the same server scope. Re-sets proxy_read_timeout / proxy_send_timeout to 24h after the common secubox-proxy include (which forces 30s) so long-lived SSE connections survive. * menu.d/45-sentinelle.json: MIND-category entry (icon: satellite, order: 45). Category emitted as lowercase `mind` to match the canonical schema used by every other package menu.d entry. * debian/secubox-sentinelle-gsm.install: ship the new files. The package uses a hand-rolled override_dh_auto_install in debian/rules, so the install file is documentation; debian/rules is extended with explicit cp steps for nginx/sentinelle-webui.conf and www/sentinelle/. menu.d already had a wildcard copy. * chore(sentinelle-gsm): bump 0.2.0 + extend privacy invariant tests (closes #340) - Append 3 new privacy invariant tests: * Alert dataclass has no plaintext IMSI/IMEI/TMSI/MSISDN/ICCID fields * TrustedPhone dataclass only stores imsi_hash (never plaintext) * AlertSink.write() raises ValueError on any 15-digit token in a textual field — non-regression guard for the OPAD-doctrine privacy invariant. - Bump debian/changelog to 0.2.0-1~bookworm1 with the full Task 1-6 feature summary: alert_sink, trusted registry, REST routes, WebUI, nginx + Hub menu integration. Notes the JSON-over-TOML deviation (python3-tomli-w not in bookworm). - 29/29 tests pass: 5 alert_sink + 4 alerts_api + 3 trusted_api + 6 trusted_registry + 11 privacy_invariant (8 prior + 3 new). --------- Co-authored-by: CyberMind-FR <gandalf@Gk2.net> |
||
|
CyberMind
|
832141cd0a
|
feat(secubox-streamlit): v1.2.2 — per-app idle timeout + wake-on-demand (closes #331) (#334)
* docs(plan): #331 streamlit idle-timeout + wake-on-demand (ref #331) * feat(streamlit): add idle-tracking helpers to streamlitctl (ref #331) Add the foundation for per-app idle timeout / wake-on-demand: - IDLE_STATE_DIR constant at /var/lib/secubox/streamlit/idle - _idle_config: read [idle] settings from /etc/secubox/streamlit.toml with safe defaults - _app_running: check whether an app is listening on its port INSIDE the LXC (existing inline ss check is host-side and unreliable) - _app_active_conns: count ESTABLISHED connections to an app's port - _app_last_active / _app_touch_active: per-app last-activity epoch tracked via state-file mtime Wire cmd_app_start to touch the state file on every successful start so a freshly-started app does not get immediately reaped by the upcoming idle-check. All helpers degrade gracefully when the LXC is down (lxc_running guard) and the helpers themselves do not change any existing behaviour — they are purely additive. * feat(streamlit): streamlitctl app idle-check — stop idle apps past timeout (ref #331) Add cmd_app_idle_check that iterates apps under $APPS_PATH and stops those that have had zero ESTABLISHED connections for longer than the configured timeout_minutes (default 30, from [idle] in streamlit.toml). Apps with active connections get their state file touched; apps with no state file yet are touched too (grace period). Master switch via [idle] enabled in /etc/secubox/streamlit.toml. Wire idle-check + wake into the `app` subcommand dispatcher; wake's function (cmd_app_wake) lands in the next commit (dead branch until then). Usage line updated to include both new subcommands. * feat(streamlit): streamlitctl app wake — lazy start + poll for port (ref #331) Adds cmd_app_wake <name> [wait_seconds] (default 30s): - returns 0 if already running OR if the app comes up within wait_seconds - returns 1 on start failure or poll timeout - returns 2 on misuse (missing name, unknown app) Uses _app_running for readiness probing and _app_touch_active to refresh the idle marker on a successful wake. Polls every 1s for portability. The case-block dispatcher (idle-check / wake) was already wired in Task 2. * feat(streamlit): systemd idle-sweep timer (every 5 min) (ref #331) Ship secubox-streamlit-idle.service (oneshot, runs `streamlitctl app idle-check`) and secubox-streamlit-idle.timer (OnBootSec=5min, OnUnitActiveSec=5min) so idle Streamlit apps are auto-suspended without a separate cron. Wire the units through debian/secubox-streamlit.install (the package uses a hand-rolled override_dh_auto_install, so dh_installsystemd's auto-detection by package.suffix name doesn't pick them up — explicit .install is needed). Enable/disable the timer in postinst/prerm alongside the main service. * feat(streamlit): POST /apps/{name}/wake — lazy start API endpoint (ref #331) * feat(streamlit): default [idle] config block (timeout 30 min) (ref #331) * chore(streamlit): bump to v1.2.2 with idle-timeout + wake summary (closes #331) --------- Co-authored-by: CyberMind-FR <gandalf@Gk2.net> |
||
| e562446383 |
docs(superpowers): capture 3 specs from operator drop 2026-05-20
* secubox-zigbee-mqtt-iot-stack — v2.4.0 rewrite-in-place spec for the IoT radio stack (Mosquitto, zigbee2mqtt, Sonoff CC2652P, 4R buffer) * secubox-iot-hub-webui — multi-connector WebUI dashboard spec (offline-first, vanilla JS, FastAPI router /api/v1/iot-hub) * secubox-sso-bridge — SSO doctrine (Authelia LXC + auth_request pattern for SSO-less backends, health-banner percolation) Saved verbatim from the operator briefing. Implementation deferred until v2.4.0 alignment is reached on the underlying modules. |
|||
|
CyberMind
|
a58b6fa55d
|
fix(lxc-modules): 9 board-deployment fixes for grafana/yacy/rustdesk install-lxc.sh + nginx wiring (#238)
Field-deploying v2.11.0's three new LXC modules on arm64 MOCHAbin
(gk2 192.168.1.200) surfaced nine bugs that the local smoke build
doesn't exercise (no lxc-create in dpkg-buildpackage). All three
modules now reach overall=green on the board with the canonical hub
vhost (admin.gk2.secubox.in) routing /api/v1/<m>/* and /<m>/ correctly.
install-lxc.sh (all 3 modules)
1. lxc-create -t download (the legacy `debian` template fails on
bookworm unprivileged containers).
2. ensure_masquerade() — nftables MASQUERADE rule for 10.100.0.0/24,
idempotent. systemd-networkd's IPMasquerade=ipv4 doesn't land on
appliances using ifupdown/NetworkManager.
3. ensure_resolv() — seed /etc/resolv.conf via lxc-attach AFTER
start_lxc + wait_for_network (rootfs is owned by mapped uid 100000,
host root can't write directly). Unlink the symlink the download
template ships (→ /run/systemd/resolve/stub-resolv.conf which
doesn't exist before systemd-resolved is up).
install-lxc.sh (grafana)
4. gpg --dearmor on apt.grafana.com key (signed-by= needs binary,
wget delivers ASCII-armored).
install-lxc.sh (yacy)
5. YACY_RELEASE_URL: v1.940 → v1.941 (old URL now returns 404).
install-lxc.sh (rustdesk)
6. Detect host arch (arm64/amd64/armhf) and pull the matching
RustDesk server build. Hardcoded amd64 binary failed with
"Exec format error" on the arm64 MOCHAbin.
7. Flatten any of amd64/, arm64v8/, armv7/ subdirs the zip ships.
debian/secubox-<m>.service (all 3 modules)
8. ExecStartPre prefix with `+` (run as root). The mkdir + chown
pre-steps run as User=secubox by default, chown fails ("Operation
not permitted") and the service can't start.
debian/rules + nginx/<m>.conf (all 3 modules)
9. Install nginx snippet to BOTH /etc/nginx/secubox.d/ AND
/etc/nginx/secubox-routes.d/. The canonical hub vhost
(sites-enabled/webui.conf serving admin.gk2.secubox.in) only
includes routes.d/, NOT secubox.d/ which is only included by the
legacy sites-available/secubox vhost. Also switch the
/api/v1/<m>/ location from `proxy_pass http://unix:.../sock:/` to
explicit `rewrite ^/api/v1/<m>/(.*)$ /$1 break; proxy_pass
http://unix:.../sock;` — matches the canonical pattern used by
secubox-users; the previous syntax did not strip the prefix so
FastAPI saw the full path and returned 404 on every route.
Docs
* docs/MODULE-GUIDELINES.md §7 extended with mandatory lifecycle
verbs install / repair / wizard / reload / uninstall. Repair detects
+ fixes drift idempotently; wizard chains interactive seed + install;
uninstall asks for confirmation, removes LXC + state + secrets.
Implementation in grafanactl/yacyctl/rustdeskctl deferred to v2.12.0
(currently install + reload are wired, the rest is on the v2.12.0
backlog).
* docs/superpowers/specs/2026-05-20-secubox-wall-ep06.md (#236) —
Quectel EP06-E modem-based rogue-base-station sensor (WALL layer).
* docs/superpowers/specs/2026-05-20-secubox-sentinelle-gsm.md (#237) —
RTL-SDR + gr-gsm passive RX-only false-BTS detection (MIND layer,
feeds WALL/OPAD). Privacy-by-design HMAC-truncated identifiers.
* .claude/{WIP,TODO}.md refreshed with v2.11.1 fix trail + v2.12.0
target + P2 sensor backlog.
Package version bump: 1.0.0 → 1.0.7 (grafana/rustdesk) / 1.0.7 (yacy),
all three end at the same .deb revision suitable for v2.11.1 tag.
Validated end-to-end on board gk2:
grafanactl status → overall=green (lxc + grafana-server + host-api all up,
admin/<password> at /etc/secubox/secrets/grafana-admin,
6 dashboards + secubox-metrics datasource provisioned)
yacyctl status → overall=green (lxc + JVM + YaCy v1.941 standalone,
admin password at /etc/secubox/secrets/yacy-admin)
rustdeskctl status → overall=green (lxc + hbbs + hbbr + host-api,
PSK at /etc/secubox/secrets/rustdesk-key,
nftables DNAT 21116/udp installed)
https://admin.gk2.secubox.in/api/v1/grafana/healthz → HTTP 200
https://admin.gk2.secubox.in/api/v1/yacy/healthz → HTTP 200
https://admin.gk2.secubox.in/api/v1/rustdesk/healthz → HTTP 200
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
|
||
| e170ef98c1 | docs: correct LXC bridge name br-secubox -> br-lxc (matches mail/gitea convention) | |||
| 71f631003d |
docs: MODULE-GUIDELINES + grafana/yacy/rustdesk LXC plan
- docs/MODULE-GUIDELINES.md (new) — canonical reference for authoring a secubox-<module> package: file structure, LXC layout, WebUI conventions, nginx wiring, Debian packaging, CTL three-fold, FastAPI shape, tests, validation checklist. Sister doc to docs/grammar.md and docs/UI-GUIDE.md. - docs/grammar.md — cross-reference MODULE-GUIDELINES.md in the header sister-docs block. - docs/MODULES.md — point new-module authors at MODULE-GUIDELINES.md. - docs/superpowers/plans/2026-05-20-grafana-yacy-rustdesk-lxc-modules.md (new) — concrete implementation plan for three new LXC-hosted modules: secubox-grafana (#229), secubox-yacy (#230), secubox-rustdesk (#231). All three follow the new MODULE-GUIDELINES.md. IP allocations 10.100.0.{70,80,90}. |
|||
|
CyberMind
|
53a630e9f8
|
docs: codify the SecuBox CTL grammar (closes #216) (#217)
Permanent documentation for the 8-verb / 8-layer modular tools box that emerged across #173, #176, #180-#184, #190, #212. Three new docs + one README section, all bearing the 1991 attribution. docs/grammar.md Canonical conceptual frame: layered map, 8 verbs in a table, composing-the-grammar worked examples, naming conventions, philosophical roots. HOWTO-grammar.md Concrete 6-step walkthrough for adding a 9th verb (~1h budget): issue framing, worktree, CTL skeleton (Bash + Python), FastAPI mirror, Debian packaging, nginx route under secubox-routes.d/, live test on the board, commit/PR conventions, common pitfalls drawn from real incidents (#163 secubox.d vs secubox-routes.d, #173 LXC default path, #176 hook trigger, #194 CAP_NET_ADMIN, #180 naming). wiki/CTL-Grammar.md Public-facing wiki page mirrored to the GitHub wiki repo (per the wiki-sync pattern established for Acknowledgments). Embeds Punk Exposure (Peek/Poke/Emancipate) as the seed concept with 1991 attribution. README.md New section under Features summarising the grammar with the 9-row layer table and a pointer to docs/grammar.md + HOWTO-grammar.md. Attribution recorded on all three documents: copyright spiritual concept Gerald Kerma (GK2), 1991, Notre-Dame-du-Cruet, Savoie. The CTL is the user's voice. The modular tools box is their organ. The language is the way they say *this is mine, this is mine, this is mine*. Co-authored-by: CyberMind-FR <gandalf@Gk2.net> |
||
|
CyberMind
|
26b747dec4
|
feat(droplet): static-publisher CLI dropletctl — port from OpenWrt (closes #196) (#199)
* docs(spec): dropletctl design — port from OpenWrt, static-only v1.1.0 (ref #196) Brainstormed scope: - in: publish/remove/rename/list, HTML+ZIP+tarball, idempotent overwrite, full delegation to metablogizerctl site publish, bats unit tests with PATH-shimmed stubs, debian rules + changelog + Depends bump - out: streamlit, hexo, gitea git-versioning, atomic rollback (v1.2.0+) Captures the 5-section design just walked through: 1. Architecture (thin orchestrator → metablogizerctl) 2. Components & interfaces (file layout, CLI surface, output contract) 3. Data flow + error handling (10-step happy path, 6 exit codes) 4. Testing (10 bats cases, fixtures, PATH-shimmed delegate stubs) 5. Packaging + deploy (debian/rules + control + changelog) Per project convention saved under docs/superpowers/specs/. Awaiting user review before invoking superpowers:writing-plans. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(plan): dropletctl implementation plan — 11 tasks, TDD, 13 bats cases (ref #196) Bite-sized TDD tasks per superpowers:writing-plans convention: Task 1 scaffold bats + fixtures + delegate stubs + CLI skeleton Task 2 publish arg validation (3 error cases) Task 3 name sanitization + single HTML happy path + metablogizerctl delegate Task 4 ZIP extraction with single-nested-dir unwrap Task 5 tarball extraction + plain-directory input Task 6 pin idempotent re-publish behavior Task 7 remove subcommand + TOML section delete Task 8 rename subcommand Task 9 list subcommand Task 10 debian packaging (rules + control + changelog 1.0.1 -> 1.1.0) Task 11 lint + final 13/13 bats sweep Spec coverage: every spec requirement maps to a task. Spec deviation noted: spec said Depends >=2.0.0 but live metablogizer is 1.1.1; plan uses >=1.1 so apt doesn't refuse. Hardware deploy on gk2 explicitly deferred to a post-merge manual step (script included). Self-reviewed: zero placeholders, no 'similar to Task N' weasel words, function names consistent (sanitize_name / stage_content / unwrap_single_nested / upsert_toml_site / remove_toml_site / default_domain / cmd_publish/remove/rename/list). Caught + fixed a spec coverage gap (plain-directory input was omitted) in Task 5. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(droplet): scaffold dropletctl CLI + bats harness (ref #196) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(droplet): publish arg validation (3 error cases) (ref #196) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(droplet): name sanitization + HTML staging + metablogizerctl delegation (ref #196) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(droplet): ZIP extraction with single-nested-dir unwrap (ref #196) * feat(droplet): tarball + plain-directory input (ref #196) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test(droplet): pin idempotent re-publish behavior (ref #196) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(droplet): remove subcommand + TOML section delete (ref #196) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(droplet): rename subcommand (ref #196) * fix(droplet): rename — collision guard + same-name guard + delegate-asymmetry comments (ref #196) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(droplet): list subcommand (ref #196) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * build(droplet): bump to 1.1.0 + add rsync/unzip/python3/metablogizer runtime deps (ref #196) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(droplet): rename stores bare base domain + reject invalid domain in publish (ref #196) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test(droplet): strengthen rename-domain test to exercise legacy-FQDN path (ref #196) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: CyberMind-FR <gandalf@Gk2.net> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
CyberMind
|
3435cfa069
|
feat(scripts): add check-dashboard-cache.py lint + CI (closes #147) (#148)
Anti-regression for the double-cache pattern documented in CLAUDE.md (« Performance Patterns — Double Caching »). Catches the bug class that caused PR #146 (secubox-waf 17 % sustained CPU under SOC polling) before it lands. What's added ============ - scripts/check-dashboard-cache.py — AST-based audit. Walks every packages/secubox-*/api/main.py, flags hot dashboard routes (/stats, /alerts, /summary, /dashboard, /metrics, /overview, /events, /history, /recent, /bans, /sessions, /connections) whose handler body does per-request I/O (open, read_text, read_bytes, subprocess.run, Popen) AND doesn't go through a cache (module-level _cache instance + .get() call, OR asyncio.create_task(refresh_*) at startup). * --check : exit 1 on unjustified violations (CI mode) * --report : human-readable table (default) * --json : machine-readable - .cache-lint-allowlist.toml — TOML allowlist for justified exceptions. Each entry needs a daemon + route + justification (issue link expected). Seeded with the 4 current legacy cases: secubox-waf (covered by PR #146 — will drop the entries when merged), secubox-config-advisor /history (admin drill-in, not polled), secubox-netdiag /connections (real-time TCP state), secubox-tor /summary (small file read, follow-up cleanup). - .github/workflows/dashboard-cache-check.yml — mirror of license-check.yml. Runs the lint + the self-tests on PRs that touch packages/**/api/main.py, the lint script, the allowlist, or the workflow itself. - docs/CACHE-PATTERN.md — short remediation guide. Three accepted shapes (read-through, background-refresh, pure in-memory), code example, allowlist syntax, references to the canonical implementations (secubox-crowdsec for read-through, secubox-system for background-refresh). Tests (14/14 pass) ================== - StatsCache pattern detection (compliant cases) - Non-compliant detection: open, subprocess.run, multiple I/O ops - Pure in-memory handler = compliant - Background-refresh task = compliant - Non-hot route (/health) not audited - @router.get treated like @app.get - Allowlist suppresses by exact (daemon, route) key - Allowlist doesn't cross-pollinate across routes - Missing allowlist file = no error - CLI --check exits 1 on unjustified - CLI --check exits 0 when clean - CLI --json output well-formed - **Real repo audit with seeded allowlist passes --check** (sanity gate: catches regression if anyone removes an allowlist entry without fixing the daemon) Baseline against the real packages/ tree ======================================== 122 daemons audited, 118 compliant, 4 with findings (all allowlisted). When PR #146 merges, the secubox-waf entries can be removed from the allowlist — the next lint run will then prove the fix is wired correctly. Follow-up ========= - Drop secubox-waf allowlist entries when #146 merges. - Per-daemon follow-up issues for the remaining 3 (config-advisor, netdiag, tor) — not blocking, allowlist documents the rationale. - Consider extending HOT_ROUTES to /health / /status once the small daemons that call `systemctl is-active` adopt a 10 s cache. Co-authored-by: CyberMind-FR <gandalf@Gk2.net> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
CyberMind
|
e91b9e9ea4
|
feat(eye-remote): multi-gadget DHCP on eye-br0 — Phase 1 (#161)
Phase 1 of issue #158: dnsmasq scoped to eye-br0 with per-MAC stable leases in 10.55.0.10–.250. FastAPI /leases + /lease-events endpoints, Round image switched to DHCP client with usb0→eye0 rename. 65 unit/integration tests pass + 1 netns acceptance test (requires root). Ref #158. |
||
|
CyberMind
|
a19486c997
|
feat: cookie audit pipeline (RGPD / ePrivacy reconciler) (#159)
* feat(mitmproxy): cookie_audit addon — Set-Cookie ledger for RGPD (ref #156) New mitmproxy addon that hooks the response flow and appends every Set-Cookie header to /var/log/secubox/cookie-audit/server.jsonl. Cookie values are sha256-hashed at the addon — the raw value never leaves the process. Companion to the upcoming browser-side cookie-inventory.js + the CookieAuditAggregator that will reconcile both streams for RGPD/ePrivacy audit on operator-owned vhosts. Also adds the implementation plan under docs/superpowers/plans/. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(hub): cookie-inventory.js browser snapshotter (ref #156) Vanilla JS module that snapshots document.cookie at DOMContentLoaded, +2s, and on visibilitychange. Names + sha256(value) are POSTed to /api/v1/cookie-audit/ingest with credentials:'omit'. Hard-capped at 8 snapshots per page to avoid amplification. Loaded into every HTML response via the WAF banner injection (next commit). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(waf): inject cookie-inventory.js alongside health banner (ref #156) Extends the existing CDN banner injection block to also append a script tag loading shared/cookie-inventory.js. Adds two new CDN-config keys: - cookie_inventory_url (default: admin.gk2.secubox.in/shared/...) - cookie_audit_ingest_url (default: admin.gk2.secubox.in/api/v1/...) Both health banner and cookie inventory now load from the same injected guard block, so a single </body> replacement covers both. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(metrics): CookieAuditAggregator + Classifier (ref #156) Aggregator class mirroring CertStatusAggregator: tails the mitmproxy JSONL Set-Cookie ledger and the browser-snapshot ingest dir, then reconciles per (vhost, cookie-name). Per-cookie source verdict: * http — server only * js — browser only -> RGPD violation unless strictly_necessary * both — server + browser Classifier loads regex rules from config, checks categories in order (strictly_necessary > functional > analytics > marketing), first match wins. Default cache file: /var/cache/secubox/metrics/cookie-audit.json. 9 unit tests covering parsing, reconciliation, persistence and malformed input. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(metrics): wire /api/v1/cookie-audit/{ingest,report,summary} (ref #156) Imports + initializes CookieAuditAggregator, schedules it in the FastAPI lifespan alongside the other aggregators, opens up CORS to POST (was GET only) for the ingest endpoint, and adds three routes: POST /api/v1/cookie-audit/ingest — browser snapshot ingest (rate-bounded, credentials: omit, returns refused if audit disabled in config) GET /api/v1/cookie-audit/report?host=… — per-vhost or global report GET /api/v1/cookie-audit/summary — global rollup Hard caps: 200 cookies/snapshot, 128B names, 128B hashes, 512B UA. All 34 existing metrics tests + 9 new cookie-audit tests pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(core): get_cookie_audit_config + RGPD classifier baseline (ref #156) New config helper get_cookie_audit_config() that returns the merged [cookie_audit] section with built-in RGPD/CNIL classifier patterns. Categories (eval order — first match wins): strictly_necessary PHPSESSID, csrftoken, cart, remember_token, ... functional lang, locale, theme, cookie_consent, euconsent, ... analytics _ga*, _gid, _gat*, _pk_*, _hjid, _clck, _matomo*, ... marketing _fbp, _fbc, __utm*, _gcl_*, _uet*, IDE, MUID, NID Operator patterns from [cookie_audit.classifier] are MERGED on top of the baseline by default. Set classifier_override = true to replace it entirely (opt-out for sites that need a different taxonomy). secubox.conf.example documents the section with commented-out examples. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: cookie-audit API + addon (ref #156) - secubox-metrics README: new "Cookie Audit" section documenting the three endpoints, the source verdict semantics (http/js/both), and the rgpd_violation rule. - secubox-mitmproxy README: new "Addons" section listing both secubox_waf and cookie_audit + the dual -s mitmdump invocation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(claude): record cookie-audit pipeline in HISTORY+WIP (ref #156) HISTORY 2026-05-16 entry captures scope (server ledger + browser snapshot reconciler), the 8 implementation tasks landed in the worktree, and the two follow-ups left (AppArmor + logrotate). WIP gets a top entry pointing at the worktree/branch with the test totals and "awaiting validation, no PR opened" status. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(banner): add Cookie Audit live tile + summary URL injection (ref #156) Adds a "CookieAudit" section to the health-banner live panel alongside VisitorOrigin / LiveHosts / CertStatus. It polls /api/v1/cookie-audit/summary every 30s and renders: - host count - RGPD verdict row (red ⚠ when violation_count > 0, green ✓ otherwise) - per-category breakdown: strict / func / analytics / marketing - unclassified row only shown when > 0 The WAF banner-injection block now also sets window.SECUBOX_COOKIE_AUDIT_SUMMARY (new cookie_audit_summary_url config key) so the banner can hit the cross-origin summary endpoint when the host page is on a non-admin vhost. Banner bumped to v1.4.0. New section hides itself silently when the aggregator is disabled or empty — zero visual impact on existing deployments. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: CyberMind-FR <gandalf@Gk2.net> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
| 18c85939bc |
docs: Phase 0 spec rev. 3 — three-LXC topology (mail / roundcube / horde)
Live-board topology changed 2026-05-16 per admin request: - Add Horde Groupware Webmail as a second webmail option - Extract Roundcube from the mail LXC into its own container Final layout: - mail LXC 10.100.0.10 Postfix + Dovecot + (Phase 2) Rspamd + ClamAV - horde LXC 10.100.0.11 Apache + Horde Webmail 5.x + MariaDB - roundcube LXC 10.100.0.12 Apache + Roundcube + SQLite Invariants reworked: - I1: "exactly ONE LXC" → "three LXCs, each single-responsibility" - I2: extended to cover all three IPs - I9: "webmail = Roundcube" → "Roundcube + Horde coexist" - I10/I11/I12: split mail-server vs webmail responsibilities - I14 (NEW): password policy disabled by admin opt-out §5.1 LXC layout, §5.2 ports table, §5.3 daemon inventory all rewritten to reflect the three containers and where each daemon lives. Webmail LXCs reach Dovecot via tls://10.100.0.10:143 — no bind-mounts cross LXCs. |
|||
| 35ba4c2c97 |
fix(mail): replay Phase 1 fix commits lost in PR #141 squash-merge
The squash-merge of PR #141 dropped the last 4 commits from feature/136,
leaving master with a broken secubox-mail 2.2.0:
* lib/install.sh / lib/lxc.sh / lib/migrate.sh shipped to wrong path
(debian/rules expects lib/mail/, package only ships users.sh).
* mailctl cmd_install/cmd_start/cmd_stop/cmd_sync/cmd_dkim still call
/usr/sbin/mailserverctl + /usr/sbin/roundcubectl — those are now
deprecation shims that exec mailctl, producing the same fork-bomb
recursion that took down 192.168.1.200 during initial deploy.
* mail-migrate-to-single-lxc.sh in-tree fallback expected old lib path.
* helpers.bash bats fixtures source wrong lib paths.
* Phase 1 post-mortem doc + smoke-gate-11 + cmd_install fix were absent.
This replays the affected files from feature/136 tip
(
|
|||
| 1f562593b6 |
docs: Phase 2 implementation plan (Rspamd migration)
8 milestones, ~25 bite-sized TDD tasks:
- A: worktree + scaffolding (red baseline for bats + pytest)
- B: lib/mail/rspamd.sh install/configure helpers + 9 config templates
- C: FastAPI rspamd router + deprecation shims for /dkim/* /spam/* /grey/*
- D: mailctl rspamd subcommand
- E: install_mail_packages adds Rspamd + postinst provisions secrets/data
- F: host nginx vhost for rspamd.gk2.secubox.in + sync-mitmproxy-routes patch
- G: secubox-mail 2.3.0 bump + 13-gate acceptance smoke script
- H: build, deploy (with explicit user-confirmation gate per Phase 1 lesson),
run gates 1-11, THEN purge legacy, run all 13
Removal ordering enforced by D9 in lib/mail/rspamd.sh: rspamd_purge_legacy
refuses unless `rspamc stat` confirms Rspamd healthy.
Phase 1 deploy lessons absorbed:
- bats test_deb_paths.bats verifies dpkg-deb -c ships every lib/mail/*.sh
- install_mail_packages adds `systemctl enable postfix`
- Acceptance smoke uses `timeout` wrappers, never raw pipes
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
| b78e2584e2 |
docs: Phase 2 spec — Rspamd migration (single-domain DKIM, no ClamAV)
Locked decisions from brainstorm: - D1: Rspamd replaces SA + OpenDKIM (Phase 0 invariant I3) - D2: ClamAV deferred to Phase 2.5 - D3: Single-domain DKIM (secubox.in, selector "default") — Phase 3 widens - D4: Rspamd web UI behind admin JWT + enable_password for writes - D9: Install Rspamd FIRST + verify, THEN purge SA/OpenDKIM - Legacy /dkim/* /spam/* /grey/* endpoints become deprecation shims Plus Phase 1 deploy lessons carried over: - systemctl enable postfix in install_mail_packages - dpkg-deb -c path-coverage bats test (test_deb_paths.bats) - sync-mitmproxy-routes.sh mail-LXC awareness patch 13-gate acceptance smoke covers Postfix milter wiring, DKIM signature present on outbound, SPF/DMARC enforcement, greylist behaviour, web UI auth, SA+OpenDKIM purge, and Phase 1 regression checks. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| 3770586a31 | docs: Phase 1 rollback recipe (pre-issue) | |||
| bd4ef55a98 |
docs: revise mail Phase 0 spec + Phase 1 plan to match board reality
Live board inspection on 2026-05-15 found that the single-mail-LXC layout has already been hand-built under /data/lxc/mail with /data/volumes/mail bind-mounts and 10.100.0.10/24 unprivileged-veth networking on br-lxc. The repo source code was out of date (still referenced /srv/lxc, /srv/mail, 192.168.255.x, mail_container/webmail_container). Spec rev. 2: - Invariants now reflect /var/lib/lxc/mail, /data/volumes/mail, 10.100.0.10 - New invariant I12: persistent data lives on host bind-mounts only - New invariant I13: existing 5-user secubox.in mailboxes MUST be preserved - Postgrey dropped entirely; ClamAV deferred to Phase 2 with Rspamd Plan rev. 2: - Reduced from ~30 tasks to ~15 — Phase 1 is now "catch source up to board" - Migration script becomes defensive scanner (no destructive ops) - guard_data_path bats test enforces the data-preservation invariant - secubox-mail bumps 2.1.0 → 2.2.0 (was incorrectly targeting 2.0.0 in rev. 1) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| b59d053c54 |
docs: Phase 1 implementation plan (mail stack LXC consolidation)
Bite-sized TDD plan covering 8 milestones (~30 tasks) to collapse the existing mailserver + roundcube two-LXC layout into a single 'mail' LXC. Resolves spec open questions: Roundcube via nginx+php-fpm; HAProxy TCP-pass-through to LXC for 25/587/993/4190. Includes data migration helper, deprecation of 3 packages to transitional, and end-to-end acceptance smoke. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| 2494b55a18 |
docs: Phase 0 architecture spec for full-featured mail+webmail stack
8-phase plan to deliver multi-domain Postfix+Dovecot+Rspamd+ClamAV+Roundcube inside a single 'mail' LXC, integrated with secubox-users (identity) and secubox-nextcloud (CardDAV/CalDAV). Locks 10 architectural invariants, deprecates secubox-mail-lxc + secubox-webmail-lxc + secubox-webmail, and records the deferred per-phase design questions. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| 78316556d4 |
docs(remote-ui): converged dashboard implementation plan (ref #135)
23-task TDD plan for extracting secubox_common from round/fb_dashboard.py and square/Phase-3 kiosk, adding pointer input on Pi 4B/400, and shipping both form factors as one PR. Gate: PR #134 (framebuffer numpy+center-pad) must merge before Task 1. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| b5e44e720a |
docs(remote-ui): converged dashboard design spec (ref #135)
Brainstorm output for converging round/ and square/ kiosks into shared remote-ui/common/python/secubox_common/ package, adding pointer (mouse + touchpad) input on Pi 4B/400, and fixing the round/ icon-loading bug surfaced during Task 19 bench (load_module_icon resolves wrong dir because module icons only live in common/, not round/). Key decisions captured in the spec: - OO layout classes: DashboardCanvas base in common/, RoundDashboard + SquareDashboard subclasses overriding layout() - Big-bang refactor: both round/ and square/ converge in the same PR - Input layer per form factor (square/ adds pointer to its existing touch_input.py; round/ stays touch-only) - Auto-hide cursor sprite on square/ (visible only when pointer motion in the last 3s) - Ship secubox_common at /var/www/common/python/ and prepend via systemd Environment="PYTHONPATH=..."; no separate .deb needed. Next: user reviews spec, then writing-plans skill creates the implementation plan. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
|
CyberMind
|
dee8bf8b81
|
feat(remote-ui): Phase 3 — Pillow+framebuffer kiosk for Pi 4B/400 (ref #127)
* feat(remote-ui/square): carry forward Phase 2 helper + debian + firstboot (ref #127) Snapshot of Phase 2 PR #131's validated infrastructure, brought into Phase 3 unmodified. The Chromium+PySide6 dual-window kiosk path is dropped (replaced by Pillow+framebuffer per the Phase 3 design spec). What carries forward: packages/secubox-eye-square/ ├── helper/ 21 pytest cases green │ ├── eye_square_helper/ FastAPI app + SO_PEERCRED auth + 4 route modules │ │ ├── app.py middleware (JSONResponse fix preserved) │ │ ├── auth.py ALLOWED_UIDS frozenset │ │ ├── __main__.py uvicorn UDS bind │ │ └── routes/ usb_gadget, service, lockdown, console │ └── tests/ 6 test files (test_e2e.py dropped — it depended │ on the right_panel package which is gone) └── debian/ arm64 package shell; control will be edited by Phase 3 plan to drop Chromium/Qt/X deps remote-ui/square/ ├── files/etc/systemd/system/ │ ├── secubox-eye-square-helper.service Helper FastAPI as secubox-eye-square user │ ├── secubox-otg-gadget.service configfs composite gadget (square variant) │ └── secubox-firstboot.service oneshot trigger for firstboot.sh ├── files/etc/udev/rules.d/ │ └── 90-secubox-otg-square.rules host-side interface rename rule ├── files/etc/apparmor.d/ │ └── secubox-eye-square-helper CAP_NET_ADMIN + CAP_SYS_ADMIN scope ├── files/etc/secubox/eye-square.toml.example ├── files/usr/local/sbin/firstboot.sh GPIO 5V check + hostname + SSH + toml ├── build-eye-square-image.sh will be heavily modified — drop Chromium/Qt/X ├── install_pi4.sh SD flash with safety guards └── deploy.sh SSH hot-update — kiosk service list will change Phase 1's remote-ui/common/ (PR #130) is NOT brought in — Phase 3 doesn't consume it. round/fb_dashboard.py stays on master untouched. What's DROPPED from Phase 2 (not brought in): - right_panel/ entire PySide6 widget tree + ipc_bridge + WebSocket server - square-bridge.js Chromium-side TM hook override - secubox-kiosk-x.service, secubox-square-chromium.service, secubox-square-right-panel.service systemd units - etc/openbox/{autostart,rc.xml}, etc/nginx/sites-available/secubox-square - home/secubox/.xinitrc Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(remote-ui): Phase 3 implementation plan — Pillow+framebuffer kiosk (ref #127) 25 tasks covering kiosk skeleton, theme + modules table, sim, framebuffer + touch input, transport manager + helper client, 4 right-pane tabs (Alerts / Module Detail / Console / Mode Controls), right_panel composer, ring_dashboard (Pillow port of Phase 2's QPainter ring renderer — visually faithful to round/'s dashboard but independently authored, since round/fb_dashboard.py must remain unchanged), __main__ event loop, systemd unit, fb udev rule, debian/control simplification (drop Qt/X/Chromium, add python3-pil/evdev), build script + firstboot + deploy.sh edits, README + CLAUDE.md docs, regression test, Pi 4B + Pi 400 hardware bench tests, open PR. Plan supersedes Phase 2 PR #131 (closed). Phase 2 helper FastAPI carries forward unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(remote-ui/square): scaffold kiosk/ package skeleton (ref #127) * feat(remote-ui/square/kiosk): theme.py + modules_table.py with TDD (ref #127) * feat(remote-ui/square/kiosk): sim.py drift generator with TDD (ref #127) * feat(remote-ui/square/kiosk): framebuffer.py mmap helper with TDD (ref #127) * feat(remote-ui/square/kiosk): touch_input.py evdev reader + classify (ref #127) * feat(remote-ui/square/kiosk): transport_manager.py with TDD (ref #127) * feat(remote-ui/square/kiosk): helper_client.py sync httpx UDS with TDD (ref #127) * feat(remote-ui/square/kiosk): tabs/alerts.py with TDD (ref #127) * feat(remote-ui/square/kiosk): tabs/module_detail.py with TDD (ref #127) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(remote-ui/square/kiosk): tabs/console.py with TDD (ref #127) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(remote-ui/square/kiosk): tabs/mode_controls.py with TDD (ref #127) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(remote-ui/square/kiosk): right_panel.py tab manager with TDD (ref #127) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(remote-ui/square/kiosk): ring_dashboard.py with easing + alerts ribbon (ref #127) The implementer adapted update_metrics() to store raw metric values in _target/_current (e.g. cpu_percent=80.0) rather than the 0..1 extracted ratio from the plan's code block — this matches the test's assertion (_target["cpu_percent"] == 80.0). Module.extract() is now applied at draw() time. Behaviour is identical from the renderer's perspective. Also added 'if m.metric in metrics' guard so partial-metric updates preserve existing targets for unspecified keys (kiosk-friendly). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(remote-ui/square/kiosk): __main__.py event loop + smoke tests (ref #127) Wires HelperClient + TransportManager + SimState + RingDashboard + RightPanel + FrameBuffer into a 30 FPS event loop. Probes transport every 30s, fetches metrics every 2s (falls back to SIM), composes 800x480 frame (dashboard + panel), blits to /dev/fb0. Touch input integration deferred; modules from Task 6 are available for the bench iteration to wire when the hardware ships. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(remote-ui/square): secubox-square-kiosk.service unit + fb0 udev rule (ref #127) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(secubox-eye-square): debian/control drops Qt/X/Chromium deps for Phase 3 (ref #127) Phase 3 replaces the Phase 2 Chromium+PySide6 stack with a single-process Pillow+/dev/fb0 kiosk. Dependency cleanup: dropped: chromium, openbox, xserver-xorg, xinit, unclutter, nginx-light, python3-pip, python3-pyside6 (Recommends), python3-qasync added: python3-pil, python3-evdev Helper FastAPI deps unchanged (carried forward from Phase 2). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(remote-ui/square): build-eye-square-image.sh slimmed for Phase 3 (ref #127) Apt install list trimmed: dropped chromium, openbox, xserver-xorg, xinit, unclutter, 12 libxcb-* libraries, nginx-light, python3-pip; added python3-pil + python3-evdev. The PySide6 pip install is gone. Removed Phase 2 vestiges: - nginx + round/ html copy + square-bridge.js injection - openbox autostart, xinitrc chmod - render group (no /dev/dri access needed without Chromium) Systemd enable list updated: - Dropped: nginx, secubox-kiosk-x, secubox-square-chromium, secubox-square-right-panel - Added: secubox-square-kiosk - Default target now multi-user.target (was graphical.target) Python install copies kiosk package (was right_panel package). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(remote-ui/square): firstboot.sh enables kiosk service (drops nginx + 3 Phase 2 units) (ref #127) Phase 3 firstboot only enables secubox-otg-gadget, secubox-eye-square-helper, and secubox-square-kiosk. The X / Chromium / right-panel / nginx units no longer exist on the image. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(remote-ui/square): deploy.sh targets kiosk service (drops chromium+right-panel+nginx) (ref #127) Hot-update flow simplified for Phase 3: - Rsync helper + kiosk Python packages only (no /var/www/, no square-bridge.js, no nginx content). - Restart secubox-eye-square-helper + secubox-square-kiosk (was chromium + right-panel + nginx reload). - Health check via systemctl is-active (was curl http://localhost/). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(remote-ui/square): README + CLAUDE.md for Phase 3 (ref #127) README rewritten for the Phase 3 architecture (Pillow+/dev/fb0 kiosk, no Chromium/Qt/X/nginx). CLAUDE.md added with file map, stack inventory, run + debug recipes, and the round/ untouched constraint. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: CyberMind-FR <gandalf@Gk2.net> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
CyberMind
|
7c37415f88
|
feat(remote-ui): Phase 1 — extract common/ shared core (ref #127)
* feat(remote-ui/common): scaffold shared-core directory (ref #127) * feat(remote-ui/common): extract palette.css from round/ (ref #127) * fix(remote-ui/common): trim palette.css to spec (6 module + 8 C3BOX tokens) (ref #127) * feat(remote-ui/common): extract base.css verbatim from round/ (ref #127) * feat(remote-ui/common): extract ICONS to icons.js (ref #127) * docs(remote-ui/common): correct icons.js header comment — sizes are {22,48,96} not 128 (ref #127) * feat(remote-ui/common): extract RINGS + CX/CY/SA to modules-table.js (ref #127) * feat(remote-ui/common): extract CFG to config.js (replaces jwt-helper.js per #127 plan revision) * feat(remote-ui/common): extract TransportManager with onModuleTap/onTransportChange hooks (ref #127) * feat(remote-ui/common): extract SIM + simStep to sim.js (ref #127) * feat(remote-ui/common): move 24 SecuBox module PNG icons to common/assets/icons/ (ref #127) * feat(remote-ui/common): move secubox-otg-gadget.sh, add GADGET_NAME env override + arm64 serial fallback (ref #127) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(remote-ui/common): trim whitespace from device-tree serial-number read (ref #127) * feat(remote-ui/common): move secubox-otg-host-up.sh + variant-aware comment (ref #127) * refactor(remote-ui/round): consume common/ via <link>/<script> tags (ref #127) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(secubox-system): add form_factor to RemoteUIConnectedRequest with TDD (ref #127) * feat(remote-ui/round): deploy.sh bundles common/ alongside round/ (ref #127) * fix(remote-ui/round): deploy.sh COMMON_SRC path resolution + rsync --delete (ref #127) * feat(remote-ui/round): build-eye-remote-image.sh embeds common/ + nginx /common/ alias (ref #127) * docs(remote-ui): document common/ dependency in round/ docs (ref #127) * docs(remote-ui/round): clarify palette.css is forward-looking (ref #127) * docs(remote-ui): Task 18 regression-gate report — structural verification (ref #127) Full diffoscope gate blocked by missing hyperpixel2r.dtbo prerequisite. Structural equivalence verified instead: Phase 1 changes are purely additive (common/ embed + nginx /common/ alias + extracted JS/CSS/icons), no behavioural change to round/'s existing logic. User must run the full image diffoscope manually after sourcing the hyperpixel2r.dtbo blob, before final merge. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * ci(eye-remote): bump workflow VERSION env 2.2.0 → 2.2.1 (ref #127) Pre-existing drift: build-eye-remote-image.sh writes secubox-eye-remote-2.2.1.img but the workflow's Compress/Checksum/ Upload steps reference ${{ env.VERSION }} = 2.2.0, so the compress step fails with "No such file or directory" after a successful build. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: CyberMind-FR <gandalf@Gk2.net> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
| 1d27705d0d |
docs(remote-ui): Phase 3 design spec — Pillow+framebuffer Pi 4B/400 kiosk (ref #127)
Supersedes Phase 2 PR #131 (Chromium + PySide6 dual-window) after bench-test revealed window-stacking fragility and ~400MB RAM footprint. Operator requested an all-Python kiosk aligned with round/'s fb_dashboard.py. Phase 3 architecture: - Single Python process, Pillow renders 800×480 frame, mmap /dev/fb0 - Left 480×480: ring dashboard (rings/pods/clock) — independently authored but visually faithful to Phase 1 round/index.html - Right 320×480: 4 tabs (Alerts/Module Detail/Console/Mode Controls), manually drawn with Pillow - Touch via python-evdev - Helper FastAPI on Unix socket — carried forward from Phase 2 unchanged - Debian packaging + firstboot.sh — carried forward - ~400MB compressed image (vs 1.5GB), ~30-50MB RAM (vs 400MB) round/fb_dashboard.py UNTOUCHED — Pi Zero W deployment stays at v2.2.1. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| fdfc829ec8 |
docs(remote-ui): Phase 2 implementation plan — square/ variant build (ref #127)
30 tasks covering: helper FastAPI (auth + 4 route modules) with SO_PEERCRED, PySide6 right column (4 tabs + IPC bridge + helper client + theme parser), Chromium-side square-bridge.js, systemd units, Openbox + nginx + AppArmor, firstboot.sh with GPIO 5V check, image build (debootstrap Bookworm arm64), SD flash + deploy scripts, Debian packaging, pytest regression, manual Pi 4B + Pi 400 acceptance, PR. Depends on Phase 1 PR #130 (merged). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| cddf9b125a |
docs(remote-ui): Phase 1 implementation plan — common/ extraction (ref #127)
20 bite-sized tasks covering: common/ skeleton, JS/CSS/icons/shell extraction, TransportManager hooks (onModuleTap, onTransportChange), form_factor Pydantic field with TDD, round/ rewire to consume common/ via <link>/<script>, deploy.sh + image build updates, three regression gates (pytest green, diffoscope timestamp-only deltas, manual Zero W bench), and PR opening. Phase 2 plan to be written after Phase 1 merges. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| ba5ea4626b |
docs(remote-ui): add Pi 400 as a supported target for square/ (ref #127)
Pi 400 shares BCM2711 with Pi 4B, runs the same arm64 image. DTB auto-selection picks bcm2711-rpi-400.dtb at boot. firstboot.sh records the board flavour. USB peripheral mode requirement (GPIO 5V power) is unchanged. Integrated keyboard is recognised via libinput. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| 7ae5e82345 |
docs(remote-ui): square/ variant v0.1 design spec (ref #127)
Two-phase design: extract remote-ui/common/ (Phase 1, refactor round/ with no behavioural change), then add remote-ui/square/ targeting Pi 4B + official 7" 800x480 touchscreen as a dual-pane kiosk — round UI in Chromium at (0,0)+480x480 plus a PySide6 right column at (480,0)+320x480 with Alerts, Module detail, Console, and Mode controls tabs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| bedc6fda45 | fix(tests): drop tests/__init__.py to allow cross-dir pytest collection; exist_ok in auth_data_dir (ref #120) | |||
| 78c8d3c4fa |
docs: implementation plan — auth rework (19 tasks, offline-mode hardening included) (ref #120)
19 bite-sized TDD tasks covering: schema v2, password policy, TOTP (pyotp + argon2id backup codes), user_store with auth.toml fallback, single mutation engine, v1→v2 migration, secubox_core.auth rewire (jti + scope), branching /auth/login + TOTP enrollment/challenge + set-password, secubox-users API handlers, usersctl Python rewrite, CLI↔API parity test, debian deps + postinst seed (admin + hostname), live smoke, and offline-mode hardening (server-side QR rendering + NTP-aware TOTP window + /auth/health endpoint). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| 14fba6ccec |
docs: auth rework design — secubox-users as identity source + TOTP 2FA (ref #120)
Brainstormed spec for replacing the plaintext auth.toml admin login with secubox-users-backed argon2id auth, RFC 6238 TOTP 2FA (mandatory for admins), kill-on-disable session revocation, CLI/API parity via a single secubox_users.engine module, and feature-flagged 3-phase rollout. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| 5fbdb15c8e |
docs(metablog-webhook): Implementation plan for deploy webhook (ref #113)
8 tasks: HMAC + secret loader, git_pull + ring buffer, dispatcher + lock, main.py wiring, install/uninstall scripts, bash smoke, README + Session 166, finish + PR. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| 42c75161df |
docs(metablog-webhook): Design spec for deploy webhook (ref #113)
Sub-project E of #49. POST /api/v1/metablogizer/webhook receives Gitea push events for metablog-* repos, verifies HMAC-SHA256, git pulls the site dir under a per-site asyncio.Lock, invalidates load_sites() cache (#111), conditionally reloads nginx if site.json:domain changed. Companion scripts/metablog-webhook-install.sh registers the webhook on all 166 metablog-* repos via Gitea API (idempotent). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| 5867740c42 |
docs(metablog-ui): Implementation plan for version dashboard (ref #103)
5 tasks covering index.html extension (3 columns + filter + sort + 60s polling), site.html drill-in page, smoke test, docs, and PR finish. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| e8588f62e4 |
docs(metablog-ui): Design spec for version dashboard (ref #103)
Sub-project D of #49. Extends existing /metablogizer/ list view with 3 columns (version, streamlit_app, last_updated) + filter box + sort + 60s polling. New site.html drill-in page that surfaces every site.json field plus 3 external links (live site, Gitea repo, Streamlit app). Tag history defers to Gitea's UI (browser auth) because the repos are private and browser-side proxying through a stored token is heavier than the MVP warrants. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| a4ada6c21f |
docs(metablog): Backfill run summary (ref #101)
165 total: 104 created + 61 skipped (including 2 fixed via --force for missing 'published' field: money, evolution). 0 failed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| 81f9c639a4 |
docs(metablog): 8-task implementation plan for site.json schema (ref #101)
8 tasks: 1. JSON Schema draft-07 file 2. Python validator + enricher module + 7 pytest cases 3. Wire helper into load_sites() (warn-only validation) 4. Add python3-jsonschema to debian/control 5. Backfill script (SSH orchestrator + Gitea probe for streamlit_app) 6. 3-gate smoke test 7. Live backfill run + summary 8. .gitignore + README + tracking + finish worktree Self-review: spec coverage complete, no placeholders, identifiers consistent (_load_site_json, _schema_enrich, _schema_validate, status keywords, paths). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| 542f236601 |
docs(metablog): Design spec for site.json schema + version metadata (ref #101)
Sub-project C of #49. JSON Schema draft-07 + Python validator/enricher + backfill script + API extension. Permissive validation (log warnings, don't reject). version + last_updated derived from git when site.json lacks them. Backfill creates complete site.json for the 105 sites without one (auto-detects streamlit_app via Gitea repo presence) and preserves the 61 existing ones unless --force. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| 2f440c6194 |
Merge remote-tracking branch 'origin/master' into feature/95-streamlit-per-site-version-pinning-conta
# Conflicts: # .claude/HISTORY.md # .claude/WIP.md # .gitignore |
|||
| 744770750e |
Merge remote-tracking branch 'origin/master' into feature/94-metablogizer-gitea-ingest-import-166-sit
# Conflicts: # .claude/HISTORY.md # .claude/WIP.md |
|||
| 4a388994d3 |
Merge remote-tracking branch 'origin/master' into feature/49-feat-metablogizer-streamlit-version-mana
# Conflicts: # .claude/HISTORY.md |
|||
| d110c00786 |
docs(streamlit): Full-run ingest summary (ref #95)
28 Streamlit apps ingested into gitea.gk2.secubox.in/gandalf/streamlit-*. First pass had 20 failures from pre-existing broken Gitea repo stubs; bulk-deleted via API and second pass cleaned to 0 fail. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| 37df7b9c9b |
docs(streamlit): 9-task implementation plan for Gitea version pinning (ref #95)
9 tasks: 1. Per-app ingest function (scripts/lib/streamlit-ingest-app.sh) 2. Orchestrator (scripts/streamlit-ingest.sh) — preflights + JSON report 3. 3-app smoke + idempotent re-run 4. Full 30-app run + summary commit 5. streamlitctl deploy --from-gitea --tag + rollback subcommand 6. FastAPI _get_apps() enrichment (current_tag, deployed_at) 7. Deploy/rollback cycle smoke (canary: yijing) 8. .gitignore + README + Session 163 tracking 9. Finish worktree + PR (Refs #49 sub-F, Closes #95) Reuses patterns + helpers from sub-project B (#97): - scripts/lib/gitea-ssh-preflight.sh (sourced) - gitea@ SSH user (not git@) - defensive | tail -1 | tr -d '\n' on status capture (FU4 from B) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| c7613288f0 |
docs(streamlit): Design spec for Gitea version pinning (ref #95)
Sub-project F of #49. Audit shows 30 directory-form apps in /srv/streamlit/apps/ (23 already have .git/, 7 don't), running inside the existing streamlit LXC at 10.100.0.50. Design: - Mirror all 30 apps to gandalf/streamlit-<app> on Gitea - Retarget existing .git + push history for the 23 (preserves any local commits); git init + push for the 7 - Tag v1.0.0 on initial state - streamlitctl deploy <app> --from-gitea --tag <vX.Y.Z>: clone tag into /srv/streamlit/apps/<app>/ with backup of the current content; restart running instance if any - streamlitctl rollback <app>: restore latest backup - New API fields: current_tag, deployed_at (read from .deploy.json or git describe --tags --exact-match) YAGNI: no multi-version side-by-side, no container per version, no webhook trigger. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
|
CyberMind
|
2744758b9e
|
Health banner: live panel (visitor-origin + live-hosts + cert-status) (#98)
* docs(spec): Health banner live panel design (ref #92) Three public banner sections sharing one polling/CORS pipeline: - VisitorOrigin: nft set seen_src + GeoLite2-ASN.mmdb, threshold-gated rollup, raw IPs discarded before persistence - LiveHosts: HAProxy admin socket, 60 x 1-min ring buffer over req_tot deltas, hostname-heuristic frontend filter - CertStatus: scan /etc/letsencrypt/live + cryptography parse, classify valid / expiring_soon / expiring_critical / expired Each section fails independently; section hidden on enabled=false, empty entries, or fetch error. All three endpoints are unauthenticated, CORS-open, Cache-Control max-age=300. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(plan): Health banner live panel implementation plan (ref #92) 14-task TDD plan: tests scaffold -> config helpers -> three aggregators (visitor-origin / live-hosts / cert-status) -> FastAPI lifespan wiring -> nftables ruleset -> geoipupdate timer -> debian packaging -> banner v1.3.0 -> README + tracking docs -> full-suite verification + PR. Also reconciles spec with codebase conventions: service user is 'secubox' (not 'secubox-metrics'); config lives in /etc/secubox/secubox.conf, not a separate metrics.toml. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(metrics): Scaffold pytest layout for new aggregators (ref #92) Adds tests/__init__.py and conftest.py that wire packages/secubox-metrics/api and the repo-wide common/ onto sys.path so individual aggregator modules can be imported in isolation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(core): Add visitor_origin / live_hosts / cert_status config helpers (ref #92) Three new section helpers in secubox_core.config that merge defaults with operator-supplied TOML overrides. Each section defaults to enabled=false so the live-panel aggregators stay quiet on systems that haven't opted in. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(metrics): Use tmp_path fixture + drop unused import (ref #92) Switches the config-helper tests from a hard-coded /tmp path to pytest's tmp_path fixture, matching the pattern in packages/secubox-haproxy/tests/. Removes the now-unused 'from unittest.mock import patch' line. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(metrics): VisitorOrigin aggregator (ref #92) Pure-Python aggregator that polls the nft seen_src set, resolves ASNs via GeoLite2 mmdb, and emits a threshold-gated top-N rollup. Private/loopback IPs are skipped at lookup time; raw IPs never leave the function scope; the threshold gate runs before persistence so the cache file never contains attributable counts. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(metrics): VisitorOrigin code-review followups (ref #92) - mmdb auto-reopen on mtime change (Important): close + reopen when stat reports a different mtime; previously _mmdb_mtime was dead state. - _read_nft_set defensive parse (Important): guard against {elem: str} shapes to prevent a latent TypeError. - current() returns a defensive copy (Minor): no more by-reference leak of internal state. - Drop unused ip_address import and unused monkeypatch parameter (Minor). - Tighten tiebreak test to assert ASN order, not just count order (Minor). - Add test for mtime-based reopen. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(metrics): Pin VisitorOrigin error-path behaviour (ref #92) Regression tests for refresh_once: disabled config, missing mmdb, and nft subprocess failure must all yield a non-throwing degraded payload. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(metrics): LiveHosts aggregator with 60x1-min ring buffer (ref #92) Reads HAProxy admin socket via raw AF_UNIX, parses 'show stat' CSV, filters internal frontends (leading underscore or no dot), ring-buffers per-frontend deltas over 60 minutes, and emits a top-N hostname rollup. Counter-reset detection (cur < prev) yields a fresh-baseline bucket instead of a negative delta. current() returns a defensive copy mirroring the VisitorOrigin fix. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(metrics): Pin LiveHosts CSV parser + missing-socket paths (ref #92) Tests the show-stat CSV parser against the real HAProxy column order and asserts that an absent admin socket returns a degraded payload rather than raising. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(metrics): CertStatus aggregator (ref #92) Scans /etc/letsencrypt/live for cert.pem files, classifies each by days remaining (valid / expiring_soon / expiring_critical / expired) using the operator's warn_days/critical_days thresholds, and emits a summary + soonest next-renewal host. A single corrupt PEM never kills the scan. Days remaining computed with math.ceil so a cert expiring in 2.99d reports 3d, consistent with certbot/renewal tooling expectations. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(metrics): correct expired boundary when using math.ceil (ref #92) math.ceil maps any cert that expired within the last 24 h to days=0, which the previous `days < 0` guard treated as expiring_critical instead of expired. Changing the guard to `days <= 0` closes the gap: with ceil, days=0 means actual remaining time is in (-86400, 0] — i.e. already past or exactly at expiry — so classifying it as expired is correct. All four existing tests continue to pass. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(metrics): Wire three live-panel aggregators into FastAPI lifespan (ref #92) Adds the three asyncio background tasks under a single lifespan and exposes their current() payloads on /api/v1/metrics/{visitor-origin,live-hosts,cert-status} with a 5-min Cache-Control. Endpoints stay unauthenticated by design — the aggregators only emit threshold-gated, hostname-only data. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(metrics): Await cancelled lifespan tasks + tidy import order (ref #92) - Important: lifespan finally now awaits gather(*tasks, return_exceptions=True) after cancel(), so blocking subprocess/socket I/O in aggregator refreshes doesn't race uvicorn's shutdown timeout. - Minor: move 'from contextlib import asynccontextmanager' into the stdlib import group at the top of the file. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(metrics): Ship nftables ingress tap for visitor-origin (ref #92) Private inet secubox_metrics table with a timeout'd src-IP set, hooked from prerouting at priority -300 so additions happen before secubox-firewall's filter chain decides whether to drop the packet. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(metrics): Weekly GeoLite2 ASN refresh timer (ref #92) Conditional on /etc/secubox/secrets/maxmind.conf existing, so the unit is a silent no-op on installs that haven't supplied a license key. RandomizedDelay spreads load when many boxes deploy together. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * build(metrics): Package live-panel deps, nft ruleset, and geoipupdate timer (ref #92) - control: add python3-maxminddb, python3-cryptography, geoipupdate, nftables - rules: install nftables/ and systemd/ assets - postinst: secubox -> haproxy group, cache + secrets + GeoIP dirs, nftables reload, timer enable - service: ReadWritePaths gains /var/cache/secubox Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(metrics): Run geoipupdate as secubox + restart svc on upgrade (ref #92) - secubox-geoipupdate.service: User/Group=secubox so .mmdb files inherit the ownership the metrics service expects when reading them. Previously the unit ran as root and created root-owned files that secubox-metrics could not open. - postinst: switch 'systemctl start' to 'systemctl restart' so the secubox user's new haproxy-group membership is picked up by an already-running service after an upgrade. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(banner): v1.3.0 live panel — visitor origin, live hosts, cert status (ref #92) Three independent fetch loops on a shared 30s cadence, three DOM sections, per-section hide on enabled=false / empty / fetch error. Uses existing design tokens (gold/cyan/matrix-green) so no new CSS variables are added. A failing section never affects the others. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(banner): Correct banner element id lookup in live-panel (ref #92) The new live-panel sectionContainer() helper looked up getElementById('sbx-health-banner'), but the actual banner element is created with id='health-banner'. The mismatch made banner null, so the three live-panel sections were silently never appended to the DOM. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: Session 160 — Health Banner Live Panel (ref #92) README documents the three new endpoints + config blocks. HISTORY / WIP / MIGRATION-MAP entries describe the feature, the spec/plan paths, and the session's outcome. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(metrics): Address whole-branch review findings (ref #92) - Wrap _read_nft_set / _read_haproxy_stats in asyncio.to_thread so blocking I/O (subprocess up to 5s, AF_UNIX recv up to 2s) no longer stalls the event loop on every refresh tick. - Replace falsy current() guard with explicit _refreshed flag. Previously, a successful refresh that produced entries=[] would fall through to the on-disk cache, serving stale non-empty data during low-traffic periods. - Move geoipupdate from Depends to Recommends. It lives in bookworm/contrib, so a hard dependency breaks 'apt install secubox-metrics' on systems without contrib enabled. The aggregator already degrades gracefully when the mmdb is absent, making Recommends the correct strength. README documents the contrib note. - prerm stops + disables secubox-geoipupdate.timer/service so 'apt remove' doesn't leave an orphan timer firing weekly with a missing unit. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: CyberMind-FR <gandalf@Gk2.net> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
| 0cb6ad6434 |
docs(metablog): Record full-run ingest summary (ref #94)
166 sites ingested into gitea.gk2.secubox.in/gandalf/metablog-*. First pass had 72 failures from pre-existing broken Gitea repo stubs; bulk-deleted via API and second pass cleaned to 0 fail. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| cf1e13ccc8 |
docs(metablog): Add 8-task implementation plan for Gitea ingest (ref #94)
8 tasks: 1. Gitea config patch (ENABLE_PUSH_CREATE_USER, DEFAULT_BRANCH=main) 2. SSH preflight + key enrolment helper 3. Per-site ingest function (idempotent, history-preserving) 4. Orchestrator with preflights + JSON report 5. Smoke test on 3 sites (incl. idempotent re-run + clone-vs-source diff) 6. Full 166-site run + verification + summary commit 7. .gitignore + README + WIP/HISTORY tracking 8. Finish worktree + PR (Refs #49 sub-B, Closes #94) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| 577dc31d20 |
docs(metablog): Design spec for Gitea ingest of 166 sites (ref #94)
Sub-project B of #49. Audit shows 166 sites in /srv/metablogizer/sites/: 83 have local .git pointing at unreachable old Gitea (192.168.255.1:3001 or git.gk2/droplet-sites), 83 are raw files. Design: - All 166 land at gandalf/metablog-<site> on gitea.gk2.secubox.in (single namespace, even for the 12 ex-droplet-sites) - Existing .git: retarget remote + push history (preserves 1 commit) - Missing .git: git init + initial commit - ENABLE_PUSH_CREATE_USER=true in Gitea app.ini so push creates repos - SSH auth via gandalf's enrolled key; no API token plumbing - Idempotent: skip if remote HEAD matches local HEAD - v1.0.0 tag on HEAD after first push (per issue text) - output/ingest-report.json captures per-site status Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
