Brainstorming-validated design: perf-driven complete replacement of the WAF
mitmproxy/mitmdump inspection layer by a dedicated host-native Go binary sbxwaf,
sharing an extracted core with sbxmitm. Covers architecture, component isolation,
full feature port (routing/rules/ban/CrowdSec/cookie-audit/media-cache/error
pages), host-native hardening, and shadow→parity→cutover→rollback migration.
Analysis: gomitmproxy (unmaintained, dropped) vs martian/goproxy (Go) vs hudsucker
(Rust) vs Squid+ICAP, mapped to the 18-addon capability set. Recommendation: Go
hot-path core + retained Python analysis sidecars. Phased plan with shadow-run +
nft-DNAT-flip rollback (no big-bang cutover). Phase-1 PoC (packages/secubox-toolbox-ng,
stdlib-only): forge from ca-wg CA, 204-block, body-inject, SNI-splice, ClientHello/JA4
capture — go vet clean, tests green, arm64 cross-compile OK. NOT wired to live R3.
## QR endpoints (python-qrcode)
GET /qr/splash.png → QR encodes the splash URL (Cabine join)
GET /qr/cert.png → QR encodes /ca/mobileconfig (iOS profile)
GET /qr/webclip.png → QR encodes /ca/webclip-cabine.mobileconfig
GET /qr/{target}.png → generic : maps splash/cert/webclip/report/
fingerprint to fixed URLs, or encodes literal
PNG output, Cache-Control: public, max-age=3600.
Verified : 232x232 1-bit grayscale, 384-553 bytes each.
debian/control adds python3-qrcode dependency.
## Splash QR card
The cert install card on the splash now shows 3 QR codes (① SPLASH,
② CERT iPHONE, ③ WEBCLIP HOME) on a white background grid below the
test button. Scannable from another device for cross-installation
(useful at the cabine : user shows QR to others without typing URLs).
## Dashboard hero : 9th widget (device fingerprint)
Mirroring the poster layout, added a full-width 9th widget below the
2 rows of 4 KPI widgets :
┌─────────────────────────────────────────────────────────┐
│ 📱📱 Empreinte device │
│ iPhone iOS 26.3 · 🟢 Chrome 148 │
│ 21 UAs distincts observés │
└─────────────────────────────────────────────────────────┘
Now matches the 9-widget metrics layout from the public poster :
🌐 connexions · 📡 hôtes · ✅ OK 2xx · 🔒 cert-pin
📺 apps · 🍪 trackers · 🌍 pays · 🎯 score
📱 device (full-width)
## README + assets/poster integration (#497)
- README.md : new section at top with poster image embed + 3 doc links
- docs/assets/poster/README.md : print specs + variants list + license
- Placeholder docs/assets/poster/village3b-A2.png expected (user provides)
Wiki integration pending : will be added via gh wiki API once issue #497
phase 2 launches.
## Rebase
Branch was rebased onto master (which now has all PR #493 Phase 3
commits). control merged : Phase 3 fonts + Phase 5 lxc deps + new
python3-qrcode in one cohesive Depends block.
Le projet est sous LicenseRef-CMSD-1.0 (Source-Disclosed License) EXCLUSIVEMENT.
Les mentions AGPL dans les textes user-facing étaient incorrectes — corrigé.
Fichiers modifiés :
- packages/secubox-toolbox/README.md (section Licence)
- packages/secubox-toolbox/conf/splash.html.j2 (footer)
- packages/secubox-toolbox/conf/report-live.html.j2 (footer)
- packages/secubox-toolbox/secubox_toolbox/reports.py (PDF footer)
- docs/marketing/PROMPT-claude-presse-gouv.md (7 mentions corrigées + 1 ajout
d'avertissement explicite "NE PAS confondre avec AGPL ou Apache" pour
guider la future session Claude)
Le prompt presse a aussi été ajouté à cette branche pour éviter la divergence
master-vs-worktree (le file était untracked dans master).
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
Static resolv.conf in the container is hijacked by systemd-resolved's empty
127.0.0.53 stub, killing DNS (no appstore/updates/external mounts) while NAT
egress still works. Drop a resolved.conf.d upstream at install. Add
docs/FAQ-LXC-DNS.md describing symptom/cause/fix. Live NC already patched.
Document the persona model: a named, browser-encrypted group of site sessions
the operator picks from a tick-list of their live logins, reapplied with one
Become click on any same-LAN box. Explains the <all_urls> read / ticked-write
posture and that ciphertext never leaves the SecuBox.
Per-package inspection of the mesh cluster found:
- secubox-mesh (Yggdrasil daemon control), secubox-meshname
(Meshname DNS resolver), secubox-daemon (Go mesh binary) — all
distinct, keep.
- secubox-mirror — miscategorized: it's an APT/CDN cache, not
mesh-related at all. Audit grouped it by prefix matching.
- secubox-master-link — effectively dead on a running system:
ships an 851-LOC API and frontend but no nginx config; its 21
API endpoints (/run/secubox/master-link.sock) are unreachable
from the web. The "master-link" UI operators actually see is
served by secubox-p2p (which has its own /master-link/status +
/master-link/token endpoints + ships the frontend to
/var/www/secubox/master-link/).
- secubox-p2p is the canonical master-link owner.
Real consolidation candidate: master-link → p2p fold. Bigger than
the mmpm/magicmirror fold (issue #381) because of asymmetric API
surface — master-link's 21 endpoints vs p2p's 2 existing
/master-link/* endpoints means the merge isn't mechanical. Defer
to a separate per-cluster issue with its own scope decisions.
Mesh cluster net reduction potential: -1 (after master-link → p2p
fold), not the -4 the original audit projected. Total realistic
floor unchanged from previous revision (~130-136 packages).
Per-package code inspection of the streamlit (already revised), dpi
(revised in 382), dns, and threats clusters all returned the same
finding: the audit's initial cluster groupings were based on naming
similarity, not architectural redundancy.
DNS cluster (5 → 5, was 5 → 1):
Five distinct DNS-layer subsystems, no shared config files. The
audit's "all overlap on /etc/resolv.conf and/or unbound" claim was
wrong — grep for /etc/resolv.conf in any of them returns zero hits.
Each touches its own config + data directory.
Threats cluster (7 → 7, was 7 → 1):
Seven distinct security capabilities with distinct backends
(crowdsec scenarios, nftables, dnsmasq logs, remote feeds, own
data dirs). Only the umbrella secubox-threats actually subscribes
to the CrowdSec/Suricata event bus the audit claimed all seven
did.
Tier 4 table revised to "candidates for inspection" — each cluster
needs the same per-package code check before any merge work.
Realistic-reduction floor updated from ~100 packages to ~130-136
(a 4-8% reduction from 141 rather than the originally projected 28%).
Where consolidation actually paid off: Tier 0 packaging bug (#378),
Tier 1 mail dead-code (#380), Tier 2 mmpm-magicmirror scaffolding
duplicate (#381), Tier 2 dpi description clarity (#382).
The pattern observation section explicitly flags this for future
work: most "fuzzy clusters" in out/clusters-fuzzy.txt are naming
overlaps, not merge targets. Treat the audit's cluster lists as
candidates for Description-clarity passes rather than packaging
merges.
After investigation in #382, the dpi cluster's four packages have
distinct operator roles, not duplicated ones:
- secubox-netifyd: daemon lifecycle dashboard
- secubox-dpi: analytics layer on top of netifyd
- secubox-ndpid: dedicated dashboard for the different nDPId engine
- secubox-mediaflow: streaming/VoIP consumer of DPI
The audit's original "dual-stream netifyd/nDPId" framing was
aspirational; the code is netifyd-only and nDPId analysis already
belongs to secubox-ndpid.
Real bug was a misleading Description on secubox-dpi (fixed in #382),
not a redundant-package situation. Keep all four packages.
Updates the realistic-reduction table: Tier 2 net is -1
(magicmirror only) rather than -3.
On closer inspection, secubox-streamlit and secubox-streamforge serve
distinct operator workflows that map to two different LuCI apps in the
OpenWrt source. Streamlit is the LXC-backed runtime that runs apps
(pulls in lxc + debootstrap + sudoers); streamforge is the
template/authoring tool (core deps only, 660-line FastAPI).
Merging would force every streamforge operator to install lxc and
debootstrap or require a complicated Recommends-vs-Depends split for
marginal benefit (net -1 package).
Also revises the realistic-reduction table: Tier 2 net is -3, not -4.
Adds scripts/audit-packages.py to scan packages/secubox-*/ and emit
structured artifacts under out/ (gitignored): packages.json,
packages.dot dependency graph, empty-packages.txt, prefix-based
clusters, and a hand-curated fuzzy-cluster report with per-member
evidence (deps, routes, services).
Findings doc 2026-05-27-secubox-consolidation-audit.md synthesises the
data into Tier 0-4 merge proposals grounded in real package contents,
not the plan stub's illustrative guesses. Key corrections vs the stub:
- 141 real packages (not "100+"); secubox-core is the substrate for 131
- streamlit-idle is already an in-package service variant, not a
separate package — that consolidation is half-done
- meta{blogizer,bolizer,catalog,oblizer} are real distinct services,
not typo proliferation (recommend rename for clarity, not merge)
- Two genuine packaging bugs surfaced: secubox-c3box binary name
collision between Python dashboard and Go daemon (issue #378),
secubox-smart-strip has no debian/ directory at all
Also imports two pre-existing 2026-05-26 plan stubs (consolidation,
build-scripts-refactor) that had been sitting untracked.
Re-run after any packaging change: python3 scripts/audit-packages.py
Anchor the pattern that secubox-lyrion v1.1.0 demonstrated (admin
static page on the canonical hub vhost + real app served at the root
of its own dedicated vhost) as a REQUIRED rule across the docs:
- docs/MODULE-GUIDELINES.md §4 — new "REQUIRED: dual-vhost pattern"
subsection with the URL table, the JS pattern for reading the
public URL from /access, and the explicit forbidden anti-pattern
(`proxy_pass /<module>/ → app LXC`).
- docs/MODULE-GUIDELINES.md §5 — nginx template updated: `/<module>/`
is `alias` (static), the dedicated vhost is a separate file. The
previous "Iframe pattern for LXC web UIs" + "LXC daemon iframe
target" sections are flagged deprecated.
- .claude/PATTERNS.md Pattern 12 — same rule reproduced for the
Claude-facing pattern catalog with the source-of-truth JS snippet.
- .claude/HISTORY.md + .claude/WIP.md — entry for today's three-
module alignment (lyrion 1.1.0 + zigbee + authelia).
- wiki/Architecture.md — short reference + pointer back to
MODULE-GUIDELINES.
- packages/secubox-lyrion/README.md — adds the URLs table at the top.
- packages/secubox-zigbee/README.md — was a verbatim copy of the
lyrion README, rewritten as a proper zigbee README with the new
URLs table + API surface + ports + files.
- packages/secubox-authelia/README.md — Quickstart points at
sso.gk2.secubox.in (was auth.maegia.tv); URLs table added.
The forbidden anti-pattern catches LMS Material, Nextcloud,
Grafana, z2m and any future module whose UI uses absolute asset
URLs — incident on gk2 2026-05-24, see secubox-lyrion 1.1.0
changelog.
* docs(mochabin): add EP06 mPCIe GPIO investigation runbook (ref #345)
scripts/probe-mpcie-gpios.sh: empirical CP0 GPIO sweep that drives
each unrequested line HIGH for a few seconds and watches dmesg/lsusb
for new USB device. Skips [used] lines, uses libgpiod gpioset
--mode=time so the line auto-restores to input on release. Modes:
--baseline (snapshot only, safe), --line gpiochipN K (single line),
no arg (full sweep gpiochip1+gpiochip2).
docs/hardware/mochabin-mpcie-ep06-runbook.md: hypothesis, procedure,
and DTS rfkill-gpio template patterned after cn9132-clearfog.dts
lines 69-122. Apply once a candidate GPIO is identified.
--baseline smoke-tested on gk2 — sweep deferred until spare EP06
arrives (no mPCIe device in J5 = no signal regardless of which line
drives W_DISABLE#).
* docs(mochabin): drop EP06 spare assumption from runbook (ref #345)
No spare EP06 on order; the existing EP06-E will be used for the
sweep when re-seated in J5. Replaces "deferred until spare" framing
with a multimeter-scoping fallback for the inconclusive case.
* fix(mpcie): v0.2.0 probe-mpcie-gpios.sh — drop blanket sweep, danger-list, dry-run default (ref #345)
The v0.1.0 "drive every unused-input CP0 GPIO HIGH for 3s one-at-a-time"
sweep took gk2 hard-down within seconds (host became unreachable; ARP
stopped responding). gpioinfo's [used] tag only reflects lines the
kernel *requested* — several unrequested CP0 GPIOs are physically
wired to critical board functions (eth switch reset, PCIe2 PERST#,
pca9554 IRQ) and driving them HIGH wedged the board.
v0.2.0:
- No blanket-sweep mode. Removed.
- Default mode = dry-run candidate listing only. No GPIO is driven.
- --commit --line CHIP N required to actually drive a single line.
- Hardcoded DANGER_LINES (gpiochip1:1, gpiochip1:9, gpiochip1:27)
skipped unconditionally, even with --commit.
- gpiochip2 entirely held off without --allow-chip2 (no per-line
schematic mapping for that bank).
- Hard refusals are vocal (REFUSED: ... reason) before exit.
Runbook updated to walk through the new dry-run → --commit flow.
HISTORY documents the incident + the v0.2.0 mitigations.
Lesson saved in memory feedback_gpio_blanket_sweep_crashed_board.md.
---------
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
* docs(plan): #349 sentinelle-gsm v0.3.1 L3 + scoring + baseline (ref #349)
* feat(sentinelle-gsm): GsmtapListener exposes raw_l3 payload bytes (ref #349)
v0.3.0's Observation carried only header metadata (arfcn, frame_nr,
channel, sub_type), which was enough to prove the GSMTAP pipe but not
to drive L3-aware scoring. v0.3.1 needs the post-header payload to
decode BCCH System Information (LAC/CI/MCC/MNC) and CCCH paging
requests (IMSI/TMSI for the privacy-hashed subscriber digest).
Slice the L3 payload from the datagram via data[hdr["hdr_len"]:] in
_on_datagram() and surface it as a new Observation field
`raw_l3: bytes = b""`. Default empty bytes keeps every existing
caller backward compatible — only downstream decoders that opt in
will consume it (wired in Task 5).
Test: existing test_listener_receives_a_datagram now appends a known
18-byte payload and asserts obs.raw_l3 matches verbatim.
* feat(sentinelle-gsm): l3_decode — BCCH SI + CCCH paging request parsing (ref #349)
Pure-Python TLV decoder for the minimal L3 subset needed by the
IMSI-catcher scoring engine: BCCH System Information Types 3/4/6 and
CCCH Paging Request Types 1/2/3. No scapy dependency — the parser
operates directly on the post-GSMTAP-header bytes exposed by
Observation.raw_l3 (Task 1).
Privacy invariant : the public dataclasses (CellInfo, PagedIdentity,
ParsedPagingRequest, ParsedFrame) NEVER carry plaintext IMSI/TMSI/IMEI.
The internal _try_extract_mobile_id() returns plaintext bytes;
_parse_paging() immediately hands them to Anonymizer.anonymize() and
only the HMAC-truncated subscriber_hash escapes into PagedIdentity.
The plaintext goes out of scope at function return.
Permissive parsing : every helper returns an empty CellInfo() / None on
unexpected layouts rather than raising — real GSMTAP frames have
surprising structural variants and exceptions in the consume loop would
crash livemon ingestion.
Tests : 9 cases (6 from the plan + 3 added for coverage). BCD
encoder/decoder round-trip helpers build synthetic frames
programmatically rather than hand-crafting bytes, which also validates
the TS 24.008 §10.5.1.3/4 nibble ordering.
* feat(sentinelle-gsm): baseline.py — operator-baseline learning + cell_baseline table (ref #349)
Adds CellBaseline wrapper + cell_baseline table colocated in observations.db.
Cells graduate to baseline once observed >= LEARN_THRESHOLD (default 3) times.
Key design choices:
* LEARN_THRESHOLD = 3 by default — three sightings before a cell is
considered part of the operator's legitimate baseline.
* set_learn_mode(seconds) skips the count entirely — every cell observed
inside that explicit-learn window is INSERTed with learn_count = 3 and
graduates on first sighting (used by the operator's calibration sweep).
* CellBaseline accepts a raw sqlite3.Connection (NOT an ObservationsDB
instance) so it stays decoupled from observations.py's public surface
while still sharing the same .db file.
* cell_baseline table is created idempotently (IF NOT EXISTS) in
ObservationsDB.__init__, so pre-existing observations.db files keep
working after upgrade.
Privacy invariant: BaselineCell + cell_baseline have NO subscriber-id
columns — paged identities never enter the baseline path.
Tests (5/5 passing):
- test_first_consider_starts_at_1_not_baseline_yet
- test_three_considers_graduate_to_baseline
- test_learn_mode_graduates_first_consider
- test_consider_updates_metadata_via_coalesce
- test_list_orders_by_last_learned_desc
Full pkg sweep: 62 passing (57 prior + 5 new).
* feat(sentinelle-gsm): scoring_engine — 8 heuristics + thresholds (ref #349)
Replaces the v0.1 shape-only scoring.py stub with the full v0.3.1 detector
brain. ScoringEngine wires against CellBaseline + L3Decode (Tasks 2-3)
and runs every parsed frame through 8 heuristics, summing the triggered
contributions into a CellScore clamped to [0, 100].
Heuristics + default scores (sum-clamped to 100):
1. cipher_downgrade 40 A5/X observed < baseline.cipher_a5
2. ghost_bts 35 cell unknown to baseline + paging seen
3. identity_mismatch 30 observed (mcc, mnc) != baseline pair
4. relocalization_storm 25 > 3 distinct LACs in 60s on cell
5. identity_request_abuse 35 > 2 paging reqs in 300s per hash
6. anomalous_neighbours 15 neighbour list went non-empty -> empty
7. t3212_out_of_band 15 T3212 outside [1, 60] min
8. orphan_arfcn 20 FR carrier announces ARFCN outside plan
Rolling-window state (in-memory only, no DB persistence in v0.3.1):
_lac_window cell_id -> deque[(ts, lac)]
_idreq_window subscriber_hash -> deque[ts]
_neighbour_baseline cell_id -> set[int] (cumulative ARFCNs)
Thresholds:
* DEFAULT_THRESHOLDS class constant carries per-heuristic config dicts.
* update_thresholds() does a per-heuristic deep-merge so callers can
pass partial updates like {"cipher_downgrade": {"score": 55}} without
clobbering "enabled" or any other unspecified field.
* thresholds() returns a deep-copy so callers can't mutate engine state.
Privacy invariant kept: identity_request_abuse keys windows on
subscriber_hash strings (HMAC-trunc, supplied by l3_decode.Anonymizer),
never plaintext IMSI/TMSI. Reasons truncate the hash to 8 chars defensively.
France GSM-900 ARFCN plan is approximate and permissive: unknown
(mcc, mnc) pairs return triggered=False to avoid false-positives.
Dropped lib/sentinelle_gsm/scoring.py — the v0.1.0 shape-only stub
(empty Baseline + score()=0) is replaced. No other module imported it.
Tests:
* api/tests/test_scoring_engine.py — 11 tests, one per heuristic
(plus permissive-unknown-carrier guard), one for evaluate()
aggregation/clamping, one for update_thresholds() deep-merge.
* Full sweep: 73 passed (was 62; +11 new).
* feat(sentinelle-gsm): consume loop wires L3 + scoring + trusted match → Alert (ref #349)
- Extracted `_process_observation(obs)` helper for testability — the consume
coroutine now delegates per-frame work so tests can drive synthesized
Observations through the full pipeline without binding UDP.
- New endpoints: GET /baseline, POST /baseline/learn (sweep window),
GET /scoring/thresholds, POST /scoring/thresholds (deep-merge).
- Audit log entry on POST /scoring/thresholds via the existing
`secubox.sentinelle-gsm.api` logger — surfaces in /journal/stream.
- Alert emission decision tree:
score < threshold → drop (no write)
score >= threshold + match → one labeled alert per matched trusted phone
score >= threshold + no match → single anomaly-only alert
- ALERT_THRESHOLD configurable via SENTINELLE_ALERT_THRESHOLD env (default 60).
- TrustedRegistry now hashes the IMSI as `imsi.encode("ascii").hex()` so
the persisted token matches the canonical form produced by the L3
decoder's paging-request path. Existing trusted tests don't pin the
hash value, so behaviour is unchanged for read-back / lookup-by-id.
- 5 new integration tests at api/tests/test_alert_emission.py cover the
below-threshold drop, anomaly-only emission, trusted-match labeling,
paging-event hash persistence, and audit-log emission on threshold
update. Full sweep 78/78 passing.
* feat(sentinelle-gsm): webui baseline + scoring panels + trusted_label chip (ref #349)
Adds two new webui panels and a trusted-label chip on the alerts row:
- #baseline panel: Cell ID / MCC / MNC / LAC / ARFCN / cipher / learn count
/ last learned table backed by GET /api/v1/sensor/gsm/baseline. "Start
learn (5 min)" button arms POST /baseline/learn?sweep_seconds=300 and
re-polls the table every 15s for the 5-minute window so cells appear as
they graduate.
- #scoring panel: per-heuristic enable + score input (0..100, clamped
client-side) backed by GET/POST /scoring/thresholds. Form values are
cached in _thresholdState so any backend-only fields (heuristic-specific
knobs) survive the POST round trip.
- Alerts table: trusted_label is now rendered as a violet pill via the
new .trusted-chip class (replaces the older .target-pill markup for
the same data path).
Local navigation gets two new anchors (Baseline, Scoring) between
Observations and Actions. The 10s scan poll now also refreshes baseline
while a scan is running.
Wires 4 new event listeners (refresh-baseline, baseline-learn,
refresh-scoring, save-scoring). No backend changes; all endpoints already
exist (Tasks 1..5).
Tests: 78/78 passing.
* chore(sentinelle-gsm): bump 0.3.1 + extend privacy invariant tests (closes#349)
Append 4 new structural privacy tests under tests/test_privacy_invariant.py
to lock the v0.3.1 invariants in place:
* test_l3_decode_returns_no_plaintext_fields — CellInfo, PagedIdentity,
and ParsedPagingRequest dataclasses must not carry any plaintext
subscriber-id field (imsi/tmsi/imei/msisdn/iccid/subscriber_id).
* test_paging_request_hashes_paged_identities — feeds a synthetic
Paging Request Type 1 with a known TMSI through L3Decode and asserts
the plaintext TMSI bytes (as hex) NEVER appear in the resulting
PagedIdentity.subscriber_hash.
* test_cell_baseline_has_no_subscriber_fields — the BaselineCell
dataclass must remain operator-side metadata only: no subscriber_hash,
no plaintext identifier.
* test_scoring_engine_reasons_contain_no_plaintext_imsi — the free-text
reason strings produced by ScoringEngine.evaluate() must never echo
a 15-digit token (plaintext-IMSI shape) for typical observations.
These four tests pass against the v0.3.1 modules (Tasks 1-6) with no
code change — the modules were designed for these invariants from the
start, this commit just locks them in regression-test form.
Full sweep: 78 -> 82 passing.
Changelog: 0.3.0 -> 0.3.1 with the v0.3.1 entry summarising the new
l3_decode, baseline, scoring_engine modules, the consume-loop wiring
that emits qualified Alerts with trusted_label, the webui panels, and
the updated test breakdown (l3_decode x9, baseline x5, scoring_engine
x11, alert_emission x5, privacy x4 = 34 new, full sweep 82/82).
---------
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
* docs(plan): #347 sentinelle-gsm v0.3.0 observation pipeline (ref #344#347)
* feat(sentinelle-gsm): livemon_runner — managed grgsm subprocess (closes#344 ref #347)
Introduces LivemonRunner, the asyncio subprocess manager that owns the
single grgsm_livemon_headless child for v0.3.0.
Device-claim invariant (the load-bearing point that closes#344): the
parent service NEVER opens the RTL-SDR itself. The RTL-SDR is claimed
ONLY by the spawned grgsm_livemon_headless child, and ONLY for the
lifetime of an active scan. start() spawns with `--args=rtl=0 -f <freq>`
so gr-osmosdr binds RTL-SDR instead of its UHD default; stop() sends
SIGTERM and falls back to SIGKILL after a 5 s timeout, releasing the
device cleanly.
Concretely this means ad-hoc tooling (rtl_test, rtl_adsb, kalibrate)
keeps working alongside an idle secubox-sentinelle-gsm service — the
race the v0.1 livemon.py USB-detect stub created is gone, because the
parent has no reason to touch the dongle outside an explicit scan.
Stderr from the child is drained into a 16 KiB-capped ring buffer so
status() can surface the last 2 KiB without unbounded memory growth,
and double-start() raises RuntimeError instead of silently leaking
processes.
Tests cover: initial-state, RTL args propagation to create_subprocess_exec,
double-start refusal, SIGTERM-on-stop + state clear, and stop-as-noop
when nothing is running. asyncio.create_subprocess_exec is fully mocked
— no real grgsm process is spawned by the test suite.
Test sweep: 5 new + 29 existing = 34 passing.
* feat(sentinelle-gsm): gsmtap_listener — async UDP 4729 + GSMTAP v2 parse (ref #347)
Task 2 of v0.3.0 observation pipeline.
- `lib/sentinelle_gsm/gsmtap_listener.py`: asyncio DatagramProtocol bound
to 127.0.0.1:4729 (configurable). Tiny manual 16-byte GSMTAP v2 header
parser — no scapy runtime dep on the hot path. scapy stays reserved
for the deeper L3 decode (BCCH/CCCH paging) landing in v0.3.1.
- `Observation` dataclass exposes ts/arfcn/frame_nr/channel/sub_type plus
optional lac/ci/mcc/mnc/cell_id/subscriber_hash. No plaintext-id field;
`subscriber_hash` is HMAC-only (populated when L3 decode lands).
- Bounded queue (maxsize=2048); on QueueFull we drop silently —
observations are stochastic, backpressure to the radio would be worse.
- 4 tests: minimum-valid parse, too-short reject, wrong-version reject,
end-to-end UDP roundtrip on port 47291 (avoids colliding with a
running grgsm on 4729).
All tests pass: 4/4 new, 38/38 sweep.
* feat(sentinelle-gsm): observations.py — SQLite sightings + paging_events (ref #347)
Adds ObservationsDB persistence layer for the v0.3 observation pipeline.
* Sighting (cell_id PK, mcc/mnc/lac/ci/arfcn, first_seen/last_seen,
sighting_count) — UPSERT semantics: INSERT on new cell_id, else bump
sighting_count + refresh last_seen with COALESCE on optional fields so
partial updates don't clobber previously-observed values.
* PagingEvent (ts, cell_id, subscriber_hash, request_type) — seeds the
paging history table that v0.3.1 will hash-match against trusted_phones.
Schema deliberately omits plaintext IMSI/TMSI/IMEI fields.
* _guard_plaintext() mirrors the regex guard in alert_sink.AlertSink.write
(same \b\d{15}\b pattern, same "plaintext-IMSI shape detected — refusing
write" error contract). Applied on every write path: upsert_sighting
guards cell_id; record_paging guards both cell_id and subscriber_hash.
* Two indexes on paging_events (ts, cell_id) for the v0.3.1 lookup path.
* 4 tests cover the contract: new-insert, bump-count, hash accept, and
the plaintext-IMSI refusal. Full sweep: 42 passing (38 prior + 4 new).
* feat(sentinelle-gsm): API — /scan/start /scan/stop /scan/status /observations (ref #347)
Task 4 of the v0.3.0 observation-pipeline plan. Wires the LivemonRunner +
GsmtapListener + ObservationsDB introduced in Tasks 1–3 to four new HTTP
endpoints behind the existing nginx + Authelia JWT gate.
- POST /scan/start {freq} → bind GSMTAP UDP listener, spawn
grgsm_livemon_headless on freq, start a
background consume task that drains
Observations into the SQLite store
- POST /scan/stop → cancel the consume task FIRST so no
orphan subprocess is left behind, then
SIGTERM the runner, then close the
listener socket (the CHILD owns the
RTL-SDR, never the parent)
- GET /scan/status → current runner state (running / pid /
freq / started_at / stderr_tail)
- GET /observations?limit → recent cell sightings, newest first;
limit clamped to [1, 1000], default 200
Synthetic cell_id formation until v0.3.1's L3 BCCH decode lands:
`f"arfcn-{obs.arfcn}-ch-{obs.channel}"`. The ObservationsDB schema
already accepts NULL for the operator quadruple (mcc/mnc/lac/ci) so
nothing downstream has to change when the real cell_id arrives.
Three new tests in api/tests/test_scan_api.py exercise the
happy path with mocked LivemonRunner + mocked GsmtapListener (so no
gr-gsm binary is spawned and no UDP socket is bound) plus a real
ObservationsDB under tmp_path so GET /observations exercises the
real SQLite path. Fixture clears `_consume_task` between tests to
keep them order-independent.
Test sweep: 42 (existing) + 3 (this) = 45 passing.
* feat(sentinelle-gsm): webui scan control + observations panels (ref #347)
Adds two new panels to the standalone MIND webui:
* Scan control — Start/Stop buttons, freq input (default 925.4M = E-GSM 900
ARFCN 975), 4-cell status row (state dot, freq, pid, started_at), and
a collapsible <details> showing the last 2 KB of stderr from
grgsm_livemon_headless. Start sends {freq: <value>} to /scan/start.
* Observations — live table of cell sightings with columns Cell ID,
ARFCN, MCC, MNC, LAC, CI, First seen, Last seen, Sightings. Badge in
the panel header tracks the current count.
JS additions (sentinelle.js):
* loadScanStatus / startScan / stopScan / loadObservations helpers
* 3 new event listeners (btnScanStart, btnScanStop, btnRefreshObs)
* Background interval (10s) that polls /scan/status always and
/observations only while a scan is running — keeps the post-scan
terminal state on screen without hammering sqlite.
CSS additions (sentinelle.css):
* .scan-status-row (4-col grid inside a panel, matches .status-strip)
* .scan-stderr / .scan-stderr-details (monospace tail block, MIND palette)
* Responsive collapse to 2 cols then 1 col matching the existing breakpoints.
API surface untouched — Tasks 1-4 already shipped /scan/start, /scan/stop,
/scan/status and /observations under /api/v1/sensor/gsm/. Test sweep still
45 passing.
* build(sentinelle-gsm): merge detect_rtlsdr_usb into livemon_runner; drop v0.1 stub; add scapy + gr-gsm deps (ref #344#347)
Co-locate RTL-SDR USB detection with the grgsm_livemon orchestrator.
- Move RTLSDR_USB_IDS + detect_rtlsdr_usb() from livemon.py into
livemon_runner.py. The function is pure sysfs (reads
/sys/bus/usb/devices/*/idVendor + idProduct) — it does NOT claim
the USB device, so it was never the source of #344. The #344 bug
was fixed by Task 1's LivemonRunner model (child-only device claim
via subprocess.create_subprocess_exec).
- Delete lib/sentinelle_gsm/livemon.py — the v0.1 stub also carried a
useless NotImplementedError docstring for listen_gsmtap_udp, fully
superseded by gsmtap_listener.py (Task 1).
- api/main.py: collapse the two-line "from sentinelle_gsm.livemon ...
/ from sentinelle_gsm.livemon_runner ..." block into a single import
pulling LivemonRunner + detect_rtlsdr_usb from livemon_runner.
- debian/control: promote python3-scapy from Recommends to Depends
(runtime requirement once the v0.3.1 L3 decode path ships); keep
gr-gsm in Recommends (subprocess that runs scans).
dpkg-buildpackage: ok (secubox-sentinelle-gsm_0.2.3-1~bookworm1_all.deb).
dpkg-deb -c | grep livemon: only livemon_runner.py + test_livemon_runner.py,
no v0.1 stub. pytest api/tests/ tests/: 45 passed.
* chore(sentinelle-gsm): bump 0.3.0 + extend privacy invariant tests (closes#344#347)
Privacy invariant coverage extended for the v0.3.0 observation pipeline:
* test_observation_has_no_plaintext_fields — Observation dataclass
(gsmtap_listener) exposes ONLY subscriber_hash, never plaintext
IMSI/TMSI/IMEI/MSISDN/ICCID/subscriber_id.
* test_paging_event_only_stores_hash — PagingEvent dataclass
(observations) has no plaintext-identifier fields and exposes
subscriber_hash.
* test_observations_db_refuses_plaintext_imsi — ObservationsDB.record_paging
raises ValueError("plaintext-IMSI ...") when handed a 15-digit
subscriber_hash (write-time shape guard).
Version bump 0.2.3 → 0.3.0 with the full v0.3.0 changelog:
livemon_runner.py (RTL-SDR subprocess wrapper, closes#344) +
gsmtap_listener.py (asyncio UDP 4729) + observations.py (SQLite
sightings/paging_events) + /scan API + webui panels + scapy promoted
to Depends + dropped v0.1 livemon.py stub. 48/48 tests passing.
---------
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
* docs(plan): #340 secubox-sentinelle-gsm v0.2 standalone webui + local alerts (ref #340)
* feat(sentinelle-gsm): alert_sink — SQLite history + SSE broadcast hub (ref #340)
Introduces lib/sentinelle_gsm/alert_sink.py: a SQLite-backed alert
history with an in-memory asyncio pub/sub hub that fans new alerts
out to live Server-Sent Events subscribers.
The Alert dataclass carries only privacy-safe fields — `subscriber_hash`
is HMAC-truncated (Anonymizer), never plaintext. AlertSink.write()
enforces this with a load-bearing privacy guard: any string field
matching `\b\d{15}\b` (plaintext IMSI shape) triggers ValueError and
the row is refused. SQLite is opened with check_same_thread=False so
the FastAPI app can read/write from multiple coroutines safely.
Adds api/tests/test_alert_sink.py with 5 tests covering write/list
roundtrip, the plaintext-IMSI privacy guard, HMAC acceptance, live
subscriber fan-out, and SSE chunk format. 5 passed locally.
* feat(sentinelle-gsm): trusted phones registry — HMAC-hashed IMSI + label (ref #340)
Privacy contract:
- Plaintext IMSI is accepted by add() ONCE, immediately HMAC-hashed via
the existing sentinelle_gsm.observer.Anonymizer.anonymize() helper.
- The plaintext never persisted, never logged, never returned by any
method. After add() returns, plaintext_imsi is out of scope.
- The on-disk store contains only {id, imsi_hash, label, added_at}.
Storage format: JSON via the stdlib `json` module instead of TOML.
`python3-tomli-w` (the only writer for the new tomllib reader) is not
in Debian bookworm — it first ships in trixie. python3-toml (legacy)
writes TOML but is deprecated. JSON is stdlib, atomically renderable,
and human-grep-able, which is what a trusted-phones registry needs.
Anonymizer integration: uses .anonymize() (HMAC-SHA256, 16 hex chars
truncated). The plan referenced .hash_subscriber_id() — that method
does not exist on Anonymizer in observer.py; .anonymize() is the
canonical hashing entry point.
Tests: 6/6 passing
- test_add_hashes_and_does_not_store_plaintext (privacy invariant
enforced: plaintext IMSI never appears on disk)
- test_lookup_by_hash_finds_added_phone
- test_lookup_by_hash_returns_none_for_unknown
- test_delete_removes_entry
- test_invalid_imsi_rejected
- test_persistence_across_instances
* feat(sentinelle-gsm): API — /alerts (history + SSE + test) + /trusted CRUD (ref #340)
Adds 6 endpoints under the existing /api/v1/sensor/gsm/ prefix
(nginx-added; mounted at root of the FastAPI app):
GET /alerts — paginated SQLite-backed history
(query: ?limit=100&since=<epoch>)
GET /alerts/stream — SSE live feed; Cache-Control: no-cache,
X-Accel-Buffering: no, Connection: keep-alive
so nginx and HTTP caches don't buffer it
POST /alerts/test — operator manual trigger; ValueError from
the AlertSink privacy guard (plaintext-IMSI
shape) surfaces as HTTP 500
GET /trusted — list trusted phones (HMAC + label, no plaintext)
POST /trusted — body {imsi, label}; ValueError → 400
DELETE /trusted/{id} — unknown id → 404
Wiring:
* AlertSink + TrustedRegistry are module-level singletons initialised
on FastAPI startup so tests can override via monkeypatch.
* `require_jwt` is a no-op dependency stub — real JWT enforcement is
done by nginx + Authelia in front of the Unix socket. The stub
exists so test code (and future host-direct callers) can swap auth
via `app.dependency_overrides[require_jwt]`.
* Anonymizer is loaded via a new `_get_anonymizer()` helper that
reads /etc/secubox/secrets/sentinelle-gsm-hmac when present and
falls back to an ephemeral key otherwise.
* trusted.json path is /etc/secubox/sentinelle-gsm/trusted.json
(the Task-2 deviation from the plan's TOML; bookworm lacks
python3-tomli-w).
* version bumped 0.1.0 → 0.2.0 in the FastAPI metadata only;
the debian/changelog bump lands in Task 6.
Tests (7 new in api/tests/):
* test_alerts_api.py — POST /alerts/test happy path, GET history
roundtrip, plaintext-IMSI guard → 500, SSE route registration
(via app.routes + /openapi.json, not by opening the stream — the
async generator awaits forever on q.get(), incompatible with sync
TestClient iteration; SSE chunk format is covered by
test_alert_sink.py::test_stream_emits_sse_format).
* test_trusted_api.py — add/list/delete roundtrip with hash-not-
plaintext invariant, invalid IMSI → 400, unknown id → 404.
Run: pytest api/tests/test_alerts_api.py api/tests/test_trusted_api.py
Result: 7 passed.
* feat(sentinelle-gsm): standalone webui — live alerts + browser Notification API (ref #340)
Adds the Task 4 standalone WebUI for sentinelle-gsm at
packages/secubox-sentinelle-gsm/www/sentinelle/:
- index.html — canonical SecuBox scaffold (body flex, aside.sidebar 220px
fixed with section anchors, main reserves the global menu bar with
padding-top: calc(48px + 1.5rem)). Four panels: status strip, alerts
table (aria-live=polite), trusted phones table, actions.
- sentinelle.css — MIND palette (--mind-violet: #3D35A0) per Charte
SecuBox, Space Grotesk titles + JetBrains Mono data/code, modal with
z-index 1500 (above the global-menu-bar z-index 998), toast banner,
score chip colour grading (low/med/high), responsive breakpoints.
- sentinelle.js — vanilla single-IIFE module, no framework, no CDN.
EventSource consumer of /api/v1/sensor/gsm/alerts/stream (auto-reconnect
surface), CRUD wrappers for /alerts /alerts/test /trusted /trusted/{id},
Browser Notification API (requested from a user-gesture handler), beep
synthesised on-the-fly via Web Audio API (no asset), localStorage mute
toggle, modal-based add flow, ESC/backdrop close, accessible aria-live
region for the alerts list.
All API calls hit root-relative /api/v1/sensor/gsm/* — Task 5 will mount
the nginx route. Local smoke test booted uvicorn against the API and
verified the HTML/CSS/JS serve and all four CRUD/test routes respond as
the UI expects.
* feat(sentinelle-gsm): nginx route /sentinelle/ + MIND menu entry + .install (ref #340)
* nginx/sentinelle-webui.conf: alias for /sentinelle/ (static webui) +
SSE-tuned override for /api/v1/sensor/gsm/alerts/stream. Uses `^~`
literal-prefix match so it wins precedence over the shorter
/api/v1/sensor/gsm/ prefix in sentinelle-gsm.conf and against any
future regex locations in the same server scope. Re-sets
proxy_read_timeout / proxy_send_timeout to 24h after the common
secubox-proxy include (which forces 30s) so long-lived SSE
connections survive.
* menu.d/45-sentinelle.json: MIND-category entry (icon: satellite,
order: 45). Category emitted as lowercase `mind` to match the
canonical schema used by every other package menu.d entry.
* debian/secubox-sentinelle-gsm.install: ship the new files. The
package uses a hand-rolled override_dh_auto_install in debian/rules,
so the install file is documentation; debian/rules is extended
with explicit cp steps for nginx/sentinelle-webui.conf and
www/sentinelle/. menu.d already had a wildcard copy.
* chore(sentinelle-gsm): bump 0.2.0 + extend privacy invariant tests (closes#340)
- Append 3 new privacy invariant tests:
* Alert dataclass has no plaintext IMSI/IMEI/TMSI/MSISDN/ICCID fields
* TrustedPhone dataclass only stores imsi_hash (never plaintext)
* AlertSink.write() raises ValueError on any 15-digit token in a
textual field — non-regression guard for the OPAD-doctrine privacy
invariant.
- Bump debian/changelog to 0.2.0-1~bookworm1 with the full Task 1-6
feature summary: alert_sink, trusted registry, REST routes, WebUI,
nginx + Hub menu integration. Notes the JSON-over-TOML deviation
(python3-tomli-w not in bookworm).
- 29/29 tests pass: 5 alert_sink + 4 alerts_api + 3 trusted_api +
6 trusted_registry + 11 privacy_invariant (8 prior + 3 new).
---------
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
* docs(plan): #331 streamlit idle-timeout + wake-on-demand (ref #331)
* feat(streamlit): add idle-tracking helpers to streamlitctl (ref #331)
Add the foundation for per-app idle timeout / wake-on-demand:
- IDLE_STATE_DIR constant at /var/lib/secubox/streamlit/idle
- _idle_config: read [idle] settings from /etc/secubox/streamlit.toml
with safe defaults
- _app_running: check whether an app is listening on its port INSIDE
the LXC (existing inline ss check is host-side and unreliable)
- _app_active_conns: count ESTABLISHED connections to an app's port
- _app_last_active / _app_touch_active: per-app last-activity epoch
tracked via state-file mtime
Wire cmd_app_start to touch the state file on every successful start
so a freshly-started app does not get immediately reaped by the
upcoming idle-check.
All helpers degrade gracefully when the LXC is down (lxc_running
guard) and the helpers themselves do not change any existing
behaviour — they are purely additive.
* feat(streamlit): streamlitctl app idle-check — stop idle apps past timeout (ref #331)
Add cmd_app_idle_check that iterates apps under $APPS_PATH and stops
those that have had zero ESTABLISHED connections for longer than the
configured timeout_minutes (default 30, from [idle] in streamlit.toml).
Apps with active connections get their state file touched; apps with no
state file yet are touched too (grace period). Master switch via [idle]
enabled in /etc/secubox/streamlit.toml.
Wire idle-check + wake into the `app` subcommand dispatcher; wake's
function (cmd_app_wake) lands in the next commit (dead branch until
then). Usage line updated to include both new subcommands.
* feat(streamlit): streamlitctl app wake — lazy start + poll for port (ref #331)
Adds cmd_app_wake <name> [wait_seconds] (default 30s):
- returns 0 if already running OR if the app comes up within wait_seconds
- returns 1 on start failure or poll timeout
- returns 2 on misuse (missing name, unknown app)
Uses _app_running for readiness probing and _app_touch_active to refresh
the idle marker on a successful wake. Polls every 1s for portability.
The case-block dispatcher (idle-check / wake) was already wired in Task 2.
* feat(streamlit): systemd idle-sweep timer (every 5 min) (ref #331)
Ship secubox-streamlit-idle.service (oneshot, runs
`streamlitctl app idle-check`) and secubox-streamlit-idle.timer
(OnBootSec=5min, OnUnitActiveSec=5min) so idle Streamlit apps are
auto-suspended without a separate cron.
Wire the units through debian/secubox-streamlit.install (the package
uses a hand-rolled override_dh_auto_install, so dh_installsystemd's
auto-detection by package.suffix name doesn't pick them up — explicit
.install is needed). Enable/disable the timer in postinst/prerm
alongside the main service.
* feat(streamlit): POST /apps/{name}/wake — lazy start API endpoint (ref #331)
* feat(streamlit): default [idle] config block (timeout 30 min) (ref #331)
* chore(streamlit): bump to v1.2.2 with idle-timeout + wake summary (closes#331)
---------
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
Field-deploying v2.11.0's three new LXC modules on arm64 MOCHAbin
(gk2 192.168.1.200) surfaced nine bugs that the local smoke build
doesn't exercise (no lxc-create in dpkg-buildpackage). All three
modules now reach overall=green on the board with the canonical hub
vhost (admin.gk2.secubox.in) routing /api/v1/<m>/* and /<m>/ correctly.
install-lxc.sh (all 3 modules)
1. lxc-create -t download (the legacy `debian` template fails on
bookworm unprivileged containers).
2. ensure_masquerade() — nftables MASQUERADE rule for 10.100.0.0/24,
idempotent. systemd-networkd's IPMasquerade=ipv4 doesn't land on
appliances using ifupdown/NetworkManager.
3. ensure_resolv() — seed /etc/resolv.conf via lxc-attach AFTER
start_lxc + wait_for_network (rootfs is owned by mapped uid 100000,
host root can't write directly). Unlink the symlink the download
template ships (→ /run/systemd/resolve/stub-resolv.conf which
doesn't exist before systemd-resolved is up).
install-lxc.sh (grafana)
4. gpg --dearmor on apt.grafana.com key (signed-by= needs binary,
wget delivers ASCII-armored).
install-lxc.sh (yacy)
5. YACY_RELEASE_URL: v1.940 → v1.941 (old URL now returns 404).
install-lxc.sh (rustdesk)
6. Detect host arch (arm64/amd64/armhf) and pull the matching
RustDesk server build. Hardcoded amd64 binary failed with
"Exec format error" on the arm64 MOCHAbin.
7. Flatten any of amd64/, arm64v8/, armv7/ subdirs the zip ships.
debian/secubox-<m>.service (all 3 modules)
8. ExecStartPre prefix with `+` (run as root). The mkdir + chown
pre-steps run as User=secubox by default, chown fails ("Operation
not permitted") and the service can't start.
debian/rules + nginx/<m>.conf (all 3 modules)
9. Install nginx snippet to BOTH /etc/nginx/secubox.d/ AND
/etc/nginx/secubox-routes.d/. The canonical hub vhost
(sites-enabled/webui.conf serving admin.gk2.secubox.in) only
includes routes.d/, NOT secubox.d/ which is only included by the
legacy sites-available/secubox vhost. Also switch the
/api/v1/<m>/ location from `proxy_pass http://unix:.../sock:/` to
explicit `rewrite ^/api/v1/<m>/(.*)$ /$1 break; proxy_pass
http://unix:.../sock;` — matches the canonical pattern used by
secubox-users; the previous syntax did not strip the prefix so
FastAPI saw the full path and returned 404 on every route.
Docs
* docs/MODULE-GUIDELINES.md §7 extended with mandatory lifecycle
verbs install / repair / wizard / reload / uninstall. Repair detects
+ fixes drift idempotently; wizard chains interactive seed + install;
uninstall asks for confirmation, removes LXC + state + secrets.
Implementation in grafanactl/yacyctl/rustdeskctl deferred to v2.12.0
(currently install + reload are wired, the rest is on the v2.12.0
backlog).
* docs/superpowers/specs/2026-05-20-secubox-wall-ep06.md (#236) —
Quectel EP06-E modem-based rogue-base-station sensor (WALL layer).
* docs/superpowers/specs/2026-05-20-secubox-sentinelle-gsm.md (#237) —
RTL-SDR + gr-gsm passive RX-only false-BTS detection (MIND layer,
feeds WALL/OPAD). Privacy-by-design HMAC-truncated identifiers.
* .claude/{WIP,TODO}.md refreshed with v2.11.1 fix trail + v2.12.0
target + P2 sensor backlog.
Package version bump: 1.0.0 → 1.0.7 (grafana/rustdesk) / 1.0.7 (yacy),
all three end at the same .deb revision suitable for v2.11.1 tag.
Validated end-to-end on board gk2:
grafanactl status → overall=green (lxc + grafana-server + host-api all up,
admin/<password> at /etc/secubox/secrets/grafana-admin,
6 dashboards + secubox-metrics datasource provisioned)
yacyctl status → overall=green (lxc + JVM + YaCy v1.941 standalone,
admin password at /etc/secubox/secrets/yacy-admin)
rustdeskctl status → overall=green (lxc + hbbs + hbbr + host-api,
PSK at /etc/secubox/secrets/rustdesk-key,
nftables DNAT 21116/udp installed)
https://admin.gk2.secubox.in/api/v1/grafana/healthz → HTTP 200
https://admin.gk2.secubox.in/api/v1/yacy/healthz → HTTP 200
https://admin.gk2.secubox.in/api/v1/rustdesk/healthz → HTTP 200
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
- docs/MODULE-GUIDELINES.md (new) — canonical reference for authoring a
secubox-<module> package: file structure, LXC layout, WebUI conventions,
nginx wiring, Debian packaging, CTL three-fold, FastAPI shape, tests,
validation checklist. Sister doc to docs/grammar.md and docs/UI-GUIDE.md.
- docs/grammar.md — cross-reference MODULE-GUIDELINES.md in the header
sister-docs block.
- docs/MODULES.md — point new-module authors at MODULE-GUIDELINES.md.
- docs/superpowers/plans/2026-05-20-grafana-yacy-rustdesk-lxc-modules.md
(new) — concrete implementation plan for three new LXC-hosted modules:
secubox-grafana (#229), secubox-yacy (#230), secubox-rustdesk (#231).
All three follow the new MODULE-GUIDELINES.md. IP allocations 10.100.0.{70,80,90}.
Permanent documentation for the 8-verb / 8-layer modular tools box that
emerged across #173, #176, #180-#184, #190, #212. Three new docs + one
README section, all bearing the 1991 attribution.
docs/grammar.md Canonical conceptual frame: layered map,
8 verbs in a table, composing-the-grammar
worked examples, naming conventions,
philosophical roots.
HOWTO-grammar.md Concrete 6-step walkthrough for adding a
9th verb (~1h budget): issue framing,
worktree, CTL skeleton (Bash + Python),
FastAPI mirror, Debian packaging, nginx
route under secubox-routes.d/, live test
on the board, commit/PR conventions,
common pitfalls drawn from real incidents
(#163 secubox.d vs secubox-routes.d,
#173 LXC default path, #176 hook trigger,
#194 CAP_NET_ADMIN, #180 naming).
wiki/CTL-Grammar.md Public-facing wiki page mirrored to the
GitHub wiki repo (per the wiki-sync
pattern established for Acknowledgments).
Embeds Punk Exposure (Peek/Poke/Emancipate)
as the seed concept with 1991 attribution.
README.md New section under Features summarising the
grammar with the 9-row layer table and a
pointer to docs/grammar.md + HOWTO-grammar.md.
Attribution recorded on all three documents: copyright spiritual
concept Gerald Kerma (GK2), 1991, Notre-Dame-du-Cruet, Savoie.
The CTL is the user's voice. The modular tools box is their organ. The
language is the way they say *this is mine, this is mine, this is mine*.
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
Anti-regression for the double-cache pattern documented in CLAUDE.md
(« Performance Patterns — Double Caching »). Catches the bug class that
caused PR #146 (secubox-waf 17 % sustained CPU under SOC polling)
before it lands.
What's added
============
- scripts/check-dashboard-cache.py — AST-based audit. Walks every
packages/secubox-*/api/main.py, flags hot dashboard routes
(/stats, /alerts, /summary, /dashboard, /metrics, /overview,
/events, /history, /recent, /bans, /sessions, /connections)
whose handler body does per-request I/O (open, read_text,
read_bytes, subprocess.run, Popen) AND doesn't go through a cache
(module-level _cache instance + .get() call, OR
asyncio.create_task(refresh_*) at startup).
* --check : exit 1 on unjustified violations (CI mode)
* --report : human-readable table (default)
* --json : machine-readable
- .cache-lint-allowlist.toml — TOML allowlist for justified
exceptions. Each entry needs a daemon + route + justification
(issue link expected). Seeded with the 4 current legacy cases:
secubox-waf (covered by PR #146 — will drop the entries when
merged), secubox-config-advisor /history (admin drill-in, not
polled), secubox-netdiag /connections (real-time TCP state),
secubox-tor /summary (small file read, follow-up cleanup).
- .github/workflows/dashboard-cache-check.yml — mirror of
license-check.yml. Runs the lint + the self-tests on PRs that
touch packages/**/api/main.py, the lint script, the allowlist,
or the workflow itself.
- docs/CACHE-PATTERN.md — short remediation guide. Three accepted
shapes (read-through, background-refresh, pure in-memory), code
example, allowlist syntax, references to the canonical
implementations (secubox-crowdsec for read-through, secubox-system
for background-refresh).
Tests (14/14 pass)
==================
- StatsCache pattern detection (compliant cases)
- Non-compliant detection: open, subprocess.run, multiple I/O ops
- Pure in-memory handler = compliant
- Background-refresh task = compliant
- Non-hot route (/health) not audited
- @router.get treated like @app.get
- Allowlist suppresses by exact (daemon, route) key
- Allowlist doesn't cross-pollinate across routes
- Missing allowlist file = no error
- CLI --check exits 1 on unjustified
- CLI --check exits 0 when clean
- CLI --json output well-formed
- **Real repo audit with seeded allowlist passes --check**
(sanity gate: catches regression if anyone removes an allowlist
entry without fixing the daemon)
Baseline against the real packages/ tree
========================================
122 daemons audited, 118 compliant, 4 with findings (all
allowlisted). When PR #146 merges, the secubox-waf entries can be
removed from the allowlist — the next lint run will then prove the
fix is wired correctly.
Follow-up
=========
- Drop secubox-waf allowlist entries when #146 merges.
- Per-daemon follow-up issues for the remaining 3 (config-advisor,
netdiag, tor) — not blocking, allowlist documents the rationale.
- Consider extending HOT_ROUTES to /health / /status once the small
daemons that call `systemctl is-active` adopt a 10 s cache.
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(mitmproxy): cookie_audit addon — Set-Cookie ledger for RGPD (ref #156)
New mitmproxy addon that hooks the response flow and appends every Set-Cookie
header to /var/log/secubox/cookie-audit/server.jsonl. Cookie values are
sha256-hashed at the addon — the raw value never leaves the process.
Companion to the upcoming browser-side cookie-inventory.js + the
CookieAuditAggregator that will reconcile both streams for RGPD/ePrivacy
audit on operator-owned vhosts.
Also adds the implementation plan under docs/superpowers/plans/.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(hub): cookie-inventory.js browser snapshotter (ref #156)
Vanilla JS module that snapshots document.cookie at DOMContentLoaded, +2s,
and on visibilitychange. Names + sha256(value) are POSTed to
/api/v1/cookie-audit/ingest with credentials:'omit'. Hard-capped at 8
snapshots per page to avoid amplification.
Loaded into every HTML response via the WAF banner injection (next commit).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(waf): inject cookie-inventory.js alongside health banner (ref #156)
Extends the existing CDN banner injection block to also append a script tag
loading shared/cookie-inventory.js. Adds two new CDN-config keys:
- cookie_inventory_url (default: admin.gk2.secubox.in/shared/...)
- cookie_audit_ingest_url (default: admin.gk2.secubox.in/api/v1/...)
Both health banner and cookie inventory now load from the same injected
guard block, so a single </body> replacement covers both.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(metrics): CookieAuditAggregator + Classifier (ref #156)
Aggregator class mirroring CertStatusAggregator: tails the mitmproxy JSONL
Set-Cookie ledger and the browser-snapshot ingest dir, then reconciles per
(vhost, cookie-name).
Per-cookie source verdict:
* http — server only
* js — browser only -> RGPD violation unless strictly_necessary
* both — server + browser
Classifier loads regex rules from config, checks categories in order
(strictly_necessary > functional > analytics > marketing), first match wins.
Default cache file: /var/cache/secubox/metrics/cookie-audit.json.
9 unit tests covering parsing, reconciliation, persistence and malformed
input.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(metrics): wire /api/v1/cookie-audit/{ingest,report,summary} (ref #156)
Imports + initializes CookieAuditAggregator, schedules it in the FastAPI
lifespan alongside the other aggregators, opens up CORS to POST (was GET
only) for the ingest endpoint, and adds three routes:
POST /api/v1/cookie-audit/ingest — browser snapshot ingest (rate-bounded,
credentials: omit, returns refused if
audit disabled in config)
GET /api/v1/cookie-audit/report?host=… — per-vhost or global report
GET /api/v1/cookie-audit/summary — global rollup
Hard caps: 200 cookies/snapshot, 128B names, 128B hashes, 512B UA.
All 34 existing metrics tests + 9 new cookie-audit tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(core): get_cookie_audit_config + RGPD classifier baseline (ref #156)
New config helper get_cookie_audit_config() that returns the merged
[cookie_audit] section with built-in RGPD/CNIL classifier patterns.
Categories (eval order — first match wins):
strictly_necessary PHPSESSID, csrftoken, cart, remember_token, ...
functional lang, locale, theme, cookie_consent, euconsent, ...
analytics _ga*, _gid, _gat*, _pk_*, _hjid, _clck, _matomo*, ...
marketing _fbp, _fbc, __utm*, _gcl_*, _uet*, IDE, MUID, NID
Operator patterns from [cookie_audit.classifier] are MERGED on top of the
baseline by default. Set classifier_override = true to replace it entirely
(opt-out for sites that need a different taxonomy).
secubox.conf.example documents the section with commented-out examples.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: cookie-audit API + addon (ref #156)
- secubox-metrics README: new "Cookie Audit" section documenting the
three endpoints, the source verdict semantics (http/js/both), and the
rgpd_violation rule.
- secubox-mitmproxy README: new "Addons" section listing both
secubox_waf and cookie_audit + the dual -s mitmdump invocation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(claude): record cookie-audit pipeline in HISTORY+WIP (ref #156)
HISTORY 2026-05-16 entry captures scope (server ledger + browser snapshot
reconciler), the 8 implementation tasks landed in the worktree, and the
two follow-ups left (AppArmor + logrotate).
WIP gets a top entry pointing at the worktree/branch with the test
totals and "awaiting validation, no PR opened" status.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(banner): add Cookie Audit live tile + summary URL injection (ref #156)
Adds a "CookieAudit" section to the health-banner live panel alongside
VisitorOrigin / LiveHosts / CertStatus. It polls /api/v1/cookie-audit/summary
every 30s and renders:
- host count
- RGPD verdict row (red ⚠ when violation_count > 0, green ✓ otherwise)
- per-category breakdown: strict / func / analytics / marketing
- unclassified row only shown when > 0
The WAF banner-injection block now also sets
window.SECUBOX_COOKIE_AUDIT_SUMMARY (new cookie_audit_summary_url config
key) so the banner can hit the cross-origin summary endpoint when the
host page is on a non-admin vhost.
Banner bumped to v1.4.0. New section hides itself silently when the
aggregator is disabled or empty — zero visual impact on existing
deployments.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The squash-merge of PR #141 dropped the last 4 commits from feature/136,
leaving master with a broken secubox-mail 2.2.0:
* lib/install.sh / lib/lxc.sh / lib/migrate.sh shipped to wrong path
(debian/rules expects lib/mail/, package only ships users.sh).
* mailctl cmd_install/cmd_start/cmd_stop/cmd_sync/cmd_dkim still call
/usr/sbin/mailserverctl + /usr/sbin/roundcubectl — those are now
deprecation shims that exec mailctl, producing the same fork-bomb
recursion that took down 192.168.1.200 during initial deploy.
* mail-migrate-to-single-lxc.sh in-tree fallback expected old lib path.
* helpers.bash bats fixtures source wrong lib paths.
* Phase 1 post-mortem doc + smoke-gate-11 + cmd_install fix were absent.
This replays the affected files from feature/136 tip
(85394e10) onto master. Net effect identical to a clean merge.
Required before Phase 2 (which sources lib/mail/install.sh).
Live board inspection on 2026-05-15 found that the single-mail-LXC layout
has already been hand-built under /data/lxc/mail with /data/volumes/mail
bind-mounts and 10.100.0.10/24 unprivileged-veth networking on br-lxc.
The repo source code was out of date (still referenced /srv/lxc, /srv/mail,
192.168.255.x, mail_container/webmail_container).
Spec rev. 2:
- Invariants now reflect /var/lib/lxc/mail, /data/volumes/mail, 10.100.0.10
- New invariant I12: persistent data lives on host bind-mounts only
- New invariant I13: existing 5-user secubox.in mailboxes MUST be preserved
- Postgrey dropped entirely; ClamAV deferred to Phase 2 with Rspamd
Plan rev. 2:
- Reduced from ~30 tasks to ~15 — Phase 1 is now "catch source up to board"
- Migration script becomes defensive scanner (no destructive ops)
- guard_data_path bats test enforces the data-preservation invariant
- secubox-mail bumps 2.1.0 → 2.2.0 (was incorrectly targeting 2.0.0 in rev. 1)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bite-sized TDD plan covering 8 milestones (~30 tasks) to collapse the
existing mailserver + roundcube two-LXC layout into a single 'mail' LXC.
Resolves spec open questions: Roundcube via nginx+php-fpm; HAProxy
TCP-pass-through to LXC for 25/587/993/4190. Includes data migration
helper, deprecation of 3 packages to transitional, and end-to-end
acceptance smoke.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
8-phase plan to deliver multi-domain Postfix+Dovecot+Rspamd+ClamAV+Roundcube
inside a single 'mail' LXC, integrated with secubox-users (identity) and
secubox-nextcloud (CardDAV/CalDAV). Locks 10 architectural invariants,
deprecates secubox-mail-lxc + secubox-webmail-lxc + secubox-webmail, and
records the deferred per-phase design questions.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
23-task TDD plan for extracting secubox_common from round/fb_dashboard.py
and square/Phase-3 kiosk, adding pointer input on Pi 4B/400, and shipping
both form factors as one PR. Gate: PR #134 (framebuffer numpy+center-pad)
must merge before Task 1.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Brainstorm output for converging round/ and square/ kiosks into shared
remote-ui/common/python/secubox_common/ package, adding pointer (mouse +
touchpad) input on Pi 4B/400, and fixing the round/ icon-loading bug
surfaced during Task 19 bench (load_module_icon resolves wrong dir
because module icons only live in common/, not round/).
Key decisions captured in the spec:
- OO layout classes: DashboardCanvas base in common/, RoundDashboard +
SquareDashboard subclasses overriding layout()
- Big-bang refactor: both round/ and square/ converge in the same PR
- Input layer per form factor (square/ adds pointer to its existing
touch_input.py; round/ stays touch-only)
- Auto-hide cursor sprite on square/ (visible only when pointer motion
in the last 3s)
- Ship secubox_common at /var/www/common/python/ and prepend via
systemd Environment="PYTHONPATH=..."; no separate .deb needed.
Next: user reviews spec, then writing-plans skill creates the
implementation plan.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
20 bite-sized tasks covering: common/ skeleton, JS/CSS/icons/shell
extraction, TransportManager hooks (onModuleTap, onTransportChange),
form_factor Pydantic field with TDD, round/ rewire to consume common/
via <link>/<script>, deploy.sh + image build updates, three regression
gates (pytest green, diffoscope timestamp-only deltas, manual Zero W
bench), and PR opening. Phase 2 plan to be written after Phase 1 merges.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pi 400 shares BCM2711 with Pi 4B, runs the same arm64 image. DTB
auto-selection picks bcm2711-rpi-400.dtb at boot. firstboot.sh records
the board flavour. USB peripheral mode requirement (GPIO 5V power) is
unchanged. Integrated keyboard is recognised via libinput.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two-phase design: extract remote-ui/common/ (Phase 1, refactor round/
with no behavioural change), then add remote-ui/square/ targeting Pi 4B
+ official 7" 800x480 touchscreen as a dual-pane kiosk — round UI in
Chromium at (0,0)+480x480 plus a PySide6 right column at (480,0)+320x480
with Alerts, Module detail, Console, and Mode controls tabs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Brainstormed spec for replacing the plaintext auth.toml admin login with
secubox-users-backed argon2id auth, RFC 6238 TOTP 2FA (mandatory for
admins), kill-on-disable session revocation, CLI/API parity via a single
secubox_users.engine module, and feature-flagged 3-phase rollout.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sub-project E of #49. POST /api/v1/metablogizer/webhook receives Gitea
push events for metablog-* repos, verifies HMAC-SHA256, git pulls the
site dir under a per-site asyncio.Lock, invalidates load_sites() cache
(#111), conditionally reloads nginx if site.json:domain changed.
Companion scripts/metablog-webhook-install.sh registers the webhook on
all 166 metablog-* repos via Gitea API (idempotent).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sub-project D of #49. Extends existing /metablogizer/ list view with
3 columns (version, streamlit_app, last_updated) + filter box + sort
+ 60s polling. New site.html drill-in page that surfaces every
site.json field plus 3 external links (live site, Gitea repo,
Streamlit app). Tag history defers to Gitea's UI (browser auth)
because the repos are private and browser-side proxying through
a stored token is heavier than the MVP warrants.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sub-project C of #49. JSON Schema draft-07 + Python validator/enricher +
backfill script + API extension. Permissive validation (log warnings,
don't reject). version + last_updated derived from git when site.json
lacks them. Backfill creates complete site.json for the 105 sites
without one (auto-detects streamlit_app via Gitea repo presence) and
preserves the 61 existing ones unless --force.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
28 Streamlit apps ingested into gitea.gk2.secubox.in/gandalf/streamlit-*.
First pass had 20 failures from pre-existing broken Gitea repo stubs;
bulk-deleted via API and second pass cleaned to 0 fail.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sub-project F of #49. Audit shows 30 directory-form apps in
/srv/streamlit/apps/ (23 already have .git/, 7 don't), running
inside the existing streamlit LXC at 10.100.0.50.
Design:
- Mirror all 30 apps to gandalf/streamlit-<app> on Gitea
- Retarget existing .git + push history for the 23 (preserves
any local commits); git init + push for the 7
- Tag v1.0.0 on initial state
- streamlitctl deploy <app> --from-gitea --tag <vX.Y.Z>:
clone tag into /srv/streamlit/apps/<app>/ with backup of
the current content; restart running instance if any
- streamlitctl rollback <app>: restore latest backup
- New API fields: current_tag, deployed_at (read from
.deploy.json or git describe --tags --exact-match)
YAGNI: no multi-version side-by-side, no container per
version, no webhook trigger.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(spec): Health banner live panel design (ref #92)
Three public banner sections sharing one polling/CORS pipeline:
- VisitorOrigin: nft set seen_src + GeoLite2-ASN.mmdb, threshold-gated
rollup, raw IPs discarded before persistence
- LiveHosts: HAProxy admin socket, 60 x 1-min ring buffer over req_tot
deltas, hostname-heuristic frontend filter
- CertStatus: scan /etc/letsencrypt/live + cryptography parse, classify
valid / expiring_soon / expiring_critical / expired
Each section fails independently; section hidden on enabled=false,
empty entries, or fetch error. All three endpoints are unauthenticated,
CORS-open, Cache-Control max-age=300.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(plan): Health banner live panel implementation plan (ref #92)
14-task TDD plan: tests scaffold -> config helpers -> three aggregators
(visitor-origin / live-hosts / cert-status) -> FastAPI lifespan wiring ->
nftables ruleset -> geoipupdate timer -> debian packaging -> banner v1.3.0
-> README + tracking docs -> full-suite verification + PR.
Also reconciles spec with codebase conventions: service user is 'secubox'
(not 'secubox-metrics'); config lives in /etc/secubox/secubox.conf, not a
separate metrics.toml.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(metrics): Scaffold pytest layout for new aggregators (ref #92)
Adds tests/__init__.py and conftest.py that wire packages/secubox-metrics/api
and the repo-wide common/ onto sys.path so individual aggregator modules can
be imported in isolation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(core): Add visitor_origin / live_hosts / cert_status config helpers (ref #92)
Three new section helpers in secubox_core.config that merge defaults with
operator-supplied TOML overrides. Each section defaults to enabled=false so
the live-panel aggregators stay quiet on systems that haven't opted in.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(metrics): Use tmp_path fixture + drop unused import (ref #92)
Switches the config-helper tests from a hard-coded /tmp path to pytest's
tmp_path fixture, matching the pattern in packages/secubox-haproxy/tests/.
Removes the now-unused 'from unittest.mock import patch' line.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(metrics): VisitorOrigin aggregator (ref #92)
Pure-Python aggregator that polls the nft seen_src set, resolves ASNs via
GeoLite2 mmdb, and emits a threshold-gated top-N rollup. Private/loopback IPs
are skipped at lookup time; raw IPs never leave the function scope; the
threshold gate runs before persistence so the cache file never contains
attributable counts.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(metrics): VisitorOrigin code-review followups (ref #92)
- mmdb auto-reopen on mtime change (Important): close + reopen when stat
reports a different mtime; previously _mmdb_mtime was dead state.
- _read_nft_set defensive parse (Important): guard against {elem: str} shapes
to prevent a latent TypeError.
- current() returns a defensive copy (Minor): no more by-reference leak of
internal state.
- Drop unused ip_address import and unused monkeypatch parameter (Minor).
- Tighten tiebreak test to assert ASN order, not just count order (Minor).
- Add test for mtime-based reopen.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(metrics): Pin VisitorOrigin error-path behaviour (ref #92)
Regression tests for refresh_once: disabled config, missing mmdb, and nft
subprocess failure must all yield a non-throwing degraded payload.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(metrics): LiveHosts aggregator with 60x1-min ring buffer (ref #92)
Reads HAProxy admin socket via raw AF_UNIX, parses 'show stat' CSV, filters
internal frontends (leading underscore or no dot), ring-buffers per-frontend
deltas over 60 minutes, and emits a top-N hostname rollup. Counter-reset
detection (cur < prev) yields a fresh-baseline bucket instead of a negative
delta. current() returns a defensive copy mirroring the VisitorOrigin fix.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(metrics): Pin LiveHosts CSV parser + missing-socket paths (ref #92)
Tests the show-stat CSV parser against the real HAProxy column order and
asserts that an absent admin socket returns a degraded payload rather than
raising.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(metrics): CertStatus aggregator (ref #92)
Scans /etc/letsencrypt/live for cert.pem files, classifies each by days
remaining (valid / expiring_soon / expiring_critical / expired) using the
operator's warn_days/critical_days thresholds, and emits a summary + soonest
next-renewal host. A single corrupt PEM never kills the scan. Days remaining
computed with math.ceil so a cert expiring in 2.99d reports 3d, consistent
with certbot/renewal tooling expectations.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(metrics): correct expired boundary when using math.ceil (ref #92)
math.ceil maps any cert that expired within the last 24 h to days=0,
which the previous `days < 0` guard treated as expiring_critical instead
of expired. Changing the guard to `days <= 0` closes the gap: with ceil,
days=0 means actual remaining time is in (-86400, 0] — i.e. already
past or exactly at expiry — so classifying it as expired is correct.
All four existing tests continue to pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(metrics): Wire three live-panel aggregators into FastAPI lifespan (ref #92)
Adds the three asyncio background tasks under a single lifespan and exposes
their current() payloads on /api/v1/metrics/{visitor-origin,live-hosts,cert-status}
with a 5-min Cache-Control. Endpoints stay unauthenticated by design — the
aggregators only emit threshold-gated, hostname-only data.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(metrics): Await cancelled lifespan tasks + tidy import order (ref #92)
- Important: lifespan finally now awaits gather(*tasks, return_exceptions=True)
after cancel(), so blocking subprocess/socket I/O in aggregator refreshes
doesn't race uvicorn's shutdown timeout.
- Minor: move 'from contextlib import asynccontextmanager' into the stdlib
import group at the top of the file.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(metrics): Ship nftables ingress tap for visitor-origin (ref #92)
Private inet secubox_metrics table with a timeout'd src-IP set, hooked from
prerouting at priority -300 so additions happen before secubox-firewall's
filter chain decides whether to drop the packet.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(metrics): Weekly GeoLite2 ASN refresh timer (ref #92)
Conditional on /etc/secubox/secrets/maxmind.conf existing, so the unit is a
silent no-op on installs that haven't supplied a license key. RandomizedDelay
spreads load when many boxes deploy together.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* build(metrics): Package live-panel deps, nft ruleset, and geoipupdate timer (ref #92)
- control: add python3-maxminddb, python3-cryptography, geoipupdate, nftables
- rules: install nftables/ and systemd/ assets
- postinst: secubox -> haproxy group, cache + secrets + GeoIP dirs,
nftables reload, timer enable
- service: ReadWritePaths gains /var/cache/secubox
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(metrics): Run geoipupdate as secubox + restart svc on upgrade (ref #92)
- secubox-geoipupdate.service: User/Group=secubox so .mmdb files inherit the
ownership the metrics service expects when reading them. Previously the
unit ran as root and created root-owned files that secubox-metrics could
not open.
- postinst: switch 'systemctl start' to 'systemctl restart' so the secubox
user's new haproxy-group membership is picked up by an already-running
service after an upgrade.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(banner): v1.3.0 live panel — visitor origin, live hosts, cert status (ref #92)
Three independent fetch loops on a shared 30s cadence, three DOM sections,
per-section hide on enabled=false / empty / fetch error. Uses existing
design tokens (gold/cyan/matrix-green) so no new CSS variables are added.
A failing section never affects the others.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(banner): Correct banner element id lookup in live-panel (ref #92)
The new live-panel sectionContainer() helper looked up
getElementById('sbx-health-banner'), but the actual banner element is
created with id='health-banner'. The mismatch made banner null, so the
three live-panel sections were silently never appended to the DOM.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: Session 160 — Health Banner Live Panel (ref #92)
README documents the three new endpoints + config blocks. HISTORY / WIP /
MIGRATION-MAP entries describe the feature, the spec/plan paths, and the
session's outcome.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(metrics): Address whole-branch review findings (ref #92)
- Wrap _read_nft_set / _read_haproxy_stats in asyncio.to_thread so blocking
I/O (subprocess up to 5s, AF_UNIX recv up to 2s) no longer stalls the
event loop on every refresh tick.
- Replace falsy current() guard with explicit _refreshed flag. Previously,
a successful refresh that produced entries=[] would fall through to the
on-disk cache, serving stale non-empty data during low-traffic periods.
- Move geoipupdate from Depends to Recommends. It lives in bookworm/contrib,
so a hard dependency breaks 'apt install secubox-metrics' on systems
without contrib enabled. The aggregator already degrades gracefully when
the mmdb is absent, making Recommends the correct strength. README
documents the contrib note.
- prerm stops + disables secubox-geoipupdate.timer/service so 'apt remove'
doesn't leave an orphan timer firing weekly with a missing unit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
166 sites ingested into gitea.gk2.secubox.in/gandalf/metablog-*.
First pass had 72 failures from pre-existing broken Gitea repo
stubs; bulk-deleted via API and second pass cleaned to 0 fail.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sub-project B of #49. Audit shows 166 sites in /srv/metablogizer/sites/:
83 have local .git pointing at unreachable old Gitea (192.168.255.1:3001
or git.gk2/droplet-sites), 83 are raw files.
Design:
- All 166 land at gandalf/metablog-<site> on gitea.gk2.secubox.in (single
namespace, even for the 12 ex-droplet-sites)
- Existing .git: retarget remote + push history (preserves 1 commit)
- Missing .git: git init + initial commit
- ENABLE_PUSH_CREATE_USER=true in Gitea app.ini so push creates repos
- SSH auth via gandalf's enrolled key; no API token plumbing
- Idempotent: skip if remote HEAD matches local HEAD
- v1.0.0 tag on HEAD after first push (per issue text)
- output/ingest-report.json captures per-site status
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(spec+plan): WebUI obfuscation design and implementation plan (ref #44)
Spec: docs/superpowers/specs/2026-05-12-webui-obfuscation-design.md
Plan: docs/superpowers/plans/2026-05-12-webui-obfuscation.md
The design locks the SecuBox WebUI to admin.<HOSTNAME>.<DOMAIN_SUFFIX>
via strict-regex ACLs in HAProxy AND nginx, both driven by
/etc/default/secubox as the single source of truth. New package
secubox-defaults ships the env file; secubox-haproxy API exposes
/webui/admin-domain (info), /webui/nginx-config (JWT, rendered vhost),
/webui/refresh (JWT, cache invalidate). A safe renderer script writes
the nginx vhost atomically; haproxyctl + Python /generate both inject
the strict ACL at the top of http-in and https-in frontends.
Plan has 15 tasks with TDD steps, exact file paths, and complete code
in every step (no TBD/TODO). Acceptance criteria + integration test
script included.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(secubox-defaults): scaffold package skeleton (ref #44)
* feat(secubox-defaults): ship /etc/default/secubox env file (ref #44)
* feat(secubox-defaults): postinst autodetect + dpkg-trigger (ref #44)
* feat(secubox-haproxy): webui_identity helper + tests (ref #44)
Implement webui_identity module that parses /etc/default/secubox
and exposes canonical admin URL + regex pattern for HAProxy acl.
New module:
- api/webui_identity.py: Parses SECUBOX_HOSTNAME and SECUBOX_DOMAIN_SUFFIX
with LRU cache and invalidation support.
Tests (TDD):
- tests/test_webui_identity.py: 5 test cases covering basic parsing,
missing hostname validation, custom suffix handling, comment handling,
and cache invalidation.
- tests/conftest.py: sys.path bootstrap for api/ imports.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(webui_identity): catch OSError on defaults read + test (ref #44)
* feat(secubox-haproxy): GET /webui/admin-domain endpoint (ref #44)
* feat(secubox-haproxy): GET /webui/nginx-config rendered vhost (ref #44)
* feat(secubox-haproxy): POST /webui/refresh cache invalidation (ref #44)
* feat(secubox-haproxy): secubox-render-nginx-webui safe renderer (ref #44)
* feat(haproxyctl): inject strict-regex WebUI ACL at frontend top (ref #44)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(secubox-haproxy): /generate symmetry with strict WebUI ACL (ref #44)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(secubox-haproxy): dpkg-trigger refresh on secubox-defaults-changed (ref #44)
* build(secubox-haproxy): depend on secubox-defaults (ref #44)
* test(infra): integration test for WebUI obfuscation (ref #44)
* docs: HISTORY Session 159 — WebUI obfuscation (ref #44)
Records the complete WebUI obfuscation work: new secubox-defaults
package, three /webui/* endpoints, nginx renderer script, haproxyctl
+ Python generator symmetry, dpkg trigger plumbing, integration test.
Includes notes on the brainstorming → spec → plan → subagent-driven
execution flow and the 6 fix iterations caught by reviewers.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sub-project A of #49: expose existing Gitea LXC (10.100.0.40) at
https://gitea.gk2.secubox.in/ and ssh://git@gitea.gk2.secubox.in:2222/.
Uses existing wildcard cert; backend nginx_vhosts (direct, no
mitmproxy WAF for git smart-HTTP traffic).
Records haproxyctl regression #91 as a hard constraint: edit
haproxy.cfg manually with backup, never run haproxyctl vhost add
until the generator is fixed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Display cert expiry days for current domain
- Thresholds: >7j ok, 3-7j warn, <3j error
- Backend: get_ssl_status() in health API
- Frontend: SSL line in banner after score
Reference: CM-SSL-BANNER-2026-05-12
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Brainstorming-validated spec for `scripts/agent-worktree.sh` plus a new
CLAUDE.md doctrine section. Enables parallel multi-agent work by binding
each task to one GitHub issue, one branch, and one external worktree
under `~/CyberMindStudio/secubox-deb-worktrees/`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
17 TDD tasks covering: scaffolding, render_header for 4 comment styles,
detect_existing tri-state, apply() per language with placement rules
(Python shebang+encoding, Bash shebang, HTML doctype, Markdown
frontmatter), walk() with skip-list and enrollment allowlist, CLI
dispatch, self-hosting, GitHub Actions workflow, README and CLAUDE.md
updates, smoke tests, and PR opening. Phase B/C left as operational
follow-up.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Brainstormed design for adding the SPDX-CMSD-1.0 header to every
first-party source file (~2,170 across 6 languages), backed by a
reusable Python tool (scripts/license-headers.py), a CI check, and
a phased per-package rollout. Spec covers scope, header rendering
per language, placement rules, tool architecture, CI integration
with an enrollment allowlist, and verification steps.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Brainstormed design for staging a signed APT repo at output/repo/
with amd64 + arm64 (mochabin) packages for bookworm, using existing
Go CLI and shell tooling. No network operations - user pushes
artifacts to apt.secubox.in out-of-band.
Layered build: base -> tier-lite -> tier-standard -> tier-pro
GPG: persistent ~/.gnupg/secubox/
License: CMSD-1.0 embedded in staged tree
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Hybrid approach:
- Client operations (apt setup, clone) in pure Go
- Server operations (apt init/publish/sync) wrap existing shell scripts
Features:
- secubox apt: full repo management + client setup
- secubox clone: interactive bootstrap wizard for new systems
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Define Manifest struct with kernel, partitions, boot, output
- Implement Generate() from Profile and Board
- Add ToYAML() for serialization with header comments
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
17 tasks with TDD approach:
- Go CLI with cobra/viper (gen, build, fetch, ota, info)
- Profile loader with inheritance merger
- Board configuration and tweaks
- Package scanner for debian/secubox.yaml
- Manifest and Makefile generation
- Interactive wizard with promptui
- Hardware detection
- APT repository with lintian compliance
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add /multigadget skill covering autorun, storage, round UI, tooling
- Add Eye-Remote-Multigadget.md unified wiki hub
- Add Build-System.md to wiki with build documentation
- Add AI-BUILD-PROMPT.md for GPT/Gemini assistance
- Update sidebar and home page with new links
- Update WIP.md with #70 and #71 done
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Login endpoint now captures:
- Client IP (from X-Forwarded-For, X-Real-IP, or connection)
- User-Agent header
Session events include this info for session storage.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- sidebar.js v2.36.0: Mobile responsive hamburger menu toggle
- Auto-show hamburger button on screens < 768px
- Overlay to close sidebar on tap outside
- Close sidebar on nav item click (mobile)
- Added toggleMobile/closeMobile to public API
- portal/login.html: Transparent redirect to /login.html
- Preserves query params and hash for external links
- Fallback for JavaScript disabled browsers
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Captured via capture-screenshots.py with 30s delay for cache refresh.
Includes thumbnails for wiki index.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- scripts/capture-screenshots.py: Auto-discovers 117 modules from packages/
- Playwright-based capture with JWT authentication
- 30-second delay per page for cache refresh
- Thumbnail generation (400x225)
- JSON manifest output
- scripts/generate-docs.py: Extended to 105 modules
- Multilingual descriptions (EN, FR, DE, ZH)
- --include-screenshots flag for wiki integration
- Category index pages (CATEGORIES-XX.md)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Create docs/wiki/MOCHAbin-Kernel.md with complete LED kernel guide
- Document Debian base config + SecuBox fragment approach
- Document GPIO polarity fix for IS31FL3199
- Add build instructions and troubleshooting
- Update sidebar with Kernel LED link
- Update HISTORY.md with session 115
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Big Bang migration plan from C3BOX (OpenWrt) to SecuBox-DEB (Debian):
- 4 phases: Prepare, Build, Sync, Cutover
- LXC containers for Mail, NextCloud, Matrix, Gitea
- Host-native gateway services (HAProxy, CrowdSec, nftables)
- Data migration strategy for 94 SSL certs + service data
- Cutover procedure with rollback plan
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
