Neutralisation des pixels traceurs dans les emails entrants, en deux couches :
réécriture à la réception (data: URI 1x1 inline, zéro requête émise) avec
archivage du brut DKIM-intact, plus le filet MITM existant à l'ouverture.
Décisions actées : heuristique conservatrice (invisible OU domaine/chemin
connu), aucune réconciliation des compteurs entre couches, intégrité du
courrier non négociable (toute exception => remise de l'original intact).
Deux trous comblés en auto-revue : emplacement/déploiement du code (LXC mail,
pas l'hôte) et synchronisation hôte->LXC de learned-trackers.txt, sans laquelle
le partage de la liste entre couches était illusoire.
Signale au passage une anomalie hors périmètre : rspamd tourne mais n'est
câblé dans aucun milter Postfix (smtpd_milters vide) => ne filtre rien.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Correctif de fond du #ads affamé : rien n'empêchait autolearn de promouvoir les
régies pub en splice. Or splice = passthrough, et le feed de candidats (#662) ne
se déclenche que sur le chemin allow/mitm ("never the block 204 / splice
paths") — une régie splicée est donc structurellement inobservable, et la boucle
d'apprentissage s'étiole en silence.
Le piège est que les endpoints pub correspondent PARFAITEMENT à l'heuristique de
splice (hits >= SPLICE_MIN_HITS ET html_hits == 0 : ils ne servent jamais de
HTML et sont très sollicités). C'est ainsi que adform/rubicon/google-analytics
et consorts ont atterri dans splice-learned.txt et gelé l'apprentissage 9 jours.
_splice_feed() filtre désormais via _never_splice(), calqué sur le NEVER_LEARN
de #685 : graine des principaux réseaux pub/analytics, vérification sur l'hôte
ET son registrable, extensible par SECUBOX_NEVER_SPLICE sans changement de code.
2 tests (échouent sans la garde) ; 19 tests autolearn/ads passent.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
L'onglet #ads affichait "aucun tracker/pub détecté dans la fenêtre" alors que
67 hôtes réels (permutive, amplitude, byteintlapi, digiteka…) attendaient leur
promotion en base. Cause : ad_candidates est un BACKLOG (upsert host+site,
promu à AD_MIN_SITES sites distincts), pas une série temporelle — le fenêtrer
à `hours` masque tout dès que l'alimentation se tarit, rendant une boucle
d'apprentissage à l'arrêt invisible au lieu d'évidente.
ad_stats() bascule sur le cumul quand la fenêtre est vide et le signale via
candidates_cumulative ; le panneau l'étiquette explicitement ("cumul — rien de
neuf dans la fenêtre") pour ne pas faire passer un backlog ancien pour de la
détection récente.
Deux tests couvrent le repli et la non-régression du cas fenêtré (le premier
échoue en KeyError sans le correctif).
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
HISTORY: entrées datées pour secubox-meshtastic (#897/PR #898/v2.15.0), les
correctifs de l'image rpi400 (deps auth, --kiosk CI, postinst mediaflow
#DEBHELPER#, prompts conffile) et la page wiki Support.
WIP: les deux chantiers passent en ✅ avec leurs follow-ups ; le blocage réseau
du rpi400 est documenté avec les deux pistes de reprise.
TODO: nouvelle section « Afficheurs & appliances » pour les trois sujets mis en
pause (réseau rpi400, reflash round eye, console double écran Pepper's ghost),
avec pour chacun le point de reprise exact.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Corrige la cause racine du login cassé sur l'image rpi400 (deps argon2/pyotp/
qrcode jamais tirées à cause de dpkg -i --force-depends), câble --kiosk dans
la CI, et corrige deux bugs latents révélés par l'apt-get strict du kiosk :
le token #DEBHELPER# dans un commentaire du postinst mediaflow (cassait
chaque install) et les prompts conffile non auto-répondus.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Two failures surfaced by the strict apt-get pass that --kiosk runs in the
rpi chroot (the lenient `dpkg -i --force-depends || true` had been swallowing
the first one on every build):
1. secubox-mediaflow postinst had `#DEBHELPER#` inside a comment. dh_installsystemd
substitutes that token textually wherever it appears, so it expanded the
systemd block mid-comment and orphaned "; kick one refresh" onto its own
line → `syntax error near ';'` → configure fails on EVERY install. Reword
the comment (real token stays alone on line 22). Bump to ~bookworm3.
2. The kiosk apt-get relied on DEBIAN_FRONTEND=noninteractive, which governs
debconf, not dpkg's conffile prompt — a pre-existing /etc/secubox/mesh.toml
triggered an interactive prompt that EOFs on the closed chroot stdin and
aborted the build. Pass --force-confdef/--force-confold (as the main
dpkg --configure pass already does) to auto-keep existing conffiles.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
secubox-auth/users crash at import on the rpi400 image because it is built
with `dpkg -i --force-depends` + debootstrap --include, so their declared
Depends (python3-argon2, python3-pyotp, python3-qrcode) are never pulled —
unlike normal apt installs on mochabin. The engine does `from argon2 import
...` / `import pyotp` / `import qrcode`, so the daemon dies at startup →
nginx 502 → no login. Add pyotp+qrcode (pure-Python) to INCLUDE_PKGS and
argon2 via the QEMU-proven pip step (mirroring the cryptography exclusion).
Also expose the kiosk GUI through CI: a `kiosk` workflow_dispatch boolean
→ build-image.sh --kiosk → forwarded to build-rpi-usb.sh, so a rebuilt
rpi400 image can ship the chromium fullscreen dashboard.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
uvicorn/uvloop refuses to rebind an existing UDS ('Address already in use') →
crash-loop on restart. ExecStartPre=+/bin/rm -f the socket (root, clears any
owner in the sticky /run/secubox) before ExecStart.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Deploy found a crash-loop: the shipped default config has [shared_grid] set, but
the opt-in mosquitto is not running, so Bridge.start()/the paho adapter's
synchronous connect() raised ConnectionRefusedError and killed the daemon.
Bridge.start() now skips an unreachable broker (log + continue); the paho adapter
uses connect_async + loop_start (non-blocking, auto-reconnect). +regression test.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Reconcile the three pieces the webui→ctl delegation needs to actually work:
NoNewPrivileges=yes on the daemon unit neutralized sudo, and sudoers.d
granted to the wrong user (secubox instead of secubox-meshtastic, the
daemon's actual User=). Verified _ctl_cb's systemd-run argv matches the
sudoers command lines for set-mode/set-grid token-for-token.
Also demote mosquitto from Depends to Recommends (module never installs/
runs it; README already documents opt-in setup) and add sudo to Depends
for the ctl delegation. Minor cleanups: unused `field` import in model.py,
dead gridpolicy re-export in daemon.py, and a malformed broker port
(e.g. host:abc) no longer crashes Bridge.start() at daemon startup.
Deferred: append a note to the multigrid design spec about the
gridpolicy/bridge default broker-port mismatch (8883 vs 1883) and the
nft hostname-resolved-once caveat for future on-grid containment work.
Finalizes the secubox-meshtastic package for install: debian/install
(api -> usr/lib/secubox/meshtastic/api, sbin -> usr/sbin, www, nginx
route, menu entry, config example, mosquitto broker reference conf),
debian/postinst (secubox-meshtastic system user in dialout, restricted
leaf dirs under 0755 shared parents per #511/#474, config seed-if-absent,
enable+start the daemon which runs fine with radio: absent), debian/prerm,
debian/rules (0440 sudoers drop-in + dh_installsystemd --name=), and
debian/control deps (fastapi/uvicorn/pydantic/paho-mqtt/mosquitto,
secubox-core). Adds README documenting the API, the three grids, passive
mode, and the meshtastic pip (non-Debian) dependency note.
Clean dpkg-buildpackage build verified (secubox-meshtastic_0.1.0-1~bookworm1_all.deb),
60/60 tests pass, SPDX headers clean.
- systemd/secubox-meshtasticd.service (+ debian/ copy): dedicated
secubox-meshtastic user/group, dialout for serial, ProtectSystem=strict
hardening, DeviceAllow for ttyUSB/ttyACM. No RuntimeDirectory=secubox
(ref #896 landmine) — daemon binds its socket directly into the shared
1777 /run/secubox.
- nginx/meshtastic.conf: static panel + dedicated-socket API proxy,
mirrors secubox-profiles' prefix-preserving proxy_pass convention.
- menu.d/71-meshtastic.json: panel entry under the mesh category.
- conf/mosquitto-secubox-meshtastic.conf: private shared_grid broker
listener bound to the MirrorNet IP (10.10.0.1), anonymous access
disabled, creds/ACL left to the operator.
Add www/meshtastic/index.html — cyan hybrid-dark panel (Courier Prime w/
monospace fallback, no external CDN/fonts/tiles) with 5 tabs: Nodes (list +
offline canvas map plotting positioned nodes, ringed/colored by SNR), Messages
(per-channel view + send via POST /send), Channels (per-channel grid
off/shared/on toggle via POST /grid), Sniffer (census table + channel-activity
bars from /packets), and Grid (radio present/absent banner + mode selector via
POST /mode). Reads localStorage.sbx_token, shows a persistent auth-required
banner on 401, and a radio-absent banner throughout.
create_app(cache, send_cb, ctl_cb) exposes GET status/nodes/messages/packets
over the injected StateCache, and POST send/mode/grid — mode and grid are
validated against config.MODES/GRIDS (422) BEFORE delegating to ctl_cb,
mirroring secubox-profiles' validate-before-delegate pattern. All routes
require the real secubox_core.auth.require_jwt.
Root-only webui->ctl CLI for /etc/secubox/meshtastic.toml, mirroring
secubox-profiles' cli.py/actuate_paths.py conventions: set-mode/set-region/
set-role/set-grid/set-psk edit the config via a small TOML dump/validate/
atomic-swap (double-buffer/4R), apply-egress renders gridpolicy's nft rules
into an nftables drop-in (empty rule list keeps DEFAULT DROP). Each verb
requires root and appends a JSON audit line. Scoped systemd-run-wrapped
sudoers grants let the panel (User=secubox) invoke each verb without a
broader sudo surface.
Carried fix from Task 4 review: the single lambda used for all three pubsub
topics forwarded `packet` for every event, but meshtastic publishes `node=`
on meshtastic.node.updated (no `packet` kwarg) and neither on
meshtastic.connection.established. Deliver the correct kwarg per topic
(packet/node/interface). Untested (no real device), no new test required
per task brief.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Implement Task 8: Engine class wiring radio → state → cache/passive/bridge
on packet receive, plus main() building real objects (config.load, open_serial,
paho-mqtt-backed Bridge factory imported lazily, PassiveCapture, StateCache).
Radio-absent and api.web-absent (Task 10 not yet done) paths degrade gracefully
instead of crashing. sbin/secubox-meshtasticd launches `python3 -m api.daemon`.
Tests drive Engine with MockRadio + FakeMqtt, no real device/broker. 33/33 tests
pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Implement Bridge class for serial↔MQTT bridging with grid-policy-driven
publishing. Tests verify correct broker connection and topic filtering per
channel grid configuration. All 30 tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Implement Task 6: PassiveCapture with packet recording (JSON lines),
node census tracking, and per-channel statistics. Payload withheld
when decrypted=False; included when decrypted=True.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
- Implement targets_for(channel, cfg) to return grid membership subset
- Implement nft_egress_rules(cfg) to generate allow-rules for enabled on-grid brokers
- TDD: 5 test cases all passing (offgrid, shared+on, on-disabled, empty rules, broker rules)
- Full test suite: 23 passed
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Implements Task 4: RadioInterface protocol with MockRadio test double and
SerialRadio wrapper. Lazy meshtastic import in open_serial() ensures test
suite never requires the library. Returns None when device absent (radio: absent
path). All tests pass; full suite clean (18/18).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Use copy.deepcopy in StateCache.get() and update() to prevent nested-mutable sharing
- Move _write_atomic() call inside lock in update() for atomicity
- Add copyright line to tests/test_cache.py header
- Add test_get_returns_deep_copy_not_live_reference() to verify isolation
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
TDD implementation: Packet dataclass with parse_packet() parser,
Node dataclass for mesh participants, MeshState with apply_packet()
and apply_nodeinfo() to build census and channel message logs.
Parser consumes meshtastic pubsub dict format.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Task 1 complete:
- Package scaffold with proper directory structure
- Config loader using tomllib with dataclass-based interface
- Comprehensive test coverage (4 tests, all passing)
- Example TOML configuration with sensible defaults
- Debian packaging (control, compat=13, changelog, rules)
- Full SPDX header compliance
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
sync-all.yml's release job does `uses: ./.github/workflows/release.yml`, but
release.yml only declared push/workflow_dispatch triggers — a reusable-workflow
reference to a workflow without `on: workflow_call:` makes the CALLER fail at
startup ('workflow file issue'), which is why every sync-all run was red at 0s.
Add a bare workflow_call trigger (tag-push + dispatch paths unchanged; the
release job is tags-guarded and sync-all runs on branch pushes, so the callable
path is not exercised — this only restores a valid reference).
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
The board ran a hand-created unhardened ROOT sbxwaf on :8085 while the package
shipped only a worker@ fan-out that crash-looped (panic on the root-only
cookie-audit log) and, as secubox-waf + RuntimeDirectory=secubox, re-chowned the
shared /run/secubox on every crash-restart — breaking every secubox-user socket
bind (profiles 502s). Ship ONE hardened unit (secubox-waf-ng.service, :8085,
User=secubox-waf, full sandbox, NO RuntimeDirectory — sbxwaf only connects to
waker.sock). postinst asserts the perms a non-root sbxwaf needs
(haproxy-routes.json 0644, cookie-audit ledger writable by secubox-waf),
disables leftover worker@ units, drops the /etc override. Validated live on gk2:
264 routes, real routing, admin/gitea/billets/yacy 200; a WAF restart no longer
touches /run/secubox and profiles stays up.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
The postinst enabled+started the sleeper, which stops idle sleepable modules —
and health-sync currently lists crowdsec among sleepable. Ship it disabled
(--no-enable --no-start; postinst try-restart only, never enable). Operator
opts in after auditing the sleepable set. Waker stays enabled (never stops).
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Sleep-when-idle + wake-on-access for public on-demand services, per-module
lifecycle policy, durable route restoration, awake-level panel setter, and a
two-phase terminal wake splash (waker + nginx error_page) generalized to all
on-demand vhosts. Includes #893 profiles-actuation robustness.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Repurpose the (dead, Task-7) nginx-sync into a transactional injector that wires
the phase-2 splash into EVERY on-demand vhost, not just yacy:
- nginx/secubox-waking.conf is now fully self-contained at server level
(proxy_intercept_errors + proxy_connect_timeout 3s + error_page → splash), so
per-vhost wiring collapses to a single `include snippets/secubox-waking.conf;`.
- nginxgen.py rewritten: find_config (by server_name), wire/unwire (idempotent,
marker-based, insert after the domain's server_name line — right block in
multi-server files), and sync_and_reload (wire every on-demand vhost via
wafsync.ondemand_vhosts, nginx -t, rollback from in-memory originals on
failure, reload on success — NO .bak in the nginx dir).
- `secubox-wakectl nginx-sync` (root) runs it; `set-lifecycle` runs it after the
waf/health resync (best-effort — opting a service on-demand auto-wires its
splash); postinst runs it so every install/upgrade re-asserts the wiring.
Live on gk2: one nginx-sync wired 5 on-demand vhosts (yacy/gitea/nextcloud/
podcaster/billets), reported streamlit as having no nginx config, idempotent on
re-run; yacy down → nginx serves the 503 splash in 3.0s. 286 tests pass.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Two-phase graceful wake UX, one shared splash page:
- templates/waking.html reworked into a pseudo-terminal "virtual screen": a CRT
panel with scanlines, an auto-executing boot log, blinking cursor, elapsed
counter, and a "taking longer / repair" panel after ~90s. Fully self-contained,
JS-driven (service name from the vhost hostname, elapsed from sessionStorage),
auto-refreshing — so it needs no server-side substitution and works verbatim
for BOTH phases.
- waker (phase 1, container down / route absent): serves the page verbatim.
- nginx (phase 2, container up but app still booting → backend 502/503/504):
new nginx/secubox-waking.conf snippet intercepts the gateway error and serves
the same splash (503 + Retry-After) instead of a raw 502. Wire per on-demand
vhost with `include snippets/secubox-waking.conf;` + `proxy_intercept_errors on;`
+ `proxy_connect_timeout 3s;` (fail fast so the splash beats HAProxy/sbxwaf).
- debian/install ships the page to /usr/share/secubox/www/waking/ and the snippet
to /etc/nginx/snippets/.
Proven live on yacy: full path (HAProxy→sbxwaf→nginx) serves the terminal splash
during the down/boot window, then the real app once it answers. 279 tests pass.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Review follow-ups for the awake-level setter:
- reject a protected module server-side (web 409) and in the ctl (refused,
reason=protected) — it is forced always-on anyway, so the write is spurious
and would dirty a core manifest; mirrors set_pin's protected refusal.
- fsync manifest_edit's atomic write before rename (the manifest is source of
truth, not a regenerable derived file) — parity with web._atomic_write.
- tests: protected refusal (web + cli), duplicate-same-key collapse, manifest
with no trailing newline.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
The scale-to-zero policy (lifecycle always-on/eager/on-demand/manual +
wake_class normal/urgent) was manifest-only. Add a per-module selector to the
/profiles/ panel that writes the manifest and resyncs the derived files, via the
webui->ctl pattern:
- api/manifest_edit.py: targeted line-edit of a manifest's lifecycle/wake_class,
preserving everything else (comments, inline portal table, …); refuses a
sectioned manifest; atomic write.
- api/cli.py: `secubox-profilectl set-lifecycle <mod> --lifecycle X --wake-class Y`
(root) — edits the manifest then re-runs waf-sync + health-sync so the change
takes effect; resync paths derive from --root (test-isolated). argparse choices=
hard-gate the enums; refuses an unknown module.
- api/web.py: POST /api/v1/profiles/lifecycle — validates enum (422) + known
module (404) BEFORE any sudo, then delegates to the root ctl via systemd-run.
- sudoers.d: one scoped grant (three bounded wildcards, panel pre-validates +
ctl re-validates — execve, no shell).
- www/profiles/index.html: two compact <select>s per module (protected → 🔒),
POSTing /lifecycle and refreshing.
16 new tests (manifest_edit, cli set-lifecycle, web route); 275 pass, no
live-state leakage, sudoers + JS validated.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
