- Wrap graph content (links + nodes) in a content <g> so d3.zoom's
transform applies there without moving the SVG itself.
- d3.zoom(): scale 0.2–6×, mouse wheel + touch pinch + drag-pan.
Filter lets node drags pass through d3.drag (no collision with the
pan gesture) and always accepts multi-touch + wheel.
- autoFit(): after the simulation alpha drops below 0.1, compute the
nodes' bounding box and transition to a zoom transform that fits
all of them with 28 px padding (capped at 2.5× so a single-node
graph doesn't blow up). Same logic re-runs on dblclick (reset)
and on viewport resize (debounced 150 ms).
- Viewport-resize observer keeps the viewBox and zoom in sync when
orientation changes.
User feedback : graph too small + 'Chargement…' / 'ZÉRO TRACKER DÉTECTÉ'
overlays kept stacking on top of the graph.
Layout reworked :
- body flex column 100vh overflow hidden — header sticky, main owns
all remaining vertical space, graph fills it.
- stats tiles → small glassy pill floating top-left over the graph.
- cards collapsed into a compact 3-cell horizontal strip at the
bottom ; open-card popover slides up over the graph so the graph
stays visible.
- graph svg measured at render-time, viewBox set dynamically — d3
force-center scales to actual viewport instead of the fixed
600×600 box.
Persistent overlays removed :
- .social-empty and .social-loading divs gone from template.
- JS no longer references them. Empty graph is its own indicator
(stats tiles show 0/0). Network errors log to console only.
Net effect : tap 'Ma carto' on splash → full-screen force-directed
graph, stats top-left, cards bottom strip, no overlay clutter.
JSON in a single-quoted HTML data-* attribute breaks on FR apostrophes
(l'effacement, L'analyse) — JSON.parse threw 'unterminated string at
line 1 column 475'. <script>window.__SOCIAL_I18N__ = {…}</script>
keeps the JSON verbatim and is the standard idiom for shipping
server-rendered config to client JS.
Root cause caught after Phase 11.B live deploy : the mitm-wg workers
run inside /opt/mitmproxy-toolbox/bin/python3 — a venv whose
site-packages does NOT include /usr/lib/secubox/toolbox/. Every addon
that does `from secubox_toolbox import ...` was therefore silently
degraded :
- inject_banner.py: dpi_class / geo / store / utiq imports all
failed → no host classification, no GeoIP, no ASN, no Utiq tile.
- social_graph.py: _social = None → record_edge() no-op → zero
social_edges rows even though the response hook fired correctly.
Adding `export PYTHONPATH=/usr/lib/secubox/toolbox` to the launcher
restores ALL helper imports for ALL addons.
Verified live on gk2 : after deploy + worker restart, 369 social_edges
recorded in 30s, fold produced cross-site links connecting 4 ad-tech
relay publishers (360yield, seedtag, smartadserver, smilewanted via
the same 35.214.136.108 endpoint).
iPhone hitting /social/me from the kbin splash on R3 tunnel was 400'ing
because the previous chain was only ?mh=<hash> + ARP _resolve(), and R3
peers are on 10.99.1.0/24 not the captive subnet.
New 3-step resolution chain (matches /report/me/html):
1. ?mh=<hash> — explicit hex hash in URL
2. X-R3-Peer header — sentinel set by mitm-wg's inject_xff
for transparent R3 flows ; we look up
the WG pubkey hash from
/var/lib/secubox/toolbox/wg-peers.json
keyed by peer IP.
3. ARP _resolve() — R2 captive subnet fallback
Verified live: GET /social/me with X-R3-Peer: 10.99.1.60 → 303 →
/social/<token-for-mac_hash 9433ceb90895a075>
Five fixes caught during the first live deploy on gk2 :
1. mitmproxy_addons/social_graph.py: `from . import local_store` was
never resolving because mitmproxy loads addons as top-level
modules (not as package members). Inlined the R3 path
(10.99.1.0/24 → sha256(wg_pubkey)[:16]) directly. Verified
against the real iPhone IP (10.99.1.60 → 9433ceb90895a075).
2. www/toolbox/social.js: the JSON fetch + wipe paths used the
nginx-prefixed `/api/v1/toolbox/social/...`. The kbin vhost
routes via HAProxy straight to uvicorn (bypassing nginx) so
that prefix never matches. Rewrote to the FastAPI router
paths `/social/graph/{token}` + `/social/wipe/{token}`.
3. secubox_toolbox/app.py: FastAPI StaticFiles mount on /toolbox
for the kbin HAProxy path (which doesn't go through nginx).
The mount serves the same files nginx would alias from
/usr/share/secubox/www/toolbox/.
4. debian/postinst: chmod 0755 /usr/share/secubox/www so the
`secubox-toolbox` service user (not in secubox group) can
traverse to the WebUI directory. Without this the StaticFiles
mount crashed the service at startup with PermissionError.
5. New /social/me endpoint + splash menu icon : self-resolving
entry point that mints a 1 h HMAC token and 303-redirects to
/social/{token}. Mirrors the existing /report/me/html pattern.
Splash page gets a 🕸️ "Ma carto" quick-nav icon.
End-to-end via gk2 live :
/social/me?mh=<hash> → 303 → /social/{token} → 200 (4883 bytes FR)
static assets : 200 / 7529 / 8359 / 279706 bytes
addon smoke test : mac_hash for 10.99.1.60 → 9433ceb90895a075 ✓
Parent #502 (Phase 11 social mapping per device). Builds on top of
the Phase 11.A backend (PR #506, branch feature/505).
- secubox_toolbox/api.py: GET /social/{token} renders the HTML view
(HMAC-token gated, FR/EN i18n via ?lang= or Accept-Language).
GET /social/favicon/{domain} server-side cached proxy (7d TTL,
strict charset, HTTPS only, hard 5s timeout, transparent gif
fallback).
- conf/social_view.html.j2: Jinja2 template implementing the
locked design from the #502 round-2 lock — Cinzel header / IM
Fell body / JetBrains Mono data, force-directed d3 graph, three
collapsible cards, tracker-node bottom-sheet, wipe dialog with
3 s countdown.
- conf/i18n/social.{fr,en}.json: single dictionary per language.
FR is the source-of-truth.
- www/toolbox/social.css: palette-precise stylesheet. Mobile-first;
desktop bumps the node-detail panel to a right-rail at >=720 px.
- www/toolbox/social.js: self-contained d3 driver. Consumes the
Phase A JSON contract, force layout (link distance 70, charge
-180), tap-to-focus pulse on tracker nodes, wipe modal countdown
timer, refresh after wipe.
- www/toolbox/d3.v7.min.js: d3 v7.9.0 self-hosted (~280 KB),
ISC-licensed, no CDN runtime dependency.
- debian/changelog: bump to 2.6.1-1~bookworm1.
Smoke tests: FR + EN templates render (4771 / 4503 bytes), api.py
parses, social.js node-checks clean.
Phase C scope deferred — legal-evidence flag computation, bilingual
PDF report, GeoIP+ASN populated values on the node detail panel,
operator dashboard /admin/social/ HTML view (the JSON
/admin/social-aggregate endpoint already ships from 2.6.0).
Caught during 2.6.0 live deploy on gk2: `debian/rules` had install lines
for `sbin/secubox-toolbox-db-tune` (override_dh_strip) and a few wg-
phase scripts but NEVER for `sbin/secubox-toolbox-mitm-wg-launch`. The
existing launcher on gk2 was a leftover from an older deploy and dpkg
upgrades silently kept the file static — which masked the Phase 11.A
addon-chain edit (social_graph was added in source but never reached
the live launcher).
- debian/rules: explicit install lines for the three sbin helpers
(lxc-provision, mitm-wg-launch, wg-provision). These were either
ad-hoc-installed before or relied on the operator copying them
by hand.
- sbin/secubox-toolbox-mitm-wg-launch: chmod 0755 (was 0664).
- sbin/secubox-toolbox-wg-restore: chmod 0755 (was 0664).
Closes a source-first regression caught by [feedback_source_first_always].
Parent #502 (Phase 11 social mapping per device). Backend-only deliverable
unblocking Phase 11.B (d3 view) and 11.C (legal-evidence PDF).
- secubox_toolbox/social.py: new module. SQLite schema for
social_edges (raw, 7d retention) + social_nodes (per-peer per-tracker
aggregate) + social_links (per-peer per-site-pair with shared-tracker
list and JA4 collision flag). ThreadPoolExecutor fire-and-forget
write path matches utiq.py. fold_recent() runs every 5 min and
derives the aggregate tables. wipe_mac() backs the RGPD art. 17
endpoint. aggregate() backs the operator dashboard.
- mitmproxy_addons/social_graph.py: new addon. Loaded after
local_store and before inject_banner in the mitm-wg chain.
Parses Set-Cookie + Cookie headers, normalizes via
cookie_id_hash = sha256(domain||name||value)[:16] — raw cookie
values are NEVER persisted. Deny-list strips
session/CSRF/auth/locale cookie names. Cheap eTLD+1
approximation for 1st- vs 3rd-party classification. Reuses
local_store helpers for the rotating MAC hash.
- secubox_toolbox/api.py: three new endpoints.
GET /social/graph/{token} per-client graph (HMAC token)
POST /social/wipe/{token} RGPD art. 17 droit à l'effacement
GET /admin/social-aggregate operator dashboard KPI view
- secubox_toolbox/app.py: two new startup background tasks —
social_fold_loop (5 min) and social_purge_loop (1 h). Match the
existing purge/threat_intel pattern.
- sbin/secubox-toolbox-mitm-wg-launch: addon chain wired
(… local_store → social_graph → inject_banner …).
Phase A scope locks: R3 mitm-wg path only. R2 captive hookup, d3
graph view, bilingual FR/EN PDF report, and the consent-state /
extra-EU legal-evidence flags are deferred per the #502 design lock
(rounds 1 + 2).
Smoke-tested on the host : hash stability, deny-list, edge insertion,
fold, fetch_graph contract shape, aggregate, wipe_mac round-trip,
addon helpers (eTLD+1, Set-Cookie parser, Cookie parser, Domain attr).
All seven invariants green.
- debian/changelog: bump to 2.6.0-1~bookworm1.
Shared-directory ownership race: secubox-toolbox + secubox-mesh created
/var/log/secubox at 0750 with their own module owner, secubox-admin at
750 root:secubox, while ~10 other packages use the correct 0755
secubox:secubox. Whichever postinst ran last won — and when toolbox or
mesh won, the aggregator (user secubox) lost traversal and the /waf/ +
/soc/ dashboards went blank (regressed on gk2 2026-06-10).
- secubox-toolbox/debian/postinst:47 0750 -> 0755 (+ guard comment)
- secubox-mesh/debian/postinst:23 0750 -> 0755
- secubox-admin/debian/postinst:11 750 -> 0755
Mode 0755 makes the shared parent world-traversable regardless of which
package's owner wins. Per-module log files + subdirs inside keep their
own restricted perms. Same class as the /etc/secubox traversal
constraint + the /usr/share/secubox/www chmod (#507).
User reported /waf/ + /soc/ dashboards showing empty threats and tracked
attackers cards. Live triage on gk2 found _get_threat_stats() iterating
all 332k JSONL entries in the 110 MB waf-threats.log on every request,
pinning the aggregator at 89% CPU with 8+ concurrent open handles.
- api/main.py: _load_stats_disk_cache + _save_stats_disk_cache helpers
persist counters + last-read byte position to
/var/lib/secubox/waf/stats-disk-cache.json (atomic .tmp -> rename so
a crash mid-write never corrupts).
- api/main.py: _get_threat_stats reseats the file at the saved byte
position, reads only the new tail since then, persists the new
position before returning.
- api/main.py: log rotation / truncation detected via size shrink;
counters reset cleanly.
- api/main.py: day rollover resets threats_today only; cumulative
counters survive.
- api/main.py: ip_countries dict capped at 1200 entries (most-active
1000 retained on overflow) so the cache file stays small.
Live on gk2 after deploy: /waf/stats steady-state 30-37 ms (vs 30s+
timeout before). /waf/alerts, /waf/bans/history, /soc/* all 200 in
under 200 ms. Aggregator CPU drops from 89% to 52% under same load.
- debian/changelog: 1.2.1 -> 1.2.2-1~bookworm1.
The two boards fail in the cross-arm64 chroot stage of build-image.yml
and (even with fail-fast: false) block the downstream release.yml job
from publishing the OTHER boards' images. Releases v2.13.9 through
v2.13.12 all hit this trap.
This change keeps both entries available via workflow_dispatch (operators
can still build them on-demand), but removes them from the push-tag
scheduled matrix so mochabin / vm-x64 / rpi400 actually ship on tag.
Board support files (board/espressobin-*/, image/build-image.sh
--board espressobin-*) stay in tree.
- build-image.yml matrix excludes espressobin-v7 + espressobin-ultra
on push and workflow_call.
- build-image.yml workflow_dispatch choice list keeps both entries
flagged as on-demand only.
- Release notes template drops the two image rows, adds a note
explaining the on-demand path.
- Install instruction adjusted (MOCHAbin + Raspberry Pi 400).
Two silent regressions surfaced during the 2.5.0 → 2.5.1 deploy on gk2:
1. kbin.gk2.secubox.in 503 for 5 min after upgrade. dpkg SIGTERMed
secubox-toolbox.service and never restarted it because
dh_installsystemd --no-start --no-enable is set in debian/rules.
2. iPhone tunnel went from ~10 s/page to unusable. postinst overwrote
the operator-deployed fanout nft drop-in
(/etc/nftables.d/secubox-toolbox-wg.nft) with the pristine
single-port-DNAT version, pinning all R3 traffic to worker@1
(97 % CPU while w2-w4 sat idle).
Fixes — postinst only, no code change:
- debian/postinst: ALWAYS install the Phase 9 fanout drop-in as
/etc/nftables.d/zz-secubox-toolbox-wg-fanout.nft. zz- prefix sorts
after the base file in /etc/nftables.conf's alphabetical glob
include, so the base file creates the table + chains and the zz
drop-in then flushes the prerouting chain and repopulates it with
the numgen mod 4 map across ports 8081..8084.
- debian/postinst: on UPGRADE ($2 set), try-restart the long-running
units (secubox-toolbox, secubox-toolbox-mitm, and the 4 worker
instances). try-restart is a no-op when the unit isn't running,
so fresh installs / unconfigured boards are unaffected.
- debian/changelog: bump to 2.5.2-1~bookworm1.
User reported the banner only appeared at end-of-load and pages were
very slow. Three host-stable lookups (classify_host, whitelist match,
GeoIP) ran on every flow without caching, and a full-body regex scan
ran just to count "trackers in body" for a tile most users don't read.
- inject_banner.py: new _host_signals(host) returning a 7-tuple,
cached @functools.lru_cache(maxsize=2048). Re-hit cost goes
from 5-50 ms to a dict lookup.
- inject_banner.py: drop _count_trackers_in_body() call from the
response hot path. is_tracker_host flag (cheap host regex)
keeps the privacy signal.
- inject_banner.py: _MAX_INJECT_BYTES = 2 MB cap on response()
via Content-Length pre-flight + defensive len() guard.
- inject_banner.py: drop the now-dead "🎯 N body trackers" tile
branch ; the cookie + ⚠ tracker-host tiles stay.
Net: banner shows at first paint, TTFB on big HTML responses no
longer banner-bound.
- debian/changelog: bump to 2.5.1-1~bookworm1.
The single-process mitm-wg saturated one ARM core at ~90 % under just
2-3 active wg peers because the Python GIL caps real parallelism
inside one mitmproxy process. No single-process tweak moves that
needle further than Phase 8.1 already did (CPU 65 % → 12 % at idle ;
at multi-peer load we're back to 90 %+).
Phase 9 ships 4 mitm worker instances and lets nft round-robin
distribute new TCP flows across them via `numgen inc mod 4`.
Conntrack pins each flow's DNAT translation for its lifetime, so a
given TCP connection sees exactly one worker from SYN to FIN —
sticky-per-flow without needing nftables 1.0.7+ jhash support
(Debian bookworm ships 1.0.6).
What's shipped :
systemd/secubox-toolbox-mitm-wg-worker@.service
Template unit ; per-instance Environment=MITM_WG_LISTEN_PORT=
808%i (8081..8084). Per-worker RuntimeMaxSec=3h, MemoryMax=128M,
TasksMax=128, User=secubox-toolbox.
sbin/secubox-toolbox-mitm-wg-launch
Now reads MITM_WG_LISTEN_PORT (default 8081 for the legacy
single-worker service).
nftables.d/secubox-toolbox-wg-fanout.nft
Replaces the single-port DNAT rules with a numgen-inc round-robin
map to 4 ports. DNS + captive-portal DNAT rules stay untouched
(small queries, no benefit from fanout).
debian/rules
Installs both the worker template and the fanout nft drop-in
next to the existing single-worker artifacts.
Activation (operator-initiated) :
systemctl disable --now secubox-toolbox-mitm-wg.service
systemctl enable --now secubox-toolbox-mitm-wg-worker@{1,2,3,4}.service
ln -sf /usr/share/secubox/toolbox/nftables.d/secubox-toolbox-wg-fanout.nft \
/etc/nftables.d/secubox-toolbox-wg.nft
nft -f /etc/nftables.d/secubox-toolbox-wg.nft
Rollback : reverse the steps above ; the legacy single-worker
service and its single-port nft drop-in remain shipped + functional.
Live numbers on gk2 with the workload active during the cut-over
(2 Linux peers + 1 iPhone, dozens of concurrent flows) :
before one process @ 90-95 % CPU on a single core, saturated
after ~55 % avg per worker × 4, 0-70 % range, all cores below saturation
The Phase 8.B addon-write fire-and-forget pattern shipped in 2.4.3
becomes load-bearing here : SQLite WAL on toolbox.db handles 4
concurrent writers cleanly because each worker's _executor serialises
its own writes, and SQLite's writer mutex handles the inter-worker
contention with no event-loop stalls.
Known limitation : the cert-pin auto-learning dynamic bypass file is
the remaining race surface (4 writers can dupe a line under burst,
the launcher's sort -u de-dupes at next reload). A real filelock
lands in Phase 9.1.
A — Captive mitm-toolbox (secubox-toolbox-mitm.service) gets the same
three flags as mitm-wg picked up in 2.4.1 :
--set http2=true
--set connection_strategy=eager
--set keep_host_header=true
The captive idles at ~0 % CPU right now (wlan AP is down), so no
visible change today. When the AP is reactivated the captive will
inherit the same ×4 CPU win the WG path saw — and it stays in
symmetry with mitm-wg so future tweaks land on both.
B — Addon SQLite writes are now fire-and-forget via singleton
ThreadPoolExecutor (max_workers=1, thread_name_prefix=…) :
local_store._insert → submitted to sbx_store_write
utiq.record_event → submitted to sbx_utiq_write
The hook returns instantly ; the _conn() open + INSERT + fsync chain
runs on the bg thread. No more event-loop stalls during peer flow
processing.
Live diagnostic on gk2 surfaced the actual bottleneck while shipping
this : one Linux PC peer (10.99.1.60) was generating ~3 client
connects/sec sustained, the second Linux PC (10.99.1.47) running in
parallel, and mitm-wg processing ~100 concurrent TLS sessions. At
that level the CPU is consumed by mitmproxy itself (TLS termination
+ per-flow H/2 stream parsing under the Python GIL), NOT by the
addon writes — so A and B don't move the needle today.
They remain shipped as defensive hygiene before Phase 9 multi-worker
fanout : when 4 mitm workers contend on the same SQLite file, the
fire-and-forget pattern matters.
The kbin.gk2.secubox.in landing page had a hand-written ordered list
showing only the iOS install steps. Visitors on Android / Linux /
macOS / Windows had to follow a generic flow then go elsewhere to
get their proper artifacts.
Refactored : the install section now renders the same 5 platform
panels as /wg/onboard, with server-side UA sniffing to open the
right panel first. Each panel keeps the same content (App Store
links / QR / .conf / .nmconnection / .mobileconfig / install
commands per OS).
Implementation : extracted _install_panels_html() from the prior
_render_onboard() so both endpoints render identical markup. CSS
classes stay landing-local (a fresh <style> block) so the existing
landing design language is preserved. No new endpoints ; the
landing.html.j2 receives `install_panels` + `install_platform`
context.
Visitor flow now : open kbin.gk2.secubox.in → "📥 Installer R3 sur
ton appareil" → already-open panel matches the visitor's device →
single click on the install button or QR scan.
Live verified on gk2 : curl with iPhone UA renders `id="install-ios"
open` first ; Linux UA renders `id="install-linux" open` first ;
HTTP 200, 28 KB rendered page.
Live diagnostic on gk2 showed the host mitm-wg process pegged at
65 % single-core CPU at near-zero throughput, with 35 enrolled peers
and 3 active. Root cause is the single Python asyncio event loop
running 10 addons per flow ; Phase 6.P disabling HTTP/2 and Phase 6.J
forcing Connection:close upstream were the right calls at the time
(memory leaks in older mitmproxy) but their cost has been CPU-bound
since the mitmproxy 11 upgrade.
Re-enabled three settings in the launcher :
--set http2=true — multiplex halves TLS handshakes
--set connection_strategy=eager — overlaps upstream RTT with
downstream parse
--set keep_host_header=true — upstream keep-alive (mitmproxy 11
fixed the leak)
To compensate the H/2 in-process state drift, the
service.d/10-runtime-max.conf drops the recycle from 6 h to 3 h.
Memory still bounded by the 400 MB envelope ; restart latency stays
sub-second so no user-visible impact.
Live verified : CPU 65 % → 12 % at the same workload level. Banner
injection + utiq_defense addon still firing correctly. iPhone tunnel
+ Linux PC tunnel both functional through the restart.
Multi-worker fanout (true horizontal scaling — N mitmproxy workers
sharded by nft DNAT hash) tracked as Phase 9 in TODO ; the
single-process quick win above is enough headroom for the current
~35 enrolled peers, deferring the worker pool work.
Operator-grade tracking detection / blocking for the Utiq consortium
(Deutsche Telekom + Orange + Telefónica + Vodafone, launched 2023).
Utiq issues a 90-day-stable mtid identifier per publisher based on
SIM + IP carrier-level auth — cookies and standard anti-fingerprint
defenses don't help.
Quick Win ships R0 + R1 :
R0 (log) — every flow involving *.utiq.com or utiqLoader.js is
stored in /var/lib/secubox/toolbox/toolbox.db so the
operator can audit. No effect on the flow.
R1 (block) — same detection, but flow.response is short-circuited
to 451 (Unavailable For Legal Reasons) before the
upstream is contacted. No mtid is ever revealed to
the publisher. Some pages may degrade — the worst
case is no targeted ad. Toggle globally via env var
UTIQ_DEFAULT_LEVEL=R1.
Per-client toggling and R2 (mask) / R3 (pseudo-avatar) ship in Phase 2.
Files :
mitmproxy_addons/utiq_defense.py
- hooks requestheaders (short-circuits before body fetched)
- matches *.utiq.com (host) + utiqLoader.js (path, for 1st-party
CNAME wrappers)
- records via secubox_toolbox.utiq.record_event
secubox_toolbox/utiq.py
- utiq_events table : id, ts, client_ip, publisher (extracted
from host), host, path, action, level, detected_mtid (future),
injected_mtid (future)
- indexes on ts, (client_ip, ts), (publisher, ts)
- recent() / aggregates() / client_recent_count() helpers
secubox_toolbox/api.py
- GET /admin/utiq-events?hours=&limit= → events + aggregates
mitmproxy_addons/inject_banner.py
- new ctx['utiq_recent_count'] from utiq.client_recent_count
- '📡 utiq:N' tile appears when count > 0
sbin/secubox-toolbox-mitm-wg-launch
- utiq_defense inserted between inject_xff and local_store
Live verified : addon loaded, /admin/utiq-events returns empty events
+ aggregates skeleton, no regressions on the existing addon chain.
Closes the Quick Win checkbox in #500.
Phase 7.E.3 follow-up — every LXC on br-lxc (10.100.0.0/24) had
DNS=10.100.0.1 from its static config but the host Unbound only
listened on 10.99.x (the WG subnets shipped in Phase 7.E.1 by
99-secubox-wg.conf). External DNS from inside any LXC failed
silently.
Symptom found 2026-06-08 : the Gitea pull-mirror that lives in the
gitea LXC was failing to sync from upstream
github.com/CyberMind-FR/secubox-deb with :
Stderr: fatal: unable to access
'https://github.com/CyberMind-FR/secubox-deb.git/':
Could not resolve host: github.com
Failure dated back to 2026-05-17 (the day before the last successful
sync). No retry alert because the mirror is "schedule, log, move on"
and Gitea doesn't promote DNS errors to dashboard warnings. The
Gitea mirror operator only noticed because someone checked the repo
was 3 weeks behind master.
Fix : new unbound drop-in 98-secubox-lxc.conf listens on 10.100.0.1
with access-control 10.100.0.0/24 allow. Postinst now installs both
drop-ins (wg + lxc) in one pass.
Live verified : `getent hosts github.com` from inside gitea LXC
returns 140.82.121.4 ; the next mirror tick (10 min for
secubox/secubox-deb, 8 h for gandalf/secubox-deb) catches up the
~50 missing commits automatically.
After today's `cscli console enroll …` the SecuBox CrowdSec dashboard
still showed "CAPI Not enrolled" + "LAPI not reachable" — even with
`cscli capi status` confirming successful CAPI auth from a root shell.
Root cause : `/api/v1/crowdsec/status` shells out to
sudo cscli lapi status
sudo cscli capi status
sudo cscli decisions list -o json
sudo cscli alerts list -o json
plus `/api/v1/crowdsec/console/status` calls
sudo cscli console status
The 1.0.6 sudoers.d drop-in only listed `cscli bouncers list` and
`nft list tables` — the 5 above had no NOPASSWD entry. Under the
aggregator user `secubox` they triggered sudo's password prompt,
returncode != 0, and the FastAPI mapped them all to `false`.
This drop adds the missing 5 read-only commands. All exact-match, no
wildcards, no shell — residual capability stays bounded to read-only
listings.
Verified live on gk2 : the /status endpoint now returns
running: True
lapi_reachable: True
capi_registered: True
decisions_count: 6
alerts_count: 50
Tracking files mis à jour pour la journée du 2026-06-08 :
WIP.md — section Phase 7.E.x avec bumps secubox-waf 1.2.1,
secubox-toolbox 2.3.2, secubox-auth 1.0.2,
secubox-users 1.4.2 + finding architectural wg-LXC
+ plan Phase 8 Utiq.
HISTORY.md — entry datée avec détail des 6 axes shippés
(LXC WAF hygiene, /wg/onboard, NM name fix, LXC wg
scaffolding, auth recovery, Phase 8 ouverture).
TODO.md — backlog Phase 8 Quick Win/Phase 2/Phase 3 + follow-ups
7.E (rebuild + dpkg upgrade live secubox-aggregator
0.2.1 et secubox-users 1.4.2) + Phase 9 (futur multi-
instance mitm-wg dispatcher).
Wiki Phase-7-Roadmap.md mis à jour en parallèle dans
secubox-deb.wiki@1610d70.
After Phase 7.D consolidation, the aggregator runs as user `secubox`.
secubox_core.user_store reads BOTH /etc/secubox/users.json and (as
fallback) /etc/secubox/auth.toml. The secubox-users postinst chowned
users.json to root:secubox 0640, but auth.toml had NEVER been touched
by any postinst — left at the package default 0600 root:root since the
secubox-auth 0.1.0 install in May 2026.
Live symptom : every admin login returned "Identifiants incorrects"
no matter what password was typed. The aggregator log showed :
user_store: fallback to auth.toml for get_user(admin)
user_store: auth.toml unreadable ([Errno 13] Permission denied:
'/etc/secubox/auth.toml')
The fallback path was the only reason auth.toml mattered at all (the
canonical store is users.json) — but with users.json reset to root:root
by an earlier intervention, the fallback became the active path and
its permission denial took down the entire auth chain.
This commit ensures both files are root:secubox 0640 after every
package install / upgrade.
Admin login UI was permanently flagging "Clock not synced" with the
warning "TOTP window widened to ±60s" even when the system clock was
correctly synced. Cause : ntp_health.probe() only knew how to read
chrony, but SecuBox boxes ship systemd-timesyncd by default ; chrony
is the operator-opt-in alternative. On a stock install the probe
caught FileNotFoundError, returned `synced: False`, and the
recommended_totp_window() widened to ±60s on EVERY login.
Added a chrony → timedatectl fallback chain :
1. Try `chronyc -n tracking` (chrony users).
2. Fall through to `timedatectl show` reading NTP +
NTPSynchronized properties (timesyncd users).
3. Only return the synthetic error if BOTH commands are missing.
The response now carries an extra `source` field
("chrony" / "timedatectl") so the operator can see which backend
answered the probe.
Continuation of the wg LXC migration. Resolved the unprivileged-LXC
bind-mount denial that blocked 2.3.1 by switching the wg target to
PRIVILEGED LXC (matches the existing mitmproxy WAF LXC convention),
and parameterised the launcher's listen-host so the LXC can bind
0.0.0.0 on its br-lxc interface.
Live cutover attempt then surfaced a deeper architectural issue :
mitmproxy transparent mode uses Linux's SO_ORIGINAL_DST socket
option to recover the original destination IP/port. SO_ORIGINAL_DST
is conntrack-backed, and conntrack is namespace-scoped. The DNAT
entries created by the host's `nft prerouting` chain are invisible
inside the LXC's netns. Every flow errors with :
Transparent mode failure: FileNotFoundError(2, 'No such file or
directory')
and gets RST'd after the TLS ClientHello.
Two viable architectures going forward, documented inline in the
LXC conf template :
1. Keep mitm-wg on the host (current path — the host service was
never disabled in source, only at runtime, and the rollback
re-enabled it ; the iPhone tunnel is back on the host mitm-wg).
2. Switch the LXC to `mitmproxy --mode wireguard` — mitmproxy
BECOMES the WG endpoint, no DNAT needed, conntrack stays in
the LXC netns. Requires migrating wg-peers.json to mitm's
server.conf format + moving UDP 51820 ingress (host nft DNAT
instead of wg-quick on host).
The LXC infrastructure (privileged template, env-driven launcher,
provisioner) stays shipped as scaffolding for the option-2 migration.
Live state of gk2 host : back on host mitm-wg, iPhone tunnel rx
84 MB / tx 733 MB, growing normally.
Two related improvements + one honest WIP note.
== NM connection id = village3b-r3-<last-octet> ==
The .nmconnection endpoint previously used `client_label` (the visitor
User-Agent) as the NetworkManager connection `id`. Operators ended up
with entries called "Mozilla/5.0 (X11; Linux x86_64; rv:151.0)
Gecko/20100101 Firefox/151.0" in the Network panel. Switched to
"village3b-r3-<last-octet>" : short, stable, distinct per peer (last
octet of the assigned 10.99.1.x).
== lxc-provision refinements ==
Multiple discoveries during the wg LXC migration attempt :
- lxc-create --template download instead of `debian` (the latter
requires PRIVILEGED LXC).
- LXC reads its runtime config from /data/lxc/<name>/config, NOT
/etc/lxc/<name>.conf. Write a clean, deduped config that
preserves the auto-generated idmap.
- Unprivileged LXCs do NOT auto-apply lxc.net.0.ipv4.address (the
kernel won't let the container set its own address without
CAP_NET_ADMIN on the netns). Push the IP + default route + DNS
+ systemd-networkd profile via lxc-attach post-start.
- Install python3 + curl + procps in the container only AFTER the
network + DNS are up — apt fails earlier.
== Known roadblock — not yet shipped live ==
The wg LXC bind-mount of /etc/secubox/toolbox/ca-wg fails with
"Permission denied" because the dir is 0750 root:secubox-toolbox and
the unprivileged LXC's mapped UID (100000) can't read it. Two
resolutions (documented inline in the conf template) :
1. chmod 0755 /etc/secubox/toolbox/ca-wg (relaxes CA file
visibility — OK for the public cert pem, not for the private
key inside it).
2. Drop lxc.idmap to switch to privileged mode (matches the
existing mitmproxy LXC convention).
The host secubox-toolbox-mitm-wg.service stays the active path
until the operator resolves this — no surprise cutover. The iPhone
tunnel is unaffected.
Three independent improvements landed together (small enough that
splitting would have churned the same files twice).
== secubox-waf 1.2.1 : LXC mitmproxy.service hygiene ==
Live diagnosis on gk2 found the LXC mitmproxy WAF process running
1 d 16 h with :
- 200 MB RSS, 9 threads
- 934 ESTAB TCP connections persisting
- 51 % response-error rate on the HAProxy backend
(eresp = 3.7 M of 7.2 M sessions)
- one core saturated (87 % of an ARM Cortex-A72)
Phase 6.P had shipped RuntimeMaxSec=21600 (6 h) drop-ins for the
host-side mitm-wg + mitm captive but never reached the LXC service.
Adding it now :
- RuntimeMaxSec=21600 : clean cycle every 6 h
- MemoryHigh=400M, MemoryMax=512M : same envelope as host units,
OOM-kill cleaner than swap thrash under attack traffic
Live restart applied — Until: 12:55:16 UTC.
== secubox-toolbox 2.3.0 : auto-detect /wg/onboard ==
End-user feedback : the current activation flow is fragmented across
"go to kbin.gk2.secubox.in, scan QR, install mobileconfig, trust CA,
…" with per-platform variations. Replaced by a single landing page
that detects the visitor's UA and renders the right one-click flow
first. Sections covered :
- iOS / iPadOS : WireGuard App Store link + QR + ca.mobileconfig
+ trust toggle reminder + battery-saver caveat.
- Android : Play Store link + QR + ca.pem + ca.cer + Settings
path + Samsung/Xiaomi user-CA hint.
- Linux : .nmconnection + .conf + ca.pem + nmcli reload one-liner
+ update-ca-certificates + Firefox-only note.
- macOS : Mac App Store link + .conf drag-onto-menu-bar +
Trousseau system import.
- Windows : official installer + add-tunnel-from-file + cert
store import + Firefox cert store caveat.
All artefacts stay at their existing paths ; the onboard page just
composes them. Hand a single URL to anyone — they get the right
flow first.
== secubox-toolbox 2.3.0 : mitm-wg LXC scaffolding ==
secubox-toolbox-lxc-provision now accepts \`transparent\` (default,
existing behaviour) or \`wg\` (new). \`wg\` provisions the
toolbox-mitm-wg LXC at 10.100.0.62 from the existing config +
service templates :
- Config template rewritten for TRANSPARENT mode on TCP 8081
(instead of mitmproxy --mode wireguard which would have required
a server.conf format peer migration). wg-quick stays on the
host ; only the mitmdump process moves to the LXC.
- Service template runs the SAME secubox-toolbox-mitm-wg-launch
script as the host service, bind-mounted in RO — single source
of truth for the addon chain + ignore_hosts compose logic.
- RuntimeMaxSec=21600 + memory caps match the WAF LXC's hygiene.
Cutover is OPT-IN — no automatic switchover in postinst. After a
successful provision the script prints :
1. lxc-attach -n toolbox-mitm-wg -- ss -tln | grep 8081
2. nft replace rule … dnat ip to 10.100.0.62:8081
3. systemctl disable --now secubox-toolbox-mitm-wg.service
4. update /etc/nftables.d/secubox-toolbox-wg.nft for boot-survival.
Operator picks the cutover moment so the live iPhone tunnel session
is never broken mid-flow.
WebUI showed firewall errors after Phase 7.D consolidation. Crowdsec
module /health returned `status: degraded` with `bouncers_ok: false,
bouncers_count: 0, nftables_ok: false` even though :
- `crowdsec.service` was active
- `crowdsec-firewall-bouncer.service` was active
- cscli + nft both worked perfectly from a root shell
- 2 bouncers were properly registered (cs-firewall-bouncer + secubox-waf)
- inet crowdsec / ip6 crowdsec6 tables were live
Three independent gaps in the new aggregator-mounted module context :
1. The /health endpoint runs `sudo cscli bouncers list -o json` and
`sudo nft list tables`, but the aggregator user (`secubox`) had
no sudoers entry → cscli refused with "permission denied" before
any actual query.
2. The aggregator unit had `NoNewPrivileges=true`. Even after
installing the right sudoers, the subprocess could not gain
privileges via the setuid sudo binary — systemd's flag blocks
all privilege escalation including narrowly-scoped sudoers.
3. The aggregator unit had `ProtectSystem=strict` which makes the
entire filesystem read-only (incl. /var). When cscli starts, it
touches perms on /var/lib/crowdsec/data/crowdsec.db. Under
ProtectSystem=strict that chmod() fails with EROFS and cscli
exits with rc=1 before printing the bouncer list — even when
sudo and the sudoers entry are both in order.
Fixes :
secubox-crowdsec :
- new sudoers.d/secubox-crowdsec drop-in shipped to
/etc/sudoers.d/. Two exact-match entries (no wildcards, no
shell, no flag escapes) :
secubox ALL=(root) NOPASSWD: /usr/bin/cscli bouncers list -o json
secubox ALL=(root) NOPASSWD: /usr/sbin/nft list tables
- debian/rules installs the drop-in to /etc/sudoers.d/ at
0440 root:root.
secubox-aggregator :
- systemd unit : NoNewPrivileges=false so sudo can escalate via
the sudoers drop-in. The sudoers entries themselves are the
defense in depth — they cap what the aggregator user can
escalate to two read-only listings.
- systemd unit : ReadWritePaths += /var/lib/crowdsec so cscli's
perm-touch on crowdsec.db succeeds ; += /etc/secubox for the
toolbox/dpi/etc modules that write config there.
Live verified : status=ok, healthy=true, bouncers_count=2,
nftables_crowdsec=true, nftables_crowdsec6=true.
Bumps :
secubox-aggregator 0.2.0 → 0.2.1
secubox-crowdsec 1.0.5 → 1.0.6
