* docs(sentinel): design — inline IOC gate + async Go analyzer + spyware packs (ref #823)
* docs(sentinel): implementation plan — 12 tasks (ref #823)
* feat(sentinel): IOC model + IOCSet matchers (ref #823)
* feat(sentinel): pack loader + live-overlay merge + hot-reload (ref #823)
Add internal/sentinel/pack.go: Pack (JSON base/overlay content bundle),
LoadBasePack, MergePacks (base + N overlays, override by Type+Value via
IOCSet's per-type maps), and Loader (NewLoader/Set/MaybeReload/YaraRules).
Loader mirrors cmd/sbxmitm/policy.go's LoadPolicy/maybeReload mtime-based
hot-reload pattern, adapted for a directory of a DYNAMIC *.json file set
(glob-based) rather than policy.go's fixed Target list. Fail-safe: a
corrupt overlay pack is skipped+logged, never wipes the base or panics;
MaybeReload is a cheap stat-only no-op when nothing on disk changed;
Set()/YaraRules() read via atomic.Pointer so a reload swap never torn-reads
a concurrent hot-path lookup.
Adds JSON tags to Task 1's IOC struct (type/value/class/severity/source/
action) so packs round-trip through encoding/json.
* fix(sentinel): pack loader — fail-closed on base-wipe + throttle MaybeReload + url_regex override + yara dedup (ref #823)
Review found 1 Critical + 2 Important + 1 Minor in Task 2 (pack loader):
- Critical: a vanished/degraded base pack directory (deleted dir, mount
blip, mid-dpkg-upgrade window) could silently empty the live detection
set on the next MaybeReload. loaderState now tracks baseCount; a reload
that would drop base IOCs from >0 to 0 is refused (prior state kept,
warning logged) instead of swapped in.
- Important: MaybeReload() is called per-flow by Task 3's inline gate: add
a 5s default throttle (mirrors cmd/sbxmitm/policy.go's reloadThrottle)
so a hot-path call is a pure no-op until the window elapses.
- Important: url_regex IOCs previously only appended, so a live overlay
could never override a base pack's URL rule (base always matched
first). Add urlIndex (pattern -> slice index) so re-adding the same
pattern replaces the entry in place, keeping deterministic match order.
- Minor: YaraRules() now dedups identical rule bodies across base+overlay
packs instead of emitting duplicates.
go test ./internal/sentinel/: 21/21 PASS. go vet/gofmt/go build clean.
* feat(sentinel): inline gate matcher (fail-open, highest-severity) (ref #823)
* feat(sentinel): bbolt verdict store (pure-Go, WAL-free, TTL prune) (ref #823)
* feat(sentinel): action scorer — heuristic/low-confidence forced report-only (ref #823)
* feat(sentinel): bounded fire-and-forget mirror channel (drop-with-count) (ref #823)
* fix(sentinel): mirror Emit always copies body — no aliasing of caller's hot-path buffer (ref #823)
* feat(sbxmitm): inline Sentinel gate — neutralize high-confidence, mirror rest, fail-open (ref #823)
Wire internal/sentinel's inline IOC gate into the shared mitmPipeline response
path (covers both the transparent R3 and CONNECT accept paths without drift):
build a FlowMeta from Host/URL/ClientIP/MacHash after up.Do, call the gate, and
on a HIGH-CONFIDENCE known-infra hit neutralize inline — block/sinkhole serve a
Sentinel block page instead of the upstream response, strip serves the response
with an emptied body. Everything else (miss, heuristic, low-confidence, report)
is mirrored bounded fire-and-forget to the async analyzer and proceeds unchanged.
Fail-open everywhere: SENTINEL_ENABLED unset/false, a pack-load failure, a nil
hook, or a gate panic all degrade to a transparent passthrough — a Sentinel
setup problem can never break browsing. With the engine off, behavior is
byte-identical to today. Env: SENTINEL_ENABLED / SENTINEL_PACK_DIR /
SENTINEL_OVERLAY_DIR / SENTINEL_MIRROR_SOCK.
Hot path (BenchmarkSentinelInspectMiss, -benchmem): 101.0 ns/op, 0 B/op,
0 allocs/op — one throttled gate Match + a non-blocking mirror Emit; no YARA or
heavy analysis inline.
go test ./cmd/sbxmitm/... -race → ok (all pre-existing + 7 new green).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(sentinel): sbx-sentinel async daemon scaffold + store wiring (ref #823)
* feat(sentinel): YARA engine (cgo, build-tagged) + no-cgo stub (ref #823)
Add internal/sentinel/yara.go (cgo && yara) wrapping
github.com/hillu/go-yara/v4 for compiled-rule body scanning, and
yara_stub.go (!yara) as the no-op default: NewYaraEngine always
succeeds, Scan/Analyze always return nil. Both expose identical
signatures (YaraEngine, NewYaraEngine, Scan, Analyze, Close) so
sbx-sentinel/sbxmitm can wire a YaraEngine unconditionally.
yara_test.go (untagged) proves the default build works with YARA
disabled. yara_real_test.go (yara tag) compiles a trivial EICAR rule
and exercises the real libyara-backed engine — split into its own
file since Go build constraints are file-scoped and can't mix a
tagged and untagged test in one file.
go-yara is now an indirect go.mod requirement and vendored, but is
only reachable via the yara build tag: default `go build ./...` and
`CGO_ENABLED=0 go build ./...` both stay clean, no libyara/cgo
required. `-tags yara` needs libyara-dev + a C toolchain (documented
in yara.go's header).
* fix(sentinel): YARA verdict confidence 90 (>= threshold) so a confirmed malware match actually strips, not report-only (ref #823)
* feat(sentinel): behavioral engine — beaconing + zero-click heuristics (report-only) (ref #823)
* fix(sentinel): tighten one-time-link heuristic — exclude file-extension + base64 shapes (cut zero-click FPs) (ref #823)
* feat(sentinel): spyware indicator engine + commercial-spyware base pack (MVT/CitizenLab) (ref #823)
* fix(sentinel): base-pack declared-block IOCs severity>=85 so they actually block + general guard test (ref #823)
Three shipped base-pack IOCs declared action:"block" but severity:80,
below scorer.go's HighConfidenceThreshold=85 — FinalizeAction silently
downgrades any Confidence below that threshold to report-only, so these
entries never actually blocked despite being declared as known-bad:
- packs/base/malware.json: ip 198.51.100.23 (class malware)
- packs/base/malware.json: cert_sha1 ...01234567 (class trojan)
- packs/base/botnet.json: ja3 deadbeef... (class botnet_c2)
Bumped all three to severity 90 so they survive FinalizeAction and
actually block, matching every other declared-block entry in the base
packs.
Added TestBasePackDeclaredBlockIsHighSeverity, a general guard over
every base pack file/entry: any IOC with an auto-neutralize action
(block/strip/sinkhole) must carry Severity >= HighConfidenceThreshold.
This generalizes the narrower TestBasePackKnownInfraIsHighSeverity
(spyware classes only) so a future pack entry in any class can't ship
a "block" that silently never blocks. Confirmed the new test fails red
against the pre-fix JSON (all 3 offenders reported) and passes green
after the severity bump.
* feat(sentinel): proposal/solution reporter — RenderReport(Verdict) (ref #823)
* feat(sentinel): wire real analyzers into sbx-sentinel + read-only status HTTP surface (ref #823)
* feat(sentinel): live threat-feed → overlay pack fetcher (fail-safe, atomic) (ref #823)
* feat(sentinel): live feed overlay + reporter + daemon analyzer wiring + packaging (ref #823)
* fix(sentinel): feeds fetcher — add curl --fail so HTTP 4xx/5xx is treated as a fetch failure (ref #823)
* fix(sbxmitm): drop mis-wired ClientIP from inline Sentinel FlowMeta (source IP must not match destination-C2 IP IOCs) (ref #823)
---------
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* feat(sbxmitm): capture HLS segments as kind=segment (Phase 2, ref #812)
Add segmentKind(path, ctype) to classify HLS/DASH segment chunks (.ts,
.m4s, .m4v paths or video/mp2t, video/iso.segment content types), and
give it precedence over mediaKind in Capture so a .ts served as
video/mp2t is tagged kind="segment" instead of "video". Whole-file
.mp4/video/* stays "video" and .m3u8/.mpd manifests stay "manifest".
IsMedia now also matches segmentKind so ambiguous-ctype segments (e.g.
.m4s over application/octet-stream) are still captured.
* feat(core): HLS media-playlist parser + segment-URI rewriter (Phase 2, ref #812)
* feat(dpi): HLS manifest replay — rewrite segment URIs to captured segments (Phase 2, ref #812)
media_replay() branches on record kind: a captured HLS manifest is parsed
with secubox_core.hls and its segment URIs are rewritten to the matching
captured kind=="segment" records' replay URLs (read-time join by absolute
URL, scoped to the same mac_hash+host). Master/ABR and AES-128 encrypted
playlists are detected and returned raw with X-SecuBox-Media:
unsupported-variant instead of a broken rewrite. Every other kind keeps the
unchanged Phase-1 FileResponse path; any manifest parse/read error falls
back to the raw FileResponse (never a 500).
* fix(dpi): log HLS manifest/segment/rewrite truncation + pin manifest replay fail-safe (Phase 2 review, ref #812)
* feat(dpi): Media tab — hide HLS segments, manifest playback/download + unsupported-variant note (Phase 2, ref #812)
* test(#812): HLS reassembly end-to-end + changelog 0.1.28 (Phase 2)
Cross-layer integration test seeding a temp media-buffer exactly as the Go
tee + janitor produce on disk (metatag JSONL + per-session object files),
then driving the real DPI media_replay handler: manifest rewrite joins each
segment URI to its captured segment by resolved URL, every rewritten URL's
id replays to byte-identical captured bytes, and a janitor-style append-only
eviction flip leaves the evicted segment unrewritten (410 on direct replay)
while its siblings keep replaying. This is the whole-branch-review-flagged
integration coverage Phase 1 was missing.
Bumps secubox-toolbox-ng changelog to 0.1.28-1~bookworm1 with the Phase 2
HLS reassembly summary (segment capture, manifest rewrite,
unsupported-variant detection for master/ABR + AES + DASH).
* chore(dpi): changelog 1.2.0 — media buffer API + HLS reassembly (#812 Phase 1+2)
---------
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
* docs(media-buffer): design spec — dashcam capture + replay + metatag lifecycle (ref #812)
* docs(media-buffer): phase-1 implementation plan (capture+janitor+API+DPI tab) (ref #812)
* feat(sbxmitm): media buffer store + metatag writer (ref #812)
* fix(sbxmitm): media-buffer review — 4096 atomic-append cap + *string buffer_ref (ref #812)
Task 1 review (Important): mediaBufferLineMax 8192→4096 to match mediacatch.go's
PIPE_BUF atomic-append guarantee across 4 workers; BufferRef string→*string so the
janitor (Task 3) can flip it to JSON null on eviction.
* feat(sbxmitm): tee media up/download bodies into the buffer (ref #812)
Wire the R4 media buffer tees into mitmPipeline: the download hook wraps
resp.Body and the upload path wraps req.Body via teeReadCloser, capturing
whole-file media into the rolling /data buffer. Add the mbuf field + flags
(--media-buffer default OFF, --media-buffer-root, --media-buffer-per-object).
Design upgrade over the plan sketch: ObjectWriter.Write is now NON-BLOCKING.
Instead of io.TeeReader driving a synchronous os.File.Write on the client read
path (which would add latency / could stall large media on ARM eMMC), Write
copies each chunk into a bounded channel and returns len(p),nil immediately; a
background goroutine started at Capture drains the channel to disk. A full
channel drops the chunk and flags truncated — never blocks, never errors the
proxied flow (design 4.2/7). Close signals the drainer, waits for it to flush
queued chunks + close the file, then appends the metatag; it is idempotent,
deadlock-free (sole closer of ch) and safe if Write is never called or races
Close.
* fix(sbxmitm): media download tee — defer closure so w.Close runs (ref #812)
defer resp.Body.Close() bound the ORIGINAL upstream body at the defer
statement, before the media-buffer tee reassigned resp.Body — so the
tee's Close() (and w.Close, which flushes the sink/metatag/drain
goroutine) never ran on any download. Wrap the close in a closure so
it reads resp.Body at return time instead.
Adds TestTeeDeferOrderingMatchesMainGo, which replicates main.go's
exact defer-then-reassign ordering and fails against the old plain
defer form (verified manually, then restored the fix).
Also: drop the dead write-only ObjectWriter.dropped field (truncated
already covers it), document Capture's reserved contentLen param, and
note that ObjectWriter.Close is allowed to block (post-stream
finalisation), unlike Write.
* feat(sbxmitm): media buffer janitor — time + LRU eviction, metatag survives (ref #812)
SweepOnce evicts a session's bytes (RemoveAll, idempotent) once
first_ts+retentionSecs elapses, then falls back to oldest-first LRU
eviction if the buffer root (media bytes only, log excluded) still
exceeds sizeCeilBytes. Eviction never rewrites the append-only
metatag log in place — it appends a corrected line with the same id
(expired:true, buffer_ref:null); readers dedup by id, last line wins,
so the flip is durable and safe across the 4 concurrent sbxmitm
worker processes each running their own RunJanitor (idempotent
RemoveAll + harmless duplicate flip lines).
RunJanitor ticks SweepOnce every 30s (test-overridable period) until
its context is cancelled; wired at sbxmitm startup as a
process-lifetime goroutine when --media-buffer is enabled.
NewMediaBuffer's signature is unchanged; retentionSecs (1200),
sizeCeilBytes (24 GiB) and nowFn now default inside the constructor.
* feat(core): media-buffer metatag reader (dedup by id, fail-empty) (ref #812)
* test(core): media-buffer reader — tail-read truncation + malformed-field regressions (ref #812)
* chore(#812): keep SDD scratch reports out of the feature diff (restore to master)
* feat(dpi): media-buffer list/replay/thumb API (admin/owner, audited, 410-on-evict, traversal-safe) (ref #812)
* harden(dpi): media API — \Z-anchored id regex + role short-circuit security note (ref #812)
* feat(sbxmitm): media-buffer retention/size-ceil flags + /data tmpfiles + opt-in worker env toggle (ref #812)
- main.go: --media-buffer-retention / --media-buffer-size-ceil flags, applied
onto the already-constructed mbuf's unexported retentionSecs/sizeCeilBytes
fields (NewMediaBuffer signature unchanged).
- tmpfiles/zz-secubox-media-buffer.conf: pre-creates /data/secubox (0755) and
/data/secubox/media-buffer (0750 secubox:secubox), installed via the
package's existing manual tmpfiles.d mechanism in debian/rules.
- secubox-toolbox-ng-worker@.service: EnvironmentFile=-/etc/secubox/toolbox-ng.env
+ trailing $SBX_MEDIA_BUFFER_ARGS on ExecStart so an operator can opt in
(SBX_MEDIA_BUFFER_ARGS=--media-buffer) without editing the shipped unit;
default stays OFF. Added ReadWritePaths=/data/secubox/media-buffer.
* fix(sbxmitm): media-buffer dir owned by writer (secubox-toolbox) + setgid group secubox; changelog 0.1.27 (ref #812)
Task 6 review Critical: buffer dir was 0750 secubox:secubox but sbxmitm workers run
as secubox-toolbox (not in group secubox) → EACCES on every capture, silently no-op'd
(errors swallowed by design). Fix: 2750 secubox-toolbox:secubox — writer owns, dpi
reader (secubox) is group, setgid propagates group secubox to created objects so the
reader can read them. Important: add debian/changelog 0.1.27 stanza (version bump).
* feat(dpi): Media tab — live capture gallery with replay/download, escaped fields (ref #812)
Adds a Media Buffer card to the DPI dashboard (packages/secubox-dpi/www/dpi/index.html)
that lists sbxmitm's live media-buffer captures via GET /api/v1/dpi/media/buffer.
- Auth: dedicated apiAuthed() reads the canonical `sbx_token` localStorage key
and sends it as a Bearer token (media/buffer requires JWT, unlike the rest
of this page's endpoints which currently ride the SSO-lite session cookie);
401 redirects to /login.html, matching sibling dashboards (e.g. crowdsec).
- Security: host/url/mac_hash are attacker-influenceable (captured live
traffic). Every card is built via createElement + textContent (or the
`.title` IDL property, never HTML-parsed) — no innerHTML string
concatenation of these fields — so a captured value like
`<img onerror=...>` renders as inert text. Replay/download URLs are built
with encodeURIComponent(id) even though the id is hex-validated server-side.
- UI: kind emoji (video/audio/manifest/file), short device id (shortDev,
reused from the existing #785 helpers), ⬆/⬇ direction, human size
(formatBytes) and age (agoStr) per card; expired captures grey out and show
"métatag seul (expiré)" with no action links.
- Refresh: wired into the existing refreshAll() plus its own ~15s interval.
Verified with a Playwright harness stubbing fetch (incl. an XSS payload in
host/url) against a local copy of the page: no console/page errors, XSS did
not fire, no <img>/<script> element materialized from the captured strings,
expired-card gating confirmed.
* fix(sbxmitm): media buffer — upload tee needs a real body, download tee 200-only (whole-branch review); drop dead item.ts (ref #812)
* docs(media-buffer): Phase 2 plan — HLS reassembly via read-time URL join (ref #812)
---------
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
secubox-dpi is aggregator-mounted (nginx /api/v1/dpi/ → aggregator.sock; dpi in
aggregator.toml), so its route handlers run in-process on the shared aggregator
uvicorn loop (~110 modules, single worker). 14 handlers were `async def` but call
blocking sync code (subprocess.run for status/logs/interface_list/tc_status/
restart/start/stop/remove_mirred; .read_text on 100–213KB caches for exfil_state/
exfil_history/realtime/block_rules/add_block_rule/delete_block_rule) with no await
— each blocked the whole loop while it ran → board-wide 502/504 (same class as the
peertube #804 and toolbox PDF #785 loop-blocks).
None use await and none are awaited internally, so plain `def` is a safe, clean
conversion: FastAPI runs def path operations in a threadpool, fully off the loop.
Board dpi/api/main.py == git source (no drift): the anti-pattern is in source, so
this is the source fix. test_media_types.py green.
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
The PeerTube dashboard bounced to /login.html?redirect=/peertube/ and looped after
a successful login. getToken() read localStorage 'jwt_token' || 'token' — keys the
canonical login.html never writes. login.html (and every other module webui: waf,
soc, crowdsec, metrics, shared sidebar.js) uses 'sbx_token'. So PeerTube always sent
an empty Authorization → API 401 → api()'s 401 handler redirected to login → login
stored sbx_token → back to /peertube/ → getToken() read jwt_token (empty) → 401 → loop.
Pre-existing divergence (present in the May-28 backup); not caused by #798. Same class
as the cookies webui drift (#749). Fix: read sbx_token first, keep the legacy keys as
fallback. Applied live on the board to unblock.
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
* fix(peertube): drain admin-ops results to a separate dir (fixes start-limit loop, ref #798)
The root peertube-ops.path watches OPS_DIR with DirectoryNotEmpty=. process-ops
wrote each <id>.result.json back into OPS_DIR, so the directory was never empty
after draining a request → the .path re-triggered peertube-ops.service in a tight
loop until systemd killed it with start-limit-hit, after which the mechanism died
after the first op.
Write results to a dedicated /run/secubox/peertube/results/ instead; process-ops
rm's the request, leaving OPS_DIR empty so the watcher goes idle. The API's
_read_op_result now polls RESULTS_DIR. tmpfiles + postinst create the dir
(0750 secubox:secubox); peertubectl honors SECUBOX_PEERTUBE_RESULTS_DIR.
test_process_ops.sh asserts results land in results/ and OPS_DIR is empty after
draining (regression guard); test_ops_api monkeypatches both dirs.
Verified live on gk2: two consecutive pings drain cleanly, NRestarts=0, no
start-limit-hit, results root:secubox 0640 (secubox-readable).
* fix(peertube): upgrade CWD must be peertube-latest/scripts + surface the error (ref #798)
The webui upgrade button failed with a generic "upgrade script failed" and no detail.
Two bugs in cmd_upgrade:
1. Wrong CWD. PeerTube's v8 shim peertube-latest/scripts/upgrade.sh execs
`../dist/scripts/upgrade.sh` relative to CWD, and that dist script runs
`node -e "require('js-yaml')…"` whose resolution is also CWD-relative. Running from
`cd /var/www/peertube` made `../dist` → /var/www/dist (missing) → "cannot open
../dist/scripts/upgrade.sh"; a neutral CWD instead fails "Cannot find module
'js-yaml'". Fix: cd /var/www/peertube/peertube-latest/scripts (../dist resolves,
and require() walks up to peertube-latest/node_modules).
2. Error was discarded to /dev/null, so the result only said "upgrade script failed".
Now capture to /run/secubox/peertube/upgrade-<id>.log and fold the last lines into
the error detail.
Verified live: with the corrected CWD, 8.2.0 → 8.2.2 upgraded cleanly (download + deps
+ migrations), service restarted, /api/v1/config 200, serverVersion 8.2.2.
---------
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
* fix(sbxwaf): forward WebSocket upgrades — don't clobber Connection: Upgrade (closes#796)
sbxwaf forced Connection: close on every upstream request (#496), which
overwrote Connection: Upgrade on WebSocket handshakes → httputil.ReverseProxy
stopped tunnelling the WS, forwarded a plain GET, and wss:// vhosts (zigbee, …)
got 404/1006. Guard the Connection rewrite with isWebSocketUpgrade() so upgrades
are tunnelled (still routed + inspected via the WAF, no bypass).
* fix(sbxwaf): statusRecorder must implement http.Hijacker for WS upgrades (#796)
The visit-stats wrapper (statusRecorder, #747) embeds http.ResponseWriter but
didn't expose Hijack, so with --visits-stats active ReverseProxy's WebSocket
upgrade failed the http.Hijacker assertion → 503. Forward Hijack to the
underlying writer. Test now enables visits so it exercises the wrapped path.
* fix(sbxwaf): cachingResponseWriter must forward Hijack + log upstream errors (#796)
Third WS-breaking layer: the media-cache tee wrapper (cachingResponseWriter)
also shadowed http.Hijacker → ReverseProxy's WS upgrade failed with 'can't
switch protocols using non-Hijacker ResponseWriter type *cachingResponseWriter'.
Forward Hijack to the underlying writer (hijacked conn is raw-tunnelled, nothing
cached). Also log upstream errors in the routes proxy ErrorHandler — they were
invisible before, which is what made this hard to diagnose.
---------
Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
The union_blobs function now guards against non-list field values (e.g. int,
bool, None) that could cause TypeError during iteration. Instead of assuming
a field is always a list, the function checks isinstance() before iterating.
Adds test_union_blobs_tolerates_malformed_fields() to verify that a valid
blob with malformed fields does not crash the entire federation sync.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Engine (sbxmitm): new operator-disabled set (mitm-filter-disabled.txt, hot-
reloaded) suppresses matching per-pattern in BOTH the bypass and splice paths —
so unchecking an entry in the webui actually stops the R3 engine applying it,
including package-seed entries. bypassEntry keeps the source pattern beside the
compiled regex; hostMatchesEnabled skips disabled splice suffixes. Tests:
TestDisabledSuppressesMatch (bypass + splice) + full suite green.
API: POST /admin/filter-control/toggle (add/remove from disabled) + /delete
(remove line from editable files; package seed → disable). /list rows gain
enabled + editable.
Webui: each Filtres MITM row gets a ✅ checkbox (bound to toggle, greyed+struck
when disabled) + 🗑 delete (locked-tooltip for seeds). Live on gk2 (workers +
portal restarted, old binary at sbxmitm.bak.pre-809).
'Pubs bloquées: 0' was misleading — ad-blocking moved to the DNS layer
(secubox-adblock-sync #740: Unbound sinkholes 757,946 ad/tracker domains via
always_nxdomain), so ads die BEFORE reaching the engine → the engine's 204
counter reads ~0. /admin/ad-stats now includes dns_sinkhole (from ad-guard
sinkhole-status.json) and the #ads tab leads with a '🛡️ Blocage DNS' card
(domains + rules + sources), relabels the engine card '204 moteur MITM'. Live gk2.
#803 folded the mitm-bypass match into shouldSplice (checked BEFORE ad-block),
so ad networks that also live in the bypass seed (adform/amazon-adsystem/
rubiconproject/smartadserver…) started SPLICING instead of being 204-blocked —
silently disabling ad-blocking for them. Move the bypass match OUT of
shouldSplice into Decide AFTER blockedByAd: explicit tls-splice → ad-block →
bypass-splice (cert-pinned apps only). Signal/banks still splice; ad nets in the
bypass list block again. New regression-guard case (adform.net→block). Live gk2.
Ships tmpfiles.d/secubox-toolbox.conf creating /run/secubox/media-catch.jsonl
0644 secubox-toolbox:secubox at boot (before services), so the R3 workers can
always append. Fixes the wrong-owner race where the file was pre-created as
secubox → workers' write-open failed → media-catch silently empty. Makes the
2026-07-04 live perms fix durable across reboots. Applied live on gk2.
ad_stats now returns total_candidates + top_candidates from ad_candidates
(observed ad/tracker hosts awaiting autolearn promotion to active 204-blocking).
The #ads tab gains a '🔎 Trackers/pubs détectés' card so the dashboard reflects
real detection even when total_blocked is 0 (nothing promoted yet) — was
showing all-zeros + empty tables despite live detection. Deployed to gk2.
The R3 Go engine only honoured the TLS-splice list, so every cert-pinned app in
the mitm-bypass (ignore_hosts) list — Signal/WhatsApp/Telegram/Apple/banks,
incl. regex-wildcard patterns a suffix list can't express (ca-.*\.fr) — was
MITM'd and broke through the tunnel. Load the three bypass files
(seed/static/dynamic) as anchored case-insensitive regexes; a match in
shouldSplice() now returns splice (never-set still wins). Hot-reloaded on edit/
autolearn like the splice lists. This is the systemic fix behind the
Claude/Signal interim splice-learned merges. Live on gk2 (workers restarted);
full sbxmitm suite green + new TestBypassRegexSplices.
populate_exempt() now calls secubox-toolbox-tor-exempt-hosts (resolves the
splice/cert-pinned list → IPs → tor_exempt) so spliced pinned upstreams (Claude,
Signal, banks) egress DIRECT when Tor egress is armed — else the spliced
passthrough is torified and the pinned API breaks. Ships the resolver +
refresh timer (30min, for CDN IP rotation), enabled in postinst. Deployed live
to gk2 (Tor egress currently ON): exempted 14 pinned hosts / 28 IPs.
- /admin/filter-control/list now also surfaces the TLS-splice lists
(tls-splice-seed + splice-learned, inline-comment-stripped) tagged
splice-seed/splice-learned, so every explicitly-spliced host is displayed +
known (was: only the 3 bypass files → splice hosts invisible). 84 entries now.
- webui: removed the slice(0,5) cap → shows all with a scroll container + count
header + 🔀 splice badges.
- add secubox-toolbox-tor-exempt-hosts resolver (#799, not yet wired/deployed).
- secubox-toolbox-autolearn now mirrors each splice-learned host into
mitm-bypass-dynamic.conf (the Filtres MITM '🔍 learned' source) as bypass
regex, so autolearned splice hosts are visible + editable in the webui and
honoured on the ignore_hosts path. Merge (never clobbers operator entries).
- secubox-mitmproxy filters.html: 'WAF Filters' → 'Filtres MITM' (title/h1/desc);
API path /api/v1/mitmproxy/waf unchanged.
Adds redirect_to to VHostCreate + generate_vhost_config: when set, the vhost is
a pure 301 redirect (no proxy, no exposure gate). create_vhost requires a
backend only for non-redirect vhosts. Covers the www.example.com → apex pattern
(deployed live for www.ganimed.fr). 18 vhost tests pass.
Only the exact api.anthropic.com was spliced; Claude Code / the SDK also hit
s-cdn.anthropic.com + statsig.anthropic.com (seen MITM'd in ja4 logs), which
cert-pin → the API broke through the wg-toolbox tunnel. Splice the whole
anthropic.com domain (suffix covers all subdomains) + claude.ai. Live fix
applied to gk2 splice-learned.txt + workers restarted; this makes it durable
and fleet-wide.
Review found the cert-pin gate only covered POST /exposure — /tor/add and
/emancipate (which routes through tor_add) could still mint an onion for a
pinned host. Gate tor_add on the ORIGINAL service (before dot-stripping) so all
three Tor-creation paths are covered. Also: _lines() now fails open on
UnicodeDecodeError (a non-UTF-8 byte no longer 500s the gate), and an
mtime-keyed cache avoids recompiling the bypass regexes on every request.
Deferred (tracked in #797): periodic re-validation of already-active onions when
a host is autolearned into exclusion later; the live Go sbxmitm engine consumes
only the splice list (bypass regex half is webui/autolearn config).
The per-vhost Tor channel now consults the toolbox's own MITM-exclusion lists
(ignore_hosts bypass regex + TLS-splice suffix, both operator-editable in the
Filtres MITM webui and autolearned) and hard-refuses (400) a hidden service for
any matching host — never anonymously re-expose an endpoint clients pin. reach
(lan/wan) still applies. Toolbox webui shows a Tor-exclu badge.
