sync-all.yml's release job does `uses: ./.github/workflows/release.yml`, but
release.yml only declared push/workflow_dispatch triggers — a reusable-workflow
reference to a workflow without `on: workflow_call:` makes the CALLER fail at
startup ('workflow file issue'), which is why every sync-all run was red at 0s.
Add a bare workflow_call trigger (tag-push + dispatch paths unchanged; the
release job is tags-guarded and sync-all runs on branch pushes, so the callable
path is not exercised — this only restores a valid reference).
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
The board ran a hand-created unhardened ROOT sbxwaf on :8085 while the package
shipped only a worker@ fan-out that crash-looped (panic on the root-only
cookie-audit log) and, as secubox-waf + RuntimeDirectory=secubox, re-chowned the
shared /run/secubox on every crash-restart — breaking every secubox-user socket
bind (profiles 502s). Ship ONE hardened unit (secubox-waf-ng.service, :8085,
User=secubox-waf, full sandbox, NO RuntimeDirectory — sbxwaf only connects to
waker.sock). postinst asserts the perms a non-root sbxwaf needs
(haproxy-routes.json 0644, cookie-audit ledger writable by secubox-waf),
disables leftover worker@ units, drops the /etc override. Validated live on gk2:
264 routes, real routing, admin/gitea/billets/yacy 200; a WAF restart no longer
touches /run/secubox and profiles stays up.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
The postinst enabled+started the sleeper, which stops idle sleepable modules —
and health-sync currently lists crowdsec among sleepable. Ship it disabled
(--no-enable --no-start; postinst try-restart only, never enable). Operator
opts in after auditing the sleepable set. Waker stays enabled (never stops).
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Sleep-when-idle + wake-on-access for public on-demand services, per-module
lifecycle policy, durable route restoration, awake-level panel setter, and a
two-phase terminal wake splash (waker + nginx error_page) generalized to all
on-demand vhosts. Includes #893 profiles-actuation robustness.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Repurpose the (dead, Task-7) nginx-sync into a transactional injector that wires
the phase-2 splash into EVERY on-demand vhost, not just yacy:
- nginx/secubox-waking.conf is now fully self-contained at server level
(proxy_intercept_errors + proxy_connect_timeout 3s + error_page → splash), so
per-vhost wiring collapses to a single `include snippets/secubox-waking.conf;`.
- nginxgen.py rewritten: find_config (by server_name), wire/unwire (idempotent,
marker-based, insert after the domain's server_name line — right block in
multi-server files), and sync_and_reload (wire every on-demand vhost via
wafsync.ondemand_vhosts, nginx -t, rollback from in-memory originals on
failure, reload on success — NO .bak in the nginx dir).
- `secubox-wakectl nginx-sync` (root) runs it; `set-lifecycle` runs it after the
waf/health resync (best-effort — opting a service on-demand auto-wires its
splash); postinst runs it so every install/upgrade re-asserts the wiring.
Live on gk2: one nginx-sync wired 5 on-demand vhosts (yacy/gitea/nextcloud/
podcaster/billets), reported streamlit as having no nginx config, idempotent on
re-run; yacy down → nginx serves the 503 splash in 3.0s. 286 tests pass.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Two-phase graceful wake UX, one shared splash page:
- templates/waking.html reworked into a pseudo-terminal "virtual screen": a CRT
panel with scanlines, an auto-executing boot log, blinking cursor, elapsed
counter, and a "taking longer / repair" panel after ~90s. Fully self-contained,
JS-driven (service name from the vhost hostname, elapsed from sessionStorage),
auto-refreshing — so it needs no server-side substitution and works verbatim
for BOTH phases.
- waker (phase 1, container down / route absent): serves the page verbatim.
- nginx (phase 2, container up but app still booting → backend 502/503/504):
new nginx/secubox-waking.conf snippet intercepts the gateway error and serves
the same splash (503 + Retry-After) instead of a raw 502. Wire per on-demand
vhost with `include snippets/secubox-waking.conf;` + `proxy_intercept_errors on;`
+ `proxy_connect_timeout 3s;` (fail fast so the splash beats HAProxy/sbxwaf).
- debian/install ships the page to /usr/share/secubox/www/waking/ and the snippet
to /etc/nginx/snippets/.
Proven live on yacy: full path (HAProxy→sbxwaf→nginx) serves the terminal splash
during the down/boot window, then the real app once it answers. 279 tests pass.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Review follow-ups for the awake-level setter:
- reject a protected module server-side (web 409) and in the ctl (refused,
reason=protected) — it is forced always-on anyway, so the write is spurious
and would dirty a core manifest; mirrors set_pin's protected refusal.
- fsync manifest_edit's atomic write before rename (the manifest is source of
truth, not a regenerable derived file) — parity with web._atomic_write.
- tests: protected refusal (web + cli), duplicate-same-key collapse, manifest
with no trailing newline.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
The scale-to-zero policy (lifecycle always-on/eager/on-demand/manual +
wake_class normal/urgent) was manifest-only. Add a per-module selector to the
/profiles/ panel that writes the manifest and resyncs the derived files, via the
webui->ctl pattern:
- api/manifest_edit.py: targeted line-edit of a manifest's lifecycle/wake_class,
preserving everything else (comments, inline portal table, …); refuses a
sectioned manifest; atomic write.
- api/cli.py: `secubox-profilectl set-lifecycle <mod> --lifecycle X --wake-class Y`
(root) — edits the manifest then re-runs waf-sync + health-sync so the change
takes effect; resync paths derive from --root (test-isolated). argparse choices=
hard-gate the enums; refuses an unknown module.
- api/web.py: POST /api/v1/profiles/lifecycle — validates enum (422) + known
module (404) BEFORE any sudo, then delegates to the root ctl via systemd-run.
- sudoers.d: one scoped grant (three bounded wildcards, panel pre-validates +
ctl re-validates — execve, no shell).
- www/profiles/index.html: two compact <select>s per module (protected → 🔒),
POSTing /lifecycle and refreshing.
16 new tests (manifest_edit, cli set-lifecycle, web route); 275 pass, no
live-state leakage, sudoers + JS validated.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
The scale-to-zero pilot proved the wake-trigger chain but found the round-trip
broken: a woken portal module started (container up, backend serving) yet stayed
unreachable via its vhost because its sbxwaf route was never re-added. wake.py
passed routes={} to apply_plan, and by wake time the route is long gone from the
live haproxy-routes.json (removed at sleep) and out of the rotated 4R snapshot.
Fix — a durable per-domain route memory (portal_routes.remember/recall,
/var/lib/secubox/profiles/portal-routes.json): the actuator persists the live
route value just before removing it on STOP; wake recalls it and passes it as the
routes map, so the existing snapshot->START->_portal_add path restores it. No
change to snapshot/apply/_portal_add semantics. Best-effort memory (a failed
write never breaks a STOP). remember_path_for(root) mirrors snap_root_for for
test isolation.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
On a 184-module fleet, DEFAULT_LIFECYCLE="eager" made every module with
no manifest opinion idle-sleep-eligible, including core services (admin,
gitea, nextcloud) that never opted into scale-to-zero. Flip the default
to always-on: sleep is now a strict opt-in via lifecycle="eager" or
"on-demand" declared explicitly in the manifest. scan()-derived
manifests already relied on this same default (no code change needed
there), so they inherit the safer behavior too.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Three findings from the pre-pilot review:
- waf-sync/health-sync _write_atomic left the produced file 0600 root-owned
(tempfile.mkstemp default) with no chmod before the rename. sbxwaf (user
secubox-waf) and secubox-hub (user secubox) could never read their own
public on-demand-vhosts.json / sleepable-modules.json — the wake trigger
and health-distinction were silently inert on install. Chmod 0644 before
os.replace in both helpers.
- The two sbxwaf workers (@1, @2) each hold independent in-memory
Begin/End vhost state but flushed to the SAME --vhost-signals path,
last-writer-wins clobbering each other every ~5s — a service kept fresh
by one worker could be read as stale by the sleeper and wrongly STOPped.
Unit now passes a per-instance path (vhost-signals.@%i.json, no Go
change needed); sleeper_daemon._signal_reader globs and merges per-vhost
(max last_request_ts, sum active_conns) across workers, falling back to
the legacy plain path when no per-worker files exist yet.
- manifest.load_manifest stored portal_domain verbatim; the rest of the
front pipeline (waker exact-match, sleeper keys, wafsync output) is
lowercase, so a hand-edited mixed-case domain would silently never wake
nor sleep. Lowercase at load time.
Full profiles suite: 251 passed. mypy --strict api/: unchanged 102
pre-existing errors (diffed against HEAD, only reordering, zero new).
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
secubox-sleeper.service actuates in-process as root (no sudo/systemd-run,
unlike the panel/waker which escape their own sandbox before touching
systemd/LXC). ProtectSystem=strict made everything under /run read-only
outside the listed ReadWritePaths, but lxc-start/lxc-stop write
LXC-internal runtime/lock/cgroup paths (/run/lxc/, /run/lock/lxc/, mount-
namespace setup) that are not part of this codebase's stable contract to
enumerate. api/actuate.py::_issue only hard-fails on rc is None, so the
resulting EROFS would have been silent: auto-sleep for every LXC-backed
on-demand module would quietly stop working.
Drop ProtectSystem=strict and ReadWritePaths=; keep ProtectHome=true,
PrivateTmp=true, NoNewPrivileges=true, and RuntimeDirectory=secubox. A
root daemon driving LXC in-process without ProtectSystem is at this
codebase's normal floor, not a regression.
debian/postinst: keep pre-creating /var/log/secubox and
/var/lib/secubox/profiles/rollback (still needed by the actuator
regardless of sandboxing), reword the comment to drop the
ProtectSystem=strict framing.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Wires the scale-to-zero services into debian/ packaging: secubox-wakectl
entry point (/usr/sbin), templates/waking.html now ships, secubox-waker
and secubox-sleeper systemd units registered via dh_installsystemd
--name=, enabled/restarted (try-restart, preserving runtime state) in
postinst alongside secubox-profiles.service. postinst now also runs
secubox-wakectl waf-sync/health-sync so sbxwaf and the health monitor
have their lists from first install (nginx-sync stays unwired per the
2026-07-20 pivot).
Hardens secubox-sleeper.service with ProtectSystem=strict and an
explicit ReadWritePaths covering the actuator's real write set, traced
through api/actuate.py/snapshot.py/audit.py: /run/secubox,
/var/lib/secubox/profiles/rollback, /var/log/secubox, /data/lxc, and
/etc/secubox/waf (haproxy-routes.json, written on a routed module's
STOP — missed by a naive reading of the write set).
Documents the lifecycle/wake_class policy, the waker/sleeper mechanism
and a pilot procedure in README.md, wiki/Architecture.md and
.claude/MODULE-COMPLIANCE.md. Bumps changelog to 0.8.0.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
boot_should_start(m) is the pure boot policy (always-on/eager start
immediately, on-demand/manual wait for a real wake) and
watchdog_should_manage(m) documents/tests that no sleepable module is ever
force-revived by secubox-watchdog.
Investigation found: HEAD's secubox-watchdog has no auto-revive logic at
all today (monitor_loop only logs up/down, restarts are manual-only); the
unmerged feat/watchdog-auto-revive branch adds one gated purely on
lxc.start.auto, which actuate.py::runtime_stop already clears before
lxc-stop specifically to avoid this race — the same flag streamlit sleepers
rely on for exclusion today. No separate exclusion file exists to reuse, so
none was invented; once that branch merges it is already #896-safe with
zero further wiring.
The real, closeable gap was secubox-hub's own sidebar health-batch, which
reported a sleeping on-demand module's inactive systemd unit as "warn".
Added a profiles-side export (secubox-wakectl health-sync ->
sleepable-modules.json, mirrors waf-sync/nginx-sync) and wired it into
secubox-hub so a sleepable module shows "Asleep (on-demand)" instead of an
alarm; a failed unit still alarms regardless. The full admin /health/ page
reads module_prober.py/prober.py, which aren't sourced in this repo yet
(TODO #393) -- documented as a precise cross-package follow-up.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Status payload now surfaces effective lifecycle, wake_class and a derived
sleep_state (up/asleep/n-a) + wake budget per module. Adds POST /wake and
POST /sleep, webui->ctl (JWT + _apply_lock + fixed sudo argv), refusing
unknown/non-sleepable modules locally before any sudo call. Two new bounded
sudoers grants: synchronous wakectl wake (distinct from the waker's
fire-and-forget grant) and profilectl apply --only. Panel gains a
sleep-state pill + manual Sleep/Wake buttons per module row.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
The Begin/End hook was placed before the WAF-inspection block, so a
request the WAF blocks (403 warning/ban) still refreshed last_request_ts —
scanner/bot traffic against public on-demand vhosts (near-constant
internet-wide scanning) kept the signal "fresh" forever, so
should_sleep()'s idle-age check never passed. Defeated auto-sleep for the
feature's primary deployment (public on-demand vhosts).
Move the Begin/defer End bracket to right after the WAF-inspection block's
403/warning/ban early-returns, immediately before the media-cache-hit
check. One placement still covers three real-response exit paths via the
single defer: the media-cache hit (counts — genuine content served),
media-cache-miss proxy, and the plain proxy. Blocked, banned, waker, and
421 requests never reach this line.
Adds TestVhostSignalsExcludedForWAFBlock (verified RED against the old
placement, GREEN after the move).
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
_signal_reader was a documented stub returning {} (auto-sleep never fired).
It now reads sbxwaf's vhost-signals.json (best-effort: missing/unreadable/
corrupt => {}, same contract as sleeper._read_wake_locked, never raises).
Critical fix: sbxwaf writes last_request_ts as unix wall-clock seconds
(time.Now().Unix()), and front_signals.vhost_signals(reader, now) computes
age = now() - last_request_ts, so `now` must be wall-clock too. main_async
was wiring time.monotonic (arbitrary epoch, unrelated to unix time) into
that slot; flipped to time.time. serve()'s `now` param has exactly one
consumer (vhost_signals) so this is an isolated fix — the separate `stamp`
param (audit timestamp string) is untouched.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
sbxwaf tracked cumulative visit counts but nothing usable to decide vhost
idleness. Add VhostSignals (mirrors visitstats.go's lock+flusher+atomic-
rename shape): Begin/End bracket every request actually proxied to a real
backend for on-demand vhosts only, a 5s-ticker flusher writes
{"<vhost>": {"last_request_ts", "active_conns"}} atomically to
--vhost-signals (default /var/cache/secubox/waf/vhost-signals.json). The
waker-splash branch is never bracketed — a sleeping vhost must not look
like it just received a real hit. Wired into the worker unit's ExecStart.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Adds the systemd/sudoers scaffolding for scale-to-zero's two new services:
secubox-waker.service (User=secubox, ProtectSystem=strict, delegates the
privileged wake to root via sudo->systemd-run->secubox-wakectl, same EROFS
lesson as secubox-profilectl) and secubox-sleeper.service (root, long-running
idle daemon that drives apply.apply_plan directly, no sandbox needed since
it is itself the actuator).
_fire_wake now wraps its sudo call in systemd-run (fire-and-forget) so the
wake escapes the waker's read-only sandbox; the matching sudoers grant is
added to sudoers.d/secubox-profiles. The waker also persists its in-memory
_last_wake set to /run/secubox/waker-active.json (TDD'd), the only channel
the sleeper's wake-lock reads to avoid racing a module it just woke.
api/sleeper_daemon.py wires api.sleeper.serve()'s production dependencies;
the front-signal reader is a documented safe stub ({}) pending a real
sbxwaf-stats source.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
The waker Director rebuilt the wake path from the raw req.Host without
lowercasing, while OnDemand.Contains matches case-insensitively. A
mixed-case Host (hand-typed URL, script caller) would pass the
on-demand gate but the waker's exact-match lookup against the
lowercase-stored portal_domain would miss, leaving the service
permanently unwoken behind the splash. Normalize the same way
(lowercase + trim) in the Director before building /_wake/<host>.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Add OnDemand (hot-reloadable set loaded from --on-demand-vhosts /
/etc/secubox/waf/on-demand-vhosts.json) and a cached unix-socket
reverse proxy to the waker. When a request hits a vhost with no live
route but present in the on-demand set (the sleeper stopped it), sbxwaf
now reverse-proxies to /run/secubox/waker.sock (path rewritten to
/_wake/<host>) instead of answering 421 — the vhost is real, just
asleep. Vhosts outside the on-demand set are unaffected. The waker
splash is excluded from the visit-stats legitimate-traffic tally, same
as the 403/421 it stands in for.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Adds run_once alongside Task 4's should_sleep: one daemon pass over the
sleepable modules, STOP-planning and apply_plan-ing each one that's up,
idle, and not wake-locked. Factors the snap_root/audit_path production-vs-
test-confinement logic (already established by wake.py in Task 2) into a
shared api/actuate_paths.py so wake and sleep never diverge on which 4R
chain/audit log they write to.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
rollback_to's own default wait_timeout=30.0 re-capped the standalone/panel
rollback path (cli.py's rollback --yes calls rollback_to without passing
wait_timeout), defeating the derived-timeout wiring on that path. Also adds
an integration test that only passes because the derived timeout (105s)
exceeds a flat 30s, using apply_plan's injected clock/sleep — the previous
happy-path test converged on the first probe and would have passed even
without the wiring.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
