Add packages/secubox-billets/api/routes/jwt_admin.py — a JWT-authed JSON
admin surface (secubox_core.auth.require_jwt: Bearer OR secubox_session
cookie) that reuses the same repo/media layer as the session HTML admin.
Endpoints: POST/PUT/DELETE /admin/api/billets, POST
/admin/api/billets/{id}/media (multipart, EXIF-strip via services.media),
GET /admin/api/comments, approve/delete comment moderation. No-op if
secubox_core is absent (isolated unit tests, session admin unaffected).
Wire it into api/main.py (register_jwt_admin after register_admin).
postinst: drop a sorted-last zz-secubox-system.pth into the venv so the
isolated billets venv can import the .deb-managed secubox_core without
shadowing its own fastapi/pydantic wheels.
Companion billets module: point EP map at /feed.json + /admin/api/*, add
an image file input to the editor and upload it after the billet is
created (POST /admin/api/billets/{id}/media, multipart). Fixes the "New
billet missing upload" gap. Bump service worker v6 -> v7.
Verified live on gk2: create -> feed, valid PNG upload -> {url,thumb},
delete — all 200.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
The billets module called `/feed` (404) — billets serves a JSON Feed at
/feed.json — and read `published_at` where JSON Feed uses `date_published`;
status_endpoint was /health (billets serves /healthz). Fixed all three, so the
#/m/billets list loads (30 items) instead of {"detail":"Not Found"}. Writes
(new billet, comment moderation) remain unavailable — billets exposes only GET
routes (its admin is the session-based web UI); those tabs already degrade
gracefully. A JWT write API for billets is a separate feature.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
The API runs unprivileged (secubox) but probed the container via lxc-info/occ
WITHOUT sudo and with lxc_path=/srv/lxc (container is /data/lxc) → status always
'Stopped', cloud.local domain, empty version/users/disk (and a 500 once the path
was corrected, from a PermissionError on /data/lxc). Now: privilege-free TCP port
probe for liveness, all container ops via 'sudo nextcloudctl' (the sole sudoers
surface, incl. its occ passthrough), a daemon-thread 60s cache for the slow occ
fields (version/users/storage) so /status stays fast, real data_path, and a
public web_url from the domain. Webui restyled to the /certs/ cyan hybrid-skin
(361 lines, emoji cards, live pulse, 30s refresh); every endpoint preserved.
Needs (board drift): NoNewPrivileges=false drop-in (sudo was blocked) and
[nextcloud] domain=nc.gk2.secubox.in in secubox.conf.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
The CrowdSec bridge was silently OFF: the unit passes --crowdsec-url but no
--crowdsec-jwt-file, so the wiring hit the 'disabled' branch and srv.crowdsec
stayed nil → Report() was never called. 198K threats detected, ~0 auto-bans
enforced (the 36k-hit offender was never nft-dropped). The LAPI /v1/alerts path
was doubly broken anyway: JWTs expire hourly and our alert schema 500s on this
build.
Add CscliReporter — the proven path the dashboard's manual ban uses
('cscli decisions add', which creates a real bouncer-enforced nft drop). It
engages when --crowdsec-url is set without a JWT file (new --crowdsec-cscli
flag, default 'cscli'). Per-IP 5-min dedup collapses the storm from the
graduated ban firing Report on every banned request (a rapid attacker otherwise
spawned dozens of concurrent cscli procs that contended until killed); on cscli
failure the IP is un-recorded so the next hit retries. Verified live: 5 honeypot
hits from a test IP → exactly 1 decision, no storm.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
The endpoint shelled to 'mailctl user passwd', which writes a host-side copy the
LXC never reads, so admin password resets silently didn't take (login kept
failing). Now generates the SHA512-CRYPT hash via the container's doveadm and
updates DATA_PATH/config/users directly (the bind-mounted file dovecot reads),
normalising any legacy 2-field line to the full 8-field record.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
crt.sh is frequently down (502/404 HTML) and its raw error leaked into scan
results. Now queries certSpotter first (reliable JSON), falls back to crt.sh,
then a clean message. whois now resolves the registrable domain for sub-domains.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
/exfil read the idle live-window state.json (empty when the wg-toolbox tunnel is
idle) so the dashboard showed no devices; now serves the cumulative rollup (real
devices) + overlays live active_flows + an engine-liveness block. Service control
repointed from dormant netifyd to the real Go collector (secubox-dpi-flowcap) via
a scoped sudoers grant (postinst). Webui regenerated in the cyan hybrid-skin look,
framed on the R3 engine, netifyd/mirred/block-rules cruft removed; the XSS-safe
media-buffer rendering is preserved verbatim.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Status/components/ports now use direct TCP probes instead of lxc-info (which
read the wrong lxc_path and reported everything Stopped); storage reads vmail.
Webui rebuilt in the /certs/ hybrid-skin look: emoji stat-cards, 15s live
pulse, External-mailboxes tab, scanline engine dropped; all endpoints preserved.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
A present-but-stale localStorage Bearer was used exclusively, ignoring a valid
secubox_session cookie, so every webui 401'd into a /login.html loop. Now both
sources are tried (Bearer first, then cookie); 401 only if neither validates.
Strictly more permissive — no regression for working Bearer clients.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
The stale-while-revalidate SW served cached core/*.js first, so every fix needed
2+ reloads (and left the old registry.js resolving module imports to /core/…).
Switch to network-first (fresh when online, cache only offline). SW→v5.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Root cause of the module 401s: the companion proxied /api/v1/auth to the
AGGREGATOR, whose minted token's jti is never written to sessions.json, so
require_jwt rejected it. Route /api/v1/auth to auth.sock (canonical, session-
recorded). Implement the real two-step login like login.html: password →
access_token, or mfa_required → prompt TOTP code → /auth/login/mfa → access_token.
Capture the token as Bearer (+ the SSO cookie login sets). Ship the nginx vhost
in deploy/. SW→v4.
Proven end-to-end (temp viewer user): login returns access_token + sets
secubox_session; podcaster/wireguard/system → 200 via both Bearer and cookie.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Dynamic import() resolves relative to registry.js (/core/), so import('./modules/x/view.js')
hit /core/modules/… → 404 → index.html fallback (HTML) → browser rejected the module
('not a valid JavaScript MIME'). Make m._path an absolute URL via new URL(path, document.baseURI).
SW→v3 to ship past the cached shell.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
secubox-auth login is cookie-based (#400 SSO-lite: require_jwt accepts the
parent-domain secubox_session cookie OR a Bearer token). Switch the client to
credentials:'include' (send the cookie) and stop treating 'no token in body' as
an error; keep an optional bearer if a build returns one. Pre-fill the box URL
with the same-origin (the companion vhost proxies /api/v1 to the box, so no
CORS). Bump SW cache to v2 to ship the fix past the cached shell.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Proves the drop-a-folder flow (core untouched) and gives the dashboard one card
per canonical group. Reads use known box routes; a few write/detail routes are
marked TODO(api).
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
The /certificates handler shelled out to openssl once per PEM (84 blocking
subprocesses) inside an async handler on the shared aggregator loop → froze the
board when the certs tab was viewed. Now: parse in-process via cryptography
(84 certs in 0.1s vs 12s), run via asyncio.to_thread, and cache 60s. Reads the
real HAProxy PEM store (/data/haproxy/certs), matching the certs module.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
The cert list + cert_count read the legacy acme.sh /etc/acme layout (empty on
this platform) → 0 certs shown. Point them at the combined HAProxy PEM store
(/data/haproxy/certs/*.pem) — the same source the certs module uses. Wildcard
filenames (_wildcard_.x.pem) map back to *.x.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Productises the one-off importer into the module:
- api/importer.py: yt-dlp URL importer (playlist or single video, or any yt-dlp
site). Per item: download video<=720p + extract audio, optional PeerTube upload
(unlisted, creds from root-only /etc/secubox/secrets/peertube-import.json),
podcaster episode (local audio, done), optional published billet (description +
source + both stream links). Off-loop via asyncio.to_thread; single job with
live JOB progress. Optional yt-cookies.txt to defeat YouTube's bot throttle;
yt-dlp errors surfaced in the progress log.
- endpoints: POST /import/url (guarded), GET /import/status.
- webui: '➕ Add from URL' button + modal (URL + PeerTube/billets toggles) +
live progress (current, done/total, bar, errors, log tail).
- Recommends: yt-dlp, ffmpeg.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Detect podcasts.apple.com / itunes.apple.com links, extract the numeric id, and
resolve the underlying feedUrl via the iTunes Lookup API before ingesting as a
normal RSS feed. Best-effort: falls back to the original URL on any error.
Declare python3-mutagen + python3-pil (used by the #855 cover extractor).
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
- backend: _extract_cover pulls ID3 APIC / MP4 covr / FLAC picture from the first
audiobook track via mutagen, downscales with Pillow, stores <feed>/cover.jpg,
sets feed image to /feeds/<fid>/cover; new public cover endpoint. Best-effort
(never breaks an upload).
- webui: cover thumbnails on episode cards/rows with graceful emoji fallback on
missing/404; portal gets a 'Now Playing' cover bar driven by a capture-phase
play listener reading data-* (does NOT rebuild the list → playback guard intact).
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
The audiobook upload wrote its temp ZIP to a hardcoded /var/lib/secubox/podcaster
(the 15GB eMMC) regardless of where media_path points. On a box whose media
store is a large SSD, a big upload still filled the tiny eMMC and failed with a
confusing 502.
- temp ZIP is now created under MEDIA (media_path), same filesystem as the
extraction target — so it lands wherever the store lives (the SSD).
- before streaming, if Content-Length * 2 + margin exceeds the media
filesystem's free space, return a clear 413 ("needs ~X MB, only Y free")
instead of failing mid-write. (No amount of code makes a 700GB ZIP fit — but
the user gets an instant honest reason.)
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
The unit chowned the socket in an ExecStartPost gated only by `sleep 1`.
uvicorn can take several seconds to bind the UDS, so the chown ran before the
socket existed (silently, via the `-` prefix) and it stayed root:root — which
nginx (not root) cannot open, so every podcaster request 502'd after a restart.
A naive poll didn't help either: it found the STALE socket from the previous
run, chowned that, and uvicorn then re-bound a fresh root:root socket.
Fix: ExecStartPre removes the stale socket first, and ExecStartPost polls up to
~15s for the genuinely-new socket before chmod/chown. Socket is now reliably
secubox:secubox on start. Verified live: owner settles secubox:secubox and
nginx serves /api/v1/podcaster/status 200.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
30s was too short for slow module operations behind the reverse proxy — a
podcaster audiobook upload (server-side ZIP extraction of ~120MB) exceeded it,
so nginx returned 502 on an upload that actually succeeded. 300s matches the
per-vhost timeouts already used elsewhere (lyrion, yacy). Applies to every
module served through secubox.d.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Fatal messages (API/HTTP error, failed action, missing) must stay until the
user dismisses them, never a 3s flash. Codify the toast()/errorToast() split
and the "report the true server outcome" rule (a proxy timeout must not turn a
real success into a permanent-looking failure).
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
The audiobook ZIP handler extracted synchronously on the single-worker event
loop. A ~120MB/20-track audiobook took longer than nginx's 30s proxy_read_timeout,
so nginx returned 502 WHILE the extraction actually completed server-side — a
false-negative "Upload failed" on a successful upload (and it froze the service
meanwhile).
- extraction moved to asyncio.to_thread (_publish_audiobook_zip); the event
loop stays responsive and the response is sent as soon as it finishes.
- ClientDisconnect during body streaming is caught → clean 400, not a 500.
- any extraction failure now deletes the partial feed (no orphaned feed; the
prior code only cleaned up the "no audio found" case).
Webui (per operator guideline): fatal messages (API/HTTP/missing) now persist
until the user dismisses them (red toast with ✕); progress + success stay
transient. Routed all 7 error sites to errorToast.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Backport of two board-live fixes so a package reinstall can't revert them:
- billets-communique.css: the communiqué permalink extends the blog base
(billets.css flips dark under prefers-color-scheme:dark), which leaked a dark
ground into the always-light press-release. Pin color-scheme:light + html/body
light. Bump billet_communique.html css ref to ?v=2.
- health-banner.js: public pages can now opt out of the operator banner with
<meta name="sbx-no-health-banner"> (reliable — the WAF injects the loader
after the page head; and CSP-safe, unlike an inline window flag). base.html
emits it, so the public blog no longer shows the banner while admin pages
(standalone templates) keep it. Replaces a board-only hostname hardcode.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Replace `try: await cur except BaseException: pass` with `await asyncio.wait({cur})`
so awaiting the cancelled download's unwind no longer swallows cancellation of
the request handler itself (client disconnect). Pure idiom cleanup; behaviour
for the cancel-and-restart path is unchanged.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Adversarial review found a corruption ship-blocker: nothing prevented two
_download_one tasks for the same episode (manual ↻ button, auto-retry re-queue
and restart-recovery all feed the same queue). Two tasks share deterministic
tmp/dest paths → interleaved writes corrupt the audio and thrash state.
- _inflight registry keyed by episode id; the worker skips a queue entry whose
download is still running (dedup).
- manual /episodes/{id}/download now cancels any live task and awaits its
unwind before re-queuing, so ↻ genuinely restarts a wedged download instead
of colliding or being deduped away.
- _download_one treats CancelledError as a clean abort (drop .part, no attempt
charged), distinct from a real failure.
Test: worker runs one download for two same-id puts, and a cancel+re-put
starts a fresh attempt.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Downloads no longer wedge a queue slot forever:
- real httpx.Timeout (connect/read/write) so a stalled stream aborts to error
instead of hanging on the previous timeout=None
- bounded auto-retry (3 tries, 10/30/60s backoff) via a new attempts column
- restart-recovery: episodes orphaned in downloading/queued are re-queued when
the worker (re)starts
- webui: manual "↻" restart button on downloading/queued episodes; a manual
(re)download resets the retry budget
Audiobook ZIP upload now uses XHR with upload.onprogress: a persistent
"Uploading N%" indicator (was a fetch with a 3s toast that hid mid-upload),
then a persistent published/error result.
Store: idempotent attempts-column migration + requeue_stuck + bump_attempts,
with tests.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Review HIGH: the headless-browser snapshot path drove Chromium to embed_url with
no SSRF check (only the og:image fallback was guarded), so an author could
screenshot an internal service (127.0.0.1/RFC1918/::1/169.254.169.254). Now
ssrf.validate_url() gates the navigation (reject → fall back to og:image), plus a
Playwright route aborts any redirect/sub-resource to a non-public host. 12 tests.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
