Authors already typed #hashtags in their bodies (37 of the 46 billets on gk2
carry one), so tags are EXTRACTED from the body rather than added via a picker:
tagging costs the author nothing and applies retroactively.
- migration 0004: tag(slug,emoji,label) + billet_tag, indexed on tag_slug.
Extracted into rows rather than re-parsed per request, so filtering hits an
index instead of LIKE-scanning every body. slug is accent-folded + lowercased
(#Réseau == #reseau); emoji is STORED so a later curation change never
silently restyles an already-published billet.
- services/tags.py: curated map + extraction. The lookbehind keeps a URL anchor
(page#anchor) from becoming a tag; unknown tags fall back to a neutral badge
rather than being rejected. Map covers this box's real content (podcast/media)
as well as the security/box vocabulary; genuinely ambiguous topics are left on
the default rather than editorialised.
- quick views: ?tag=slug on / and /feed.json, plus /tags.json and
/admin/api/tags for the chip bar. Uses EXISTS, not a JOIN, so keyset paging
cannot duplicate a row per matching tag; the pager carries the active tag.
- feed resumes: long billets show a short excerpt + "Lire la suite"; short ones
render whole. `is_long` is measured on the markdown-collapsed text, so a short
billet padded with link syntax is not "resumed" into a copy of itself.
- feed.json gains `summary` + `tags` (both JSON Feed 1.1 standard) with the
emoji in the spec-legal `_secubox` extension; optional keys are omitted, not
nulled.
- GET /admin/api/billets: the authoring list. The public feed cannot serve it —
its item id is a permalink URL (so edit/delete could not address a row, which
is why they 404'd) and it hides drafts, which an authoring client must see.
- manage.py backfill-tags: idempotent; also re-applies the curated map to
existing tags as an explicit operator step.
Verified on gk2: backfill tagged 37/46 billets; ?tag=podcast→37, jfk→4, nope→0;
a draft's tag correctly stays out of the public chip bar; delete cascades.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
/login/mfa and /totp/confirm both hardcoded "ip": "" and omitted user_agent
entirely, while only the password path captured them. admin is forced-TOTP,
so EVERY real admin login took the MFA path — every session row landed
unauditable (who logged in, from where, with what), and the users webui
sessions tab rendered blanks for data it was already asking for.
Factor the extraction into _client_meta(request) (X-Forwarded-For first: nginx
and HAProxy front every login, so request.client.host is only ever the proxy)
and use it on all three paths.
Verified by driving the real /login/mfa handler in-process with
SECUBOX_AUTH_SESSIONS pointed at a temp file: the session row now records
ip=192.168.1.77 (the first XFF hop, not the proxy) and the full user-agent.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
A plain fetch(req) in the network-first handler still consults the browser
HTTP cache, which heuristically caches module JS (no Cache-Control on .js).
That let an old view.js survive a deploy — the recurring 'still broken after
hard refresh'. Refetch with cache:'reload' so every online load pulls fresh
app code; offline still falls back to the Cache API. Bump v7 -> v8.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Add packages/secubox-billets/api/routes/jwt_admin.py — a JWT-authed JSON
admin surface (secubox_core.auth.require_jwt: Bearer OR secubox_session
cookie) that reuses the same repo/media layer as the session HTML admin.
Endpoints: POST/PUT/DELETE /admin/api/billets, POST
/admin/api/billets/{id}/media (multipart, EXIF-strip via services.media),
GET /admin/api/comments, approve/delete comment moderation. No-op if
secubox_core is absent (isolated unit tests, session admin unaffected).
Wire it into api/main.py (register_jwt_admin after register_admin).
postinst: drop a sorted-last zz-secubox-system.pth into the venv so the
isolated billets venv can import the .deb-managed secubox_core without
shadowing its own fastapi/pydantic wheels.
Companion billets module: point EP map at /feed.json + /admin/api/*, add
an image file input to the editor and upload it after the billet is
created (POST /admin/api/billets/{id}/media, multipart). Fixes the "New
billet missing upload" gap. Bump service worker v6 -> v7.
Verified live on gk2: create -> feed, valid PNG upload -> {url,thumb},
delete — all 200.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
The billets module called `/feed` (404) — billets serves a JSON Feed at
/feed.json — and read `published_at` where JSON Feed uses `date_published`;
status_endpoint was /health (billets serves /healthz). Fixed all three, so the
#/m/billets list loads (30 items) instead of {"detail":"Not Found"}. Writes
(new billet, comment moderation) remain unavailable — billets exposes only GET
routes (its admin is the session-based web UI); those tabs already degrade
gracefully. A JWT write API for billets is a separate feature.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
The API runs unprivileged (secubox) but probed the container via lxc-info/occ
WITHOUT sudo and with lxc_path=/srv/lxc (container is /data/lxc) → status always
'Stopped', cloud.local domain, empty version/users/disk (and a 500 once the path
was corrected, from a PermissionError on /data/lxc). Now: privilege-free TCP port
probe for liveness, all container ops via 'sudo nextcloudctl' (the sole sudoers
surface, incl. its occ passthrough), a daemon-thread 60s cache for the slow occ
fields (version/users/storage) so /status stays fast, real data_path, and a
public web_url from the domain. Webui restyled to the /certs/ cyan hybrid-skin
(361 lines, emoji cards, live pulse, 30s refresh); every endpoint preserved.
Needs (board drift): NoNewPrivileges=false drop-in (sudo was blocked) and
[nextcloud] domain=nc.gk2.secubox.in in secubox.conf.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
The CrowdSec bridge was silently OFF: the unit passes --crowdsec-url but no
--crowdsec-jwt-file, so the wiring hit the 'disabled' branch and srv.crowdsec
stayed nil → Report() was never called. 198K threats detected, ~0 auto-bans
enforced (the 36k-hit offender was never nft-dropped). The LAPI /v1/alerts path
was doubly broken anyway: JWTs expire hourly and our alert schema 500s on this
build.
Add CscliReporter — the proven path the dashboard's manual ban uses
('cscli decisions add', which creates a real bouncer-enforced nft drop). It
engages when --crowdsec-url is set without a JWT file (new --crowdsec-cscli
flag, default 'cscli'). Per-IP 5-min dedup collapses the storm from the
graduated ban firing Report on every banned request (a rapid attacker otherwise
spawned dozens of concurrent cscli procs that contended until killed); on cscli
failure the IP is un-recorded so the next hit retries. Verified live: 5 honeypot
hits from a test IP → exactly 1 decision, no storm.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
The endpoint shelled to 'mailctl user passwd', which writes a host-side copy the
LXC never reads, so admin password resets silently didn't take (login kept
failing). Now generates the SHA512-CRYPT hash via the container's doveadm and
updates DATA_PATH/config/users directly (the bind-mounted file dovecot reads),
normalising any legacy 2-field line to the full 8-field record.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
crt.sh is frequently down (502/404 HTML) and its raw error leaked into scan
results. Now queries certSpotter first (reliable JSON), falls back to crt.sh,
then a clean message. whois now resolves the registrable domain for sub-domains.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
/exfil read the idle live-window state.json (empty when the wg-toolbox tunnel is
idle) so the dashboard showed no devices; now serves the cumulative rollup (real
devices) + overlays live active_flows + an engine-liveness block. Service control
repointed from dormant netifyd to the real Go collector (secubox-dpi-flowcap) via
a scoped sudoers grant (postinst). Webui regenerated in the cyan hybrid-skin look,
framed on the R3 engine, netifyd/mirred/block-rules cruft removed; the XSS-safe
media-buffer rendering is preserved verbatim.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Status/components/ports now use direct TCP probes instead of lxc-info (which
read the wrong lxc_path and reported everything Stopped); storage reads vmail.
Webui rebuilt in the /certs/ hybrid-skin look: emoji stat-cards, 15s live
pulse, External-mailboxes tab, scanline engine dropped; all endpoints preserved.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
A present-but-stale localStorage Bearer was used exclusively, ignoring a valid
secubox_session cookie, so every webui 401'd into a /login.html loop. Now both
sources are tried (Bearer first, then cookie); 401 only if neither validates.
Strictly more permissive — no regression for working Bearer clients.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
The stale-while-revalidate SW served cached core/*.js first, so every fix needed
2+ reloads (and left the old registry.js resolving module imports to /core/…).
Switch to network-first (fresh when online, cache only offline). SW→v5.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Root cause of the module 401s: the companion proxied /api/v1/auth to the
AGGREGATOR, whose minted token's jti is never written to sessions.json, so
require_jwt rejected it. Route /api/v1/auth to auth.sock (canonical, session-
recorded). Implement the real two-step login like login.html: password →
access_token, or mfa_required → prompt TOTP code → /auth/login/mfa → access_token.
Capture the token as Bearer (+ the SSO cookie login sets). Ship the nginx vhost
in deploy/. SW→v4.
Proven end-to-end (temp viewer user): login returns access_token + sets
secubox_session; podcaster/wireguard/system → 200 via both Bearer and cookie.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Dynamic import() resolves relative to registry.js (/core/), so import('./modules/x/view.js')
hit /core/modules/… → 404 → index.html fallback (HTML) → browser rejected the module
('not a valid JavaScript MIME'). Make m._path an absolute URL via new URL(path, document.baseURI).
SW→v3 to ship past the cached shell.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
secubox-auth login is cookie-based (#400 SSO-lite: require_jwt accepts the
parent-domain secubox_session cookie OR a Bearer token). Switch the client to
credentials:'include' (send the cookie) and stop treating 'no token in body' as
an error; keep an optional bearer if a build returns one. Pre-fill the box URL
with the same-origin (the companion vhost proxies /api/v1 to the box, so no
CORS). Bump SW cache to v2 to ship the fix past the cached shell.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Proves the drop-a-folder flow (core untouched) and gives the dashboard one card
per canonical group. Reads use known box routes; a few write/detail routes are
marked TODO(api).
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
The /certificates handler shelled out to openssl once per PEM (84 blocking
subprocesses) inside an async handler on the shared aggregator loop → froze the
board when the certs tab was viewed. Now: parse in-process via cryptography
(84 certs in 0.1s vs 12s), run via asyncio.to_thread, and cache 60s. Reads the
real HAProxy PEM store (/data/haproxy/certs), matching the certs module.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
The cert list + cert_count read the legacy acme.sh /etc/acme layout (empty on
this platform) → 0 certs shown. Point them at the combined HAProxy PEM store
(/data/haproxy/certs/*.pem) — the same source the certs module uses. Wildcard
filenames (_wildcard_.x.pem) map back to *.x.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Productises the one-off importer into the module:
- api/importer.py: yt-dlp URL importer (playlist or single video, or any yt-dlp
site). Per item: download video<=720p + extract audio, optional PeerTube upload
(unlisted, creds from root-only /etc/secubox/secrets/peertube-import.json),
podcaster episode (local audio, done), optional published billet (description +
source + both stream links). Off-loop via asyncio.to_thread; single job with
live JOB progress. Optional yt-cookies.txt to defeat YouTube's bot throttle;
yt-dlp errors surfaced in the progress log.
- endpoints: POST /import/url (guarded), GET /import/status.
- webui: '➕ Add from URL' button + modal (URL + PeerTube/billets toggles) +
live progress (current, done/total, bar, errors, log tail).
- Recommends: yt-dlp, ffmpeg.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Detect podcasts.apple.com / itunes.apple.com links, extract the numeric id, and
resolve the underlying feedUrl via the iTunes Lookup API before ingesting as a
normal RSS feed. Best-effort: falls back to the original URL on any error.
Declare python3-mutagen + python3-pil (used by the #855 cover extractor).
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
- backend: _extract_cover pulls ID3 APIC / MP4 covr / FLAC picture from the first
audiobook track via mutagen, downscales with Pillow, stores <feed>/cover.jpg,
sets feed image to /feeds/<fid>/cover; new public cover endpoint. Best-effort
(never breaks an upload).
- webui: cover thumbnails on episode cards/rows with graceful emoji fallback on
missing/404; portal gets a 'Now Playing' cover bar driven by a capture-phase
play listener reading data-* (does NOT rebuild the list → playback guard intact).
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
The audiobook upload wrote its temp ZIP to a hardcoded /var/lib/secubox/podcaster
(the 15GB eMMC) regardless of where media_path points. On a box whose media
store is a large SSD, a big upload still filled the tiny eMMC and failed with a
confusing 502.
- temp ZIP is now created under MEDIA (media_path), same filesystem as the
extraction target — so it lands wherever the store lives (the SSD).
- before streaming, if Content-Length * 2 + margin exceeds the media
filesystem's free space, return a clear 413 ("needs ~X MB, only Y free")
instead of failing mid-write. (No amount of code makes a 700GB ZIP fit — but
the user gets an instant honest reason.)
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
The unit chowned the socket in an ExecStartPost gated only by `sleep 1`.
uvicorn can take several seconds to bind the UDS, so the chown ran before the
socket existed (silently, via the `-` prefix) and it stayed root:root — which
nginx (not root) cannot open, so every podcaster request 502'd after a restart.
A naive poll didn't help either: it found the STALE socket from the previous
run, chowned that, and uvicorn then re-bound a fresh root:root socket.
Fix: ExecStartPre removes the stale socket first, and ExecStartPost polls up to
~15s for the genuinely-new socket before chmod/chown. Socket is now reliably
secubox:secubox on start. Verified live: owner settles secubox:secubox and
nginx serves /api/v1/podcaster/status 200.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
30s was too short for slow module operations behind the reverse proxy — a
podcaster audiobook upload (server-side ZIP extraction of ~120MB) exceeded it,
so nginx returned 502 on an upload that actually succeeded. 300s matches the
per-vhost timeouts already used elsewhere (lyrion, yacy). Applies to every
module served through secubox.d.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Fatal messages (API/HTTP error, failed action, missing) must stay until the
user dismisses them, never a 3s flash. Codify the toast()/errorToast() split
and the "report the true server outcome" rule (a proxy timeout must not turn a
real success into a permanent-looking failure).
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
The audiobook ZIP handler extracted synchronously on the single-worker event
loop. A ~120MB/20-track audiobook took longer than nginx's 30s proxy_read_timeout,
so nginx returned 502 WHILE the extraction actually completed server-side — a
false-negative "Upload failed" on a successful upload (and it froze the service
meanwhile).
- extraction moved to asyncio.to_thread (_publish_audiobook_zip); the event
loop stays responsive and the response is sent as soon as it finishes.
- ClientDisconnect during body streaming is caught → clean 400, not a 500.
- any extraction failure now deletes the partial feed (no orphaned feed; the
prior code only cleaned up the "no audio found" case).
Webui (per operator guideline): fatal messages (API/HTTP/missing) now persist
until the user dismisses them (red toast with ✕); progress + success stay
transient. Routed all 7 error sites to errorToast.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Backport of two board-live fixes so a package reinstall can't revert them:
- billets-communique.css: the communiqué permalink extends the blog base
(billets.css flips dark under prefers-color-scheme:dark), which leaked a dark
ground into the always-light press-release. Pin color-scheme:light + html/body
light. Bump billet_communique.html css ref to ?v=2.
- health-banner.js: public pages can now opt out of the operator banner with
<meta name="sbx-no-health-banner"> (reliable — the WAF injects the loader
after the page head; and CSP-safe, unlike an inline window flag). base.html
emits it, so the public blog no longer shows the banner while admin pages
(standalone templates) keep it. Replaces a board-only hostname hardcode.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Replace `try: await cur except BaseException: pass` with `await asyncio.wait({cur})`
so awaiting the cancelled download's unwind no longer swallows cancellation of
the request handler itself (client disconnect). Pure idiom cleanup; behaviour
for the cancel-and-restart path is unchanged.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
Adversarial review found a corruption ship-blocker: nothing prevented two
_download_one tasks for the same episode (manual ↻ button, auto-retry re-queue
and restart-recovery all feed the same queue). Two tasks share deterministic
tmp/dest paths → interleaved writes corrupt the audio and thrash state.
- _inflight registry keyed by episode id; the worker skips a queue entry whose
download is still running (dedup).
- manual /episodes/{id}/download now cancels any live task and awaits its
unwind before re-queuing, so ↻ genuinely restarts a wedged download instead
of colliding or being deduped away.
- _download_one treats CancelledError as a clean abort (drop .part, no attempt
charged), distinct from a real failure.
Test: worker runs one download for two same-id puts, and a cancel+re-put
starts a fresh attempt.
Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
