secubox-deb/packages/secubox-billets
CyberMind-FR f9302df075 feat(billets): emoji hashtags, quick views, and feed resumes
Authors already typed #hashtags in their bodies (37 of the 46 billets on gk2
carry one), so tags are EXTRACTED from the body rather than added via a picker:
tagging costs the author nothing and applies retroactively.

- migration 0004: tag(slug,emoji,label) + billet_tag, indexed on tag_slug.
  Extracted into rows rather than re-parsed per request, so filtering hits an
  index instead of LIKE-scanning every body. slug is accent-folded + lowercased
  (#Réseau == #reseau); emoji is STORED so a later curation change never
  silently restyles an already-published billet.
- services/tags.py: curated map + extraction. The lookbehind keeps a URL anchor
  (page#anchor) from becoming a tag; unknown tags fall back to a neutral badge
  rather than being rejected. Map covers this box's real content (podcast/media)
  as well as the security/box vocabulary; genuinely ambiguous topics are left on
  the default rather than editorialised.
- quick views: ?tag=slug on / and /feed.json, plus /tags.json and
  /admin/api/tags for the chip bar. Uses EXISTS, not a JOIN, so keyset paging
  cannot duplicate a row per matching tag; the pager carries the active tag.
- feed resumes: long billets show a short excerpt + "Lire la suite"; short ones
  render whole. `is_long` is measured on the markdown-collapsed text, so a short
  billet padded with link syntax is not "resumed" into a copy of itself.
- feed.json gains `summary` + `tags` (both JSON Feed 1.1 standard) with the
  emoji in the spec-legal `_secubox` extension; optional keys are omitted, not
  nulled.
- GET /admin/api/billets: the authoring list. The public feed cannot serve it —
  its item id is a permalink URL (so edit/delete could not address a row, which
  is why they 404'd) and it hides drafts, which an authoring client must see.
- manage.py backfill-tags: idempotent; also re-applies the curated map to
  existing tags as an explicit operator step.

Verified on gk2: backfill tagged 37/46 billets; ?tag=podcast→37, jfk→4, nope→0;
a draft's tag correctly stays out of the public chip bar; delete cascades.

Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
2026-07-17 10:35:24 +02:00
..
api feat(billets): emoji hashtags, quick views, and feed resumes 2026-07-17 10:35:24 +02:00
debian feat(billets): JWT admin surface + Companion authoring & image upload 2026-07-17 07:48:12 +02:00
deploy feat(billets): image upload with zoomable vignette gallery + portable export 2026-07-12 08:11:43 +02:00
menu.d feat(billets): SecuBox webui panel + navbar integration 2026-07-11 16:19:35 +02:00
reference feat(billets): communiqué style + embed snapshot vignette + portable HTML archive (ref #851) 2026-07-14 07:57:54 +02:00
tests fix(billets): SSRF — validate embed_url public-IP before Chromium screenshot (ref #851) 2026-07-14 08:05:35 +02:00
www/billets feat(billets): complete operator panel + generate-password override 2026-07-12 08:20:02 +02:00
pyproject.toml feat(billets): communiqué style + embed snapshot vignette + portable HTML archive (ref #851) 2026-07-14 07:57:54 +02:00
README.md feat(billets): communiqué style + embed snapshot vignette + portable HTML archive (ref #851) 2026-07-14 07:57:54 +02:00
requirements.txt feat(billets): image upload with zoomable vignette gallery + portable export 2026-07-12 08:11:43 +02:00

billets — micro-blog gateway inter-médias sociaux

Self-hosted micro-blog + social-media gateway. Author-published short posts; a public read feed with comments, emoji reactions and republish share-intents. It ingests links/embeds from social networks (SSRF-guarded oEmbed, sanitized) and lets each billet be re-shared back to them.

Built as a SecuBox module (secubox-billets): FastAPI + aiosqlite (WAL) + Jinja2, served on a Unix socket behind nginx.

Architecture

api/
  asgi.py        runtime entrypoint (uvicorn api.asgi:app) — lifespan opens the DB
  main.py        app factory + public feed / permalink / feeds / oEmbed
  db.py          aiosqlite WAL + numbered .sql migrations
  models.py      Pydantic v2 (strict, size-limited, https-only URLs)
  repo.py        billet/comment/reaction/author queries (keyset pagination)
  ids.py         ULID (time-sortable)
  routes/        admin.py (auth + CRUD + moderation), public.py (react/comment)
  services/      render, sanitize, ssrf, oembed, linkcard, eventlog, revisions,
                 antispam, security, feeds
  templates/  static/  migrations/
  manage.py      CLI: create-author / set-password / seed

Invariants. Every security-relevant action (publish/edit/delete, comment moderation, login) appends to an append-only BLAKE2b hash-chained event_log (services/eventlog, verify_chain). Every billet's content and style are versioned in a Gitea repo (services/revisions, one front-matter .md per billet, full git log --follow history).

Install

Debian package

dpkg -i secubox-billets_*.deb
# postinst creates the secubox user, /var/lib/secubox/billets, the signing
# secret, the venv (pip install requirements.txt), and the nginx vhost.
cd /usr/lib/secubox/billets
sudo -u secubox venv/bin/python -m api.manage create-author admin

Set the public origin so feeds/oEmbed emit absolute URLs — add to the unit: Environment=BILLETS_SITE_URL=https://billets.example.com.

Standalone

sudo deploy/install.sh (idempotent). In SecuBox, billets is fronted by HAProxy (TLS 1.3) → sbxwaf → nginx → the socket.

Configuration (environment)

Var Default Purpose
BILLETS_DB /var/lib/secubox/billets/billets.db SQLite WAL file
BILLETS_REVISIONS_DIR /var/lib/secubox/billets/revisions Gitea revision repo
BILLETS_SECRET_FILE /etc/secubox/secrets/billets signing secret (0600 secubox)
BILLETS_SECRET secret override (takes precedence)
BILLETS_SITE_URL derived from request absolute origin for feeds/oEmbed
BILLETS_MEDIA_DIR /var/lib/secubox/billets/media re-encoded media + embed snapshots
BILLETS_SNAPSHOT_BROWSER 1 set 0 to disable the headless-Chromium capture

Communiqué style + embed snapshot

Each billet has a style: default (feed look) or communique (a poster/affiche layout — ROOT→MESH→MIND side-band, a framed "TRANSMISSION · CANAL PUBLIC" embed, stamp/kicker/slogans/footer). Pick it in the editor.

For communiqué billets with an embed, the module captures a still vignette of the embed (stored like any media file, shown in the poster frame as a click-to-load thumbnail and in the feed as a compact framed image):

  1. Headless Chromium (a real full-page screenshot) — OPTIONAL. Enable it with pip install playwright && playwright install chromium (or install the snapshot extra). Any missing package / browser / timeout degrades silently.
  2. og:image fallback — the page's og:image fetched through the same SSRF guard (https-only, must resolve to a public IP, redirects re-validated, size-capped). This is the default path when Chromium is not installed, so the feature works fully without a browser.

The capture runs OFF the shared event loop (asyncio.to_thread) and only re-captures when the embed_url changes.

GET /admin/billets/{id}/archive.html (JWT-gated) exports a portable, fully self-contained single .html of a billet in the communiqué layout: CSS inlined, system fonts (no network), media + the embed vignette inlined as data: URIs, and the embed rendered as its snapshot image linking to the original (no live iframe, so it opens offline). Button: "💾 Archive HTML".

To push revisions to Gitea, add an origin remote in the revisions repo as the secubox user (else commits stay local — still a full history).

Backup / restore

SQLite is the single source of truth. Take a consistent snapshot:

sudo -u secubox sqlite3 /var/lib/secubox/billets/billets.db ".backup '/var/backups/billets-$(date +%F).db'"

Restore: stop the service, copy the backup over billets.db (and the -wal/-shm are rebuilt), start again. The Gitea revision repo and /etc/secubox/secrets/billets should be backed up alongside.

Tests

PYTHONPATH=. python -m pytest -q     # from packages/secubox-billets

Covers data layer, feed/pagination, admin auth/CRUD, SSRF + XSS (the critical step), comments/anti-spam, reactions, feeds/oEmbed. pip-audit runs in CI.

Design decisions (documented per the brief)

  • aiosqlite (async), not sync sqlite3. As a SecuBox module, billets shares the aggregator's event loop path; a blocking DB call would stall the box. All DB access is awaited off-loop.
  • api/ layout + Unix socket, following SecuBox module conventions (rather than the standalone app/ layout).
  • Own author auth (argon2id + optional TOTP) kept per spec; SecuBox-auth SSO is a possible v2.
  • htmx replaced by a tiny vanilla shim (static/billets.js, CSP script-src 'self'). The reaction endpoint returns an htmx-compatible fragment, so dropping in a vendored htmx.min.js at deploy works unchanged — this avoids an un-fetchable CDN dependency while keeping graceful no-JS degradation.
  • Emails hashed (BLAKE2b), not encrypted — never displayed, only used to detect a returning approved commenter (spec allows "chiffré au repos ou hashé").
  • oEmbed discovery is same-origin only (Mastodon/PeerTube self-hosted); an unknown host with no same-origin oEmbed link falls back to a local OpenGraph card.