Authors already typed #hashtags in their bodies (37 of the 46 billets on gk2 carry one), so tags are EXTRACTED from the body rather than added via a picker: tagging costs the author nothing and applies retroactively. - migration 0004: tag(slug,emoji,label) + billet_tag, indexed on tag_slug. Extracted into rows rather than re-parsed per request, so filtering hits an index instead of LIKE-scanning every body. slug is accent-folded + lowercased (#Réseau == #reseau); emoji is STORED so a later curation change never silently restyles an already-published billet. - services/tags.py: curated map + extraction. The lookbehind keeps a URL anchor (page#anchor) from becoming a tag; unknown tags fall back to a neutral badge rather than being rejected. Map covers this box's real content (podcast/media) as well as the security/box vocabulary; genuinely ambiguous topics are left on the default rather than editorialised. - quick views: ?tag=slug on / and /feed.json, plus /tags.json and /admin/api/tags for the chip bar. Uses EXISTS, not a JOIN, so keyset paging cannot duplicate a row per matching tag; the pager carries the active tag. - feed resumes: long billets show a short excerpt + "Lire la suite"; short ones render whole. `is_long` is measured on the markdown-collapsed text, so a short billet padded with link syntax is not "resumed" into a copy of itself. - feed.json gains `summary` + `tags` (both JSON Feed 1.1 standard) with the emoji in the spec-legal `_secubox` extension; optional keys are omitted, not nulled. - GET /admin/api/billets: the authoring list. The public feed cannot serve it — its item id is a permalink URL (so edit/delete could not address a row, which is why they 404'd) and it hides drafts, which an authoring client must see. - manage.py backfill-tags: idempotent; also re-applies the curated map to existing tags as an explicit operator step. Verified on gk2: backfill tagged 37/46 billets; ?tag=podcast→37, jfk→4, nope→0; a draft's tag correctly stays out of the public chip bar; delete cascades. Co-Authored-By: Gerald KERMA <devel@cybermind.fr> |
||
|---|---|---|
| .. | ||
| api | ||
| debian | ||
| deploy | ||
| menu.d | ||
| reference | ||
| tests | ||
| www/billets | ||
| pyproject.toml | ||
| README.md | ||
| requirements.txt | ||
billets — micro-blog gateway inter-médias sociaux
Self-hosted micro-blog + social-media gateway. Author-published short posts; a public read feed with comments, emoji reactions and republish share-intents. It ingests links/embeds from social networks (SSRF-guarded oEmbed, sanitized) and lets each billet be re-shared back to them.
Built as a SecuBox module (secubox-billets): FastAPI + aiosqlite (WAL) +
Jinja2, served on a Unix socket behind nginx.
Architecture
api/
asgi.py runtime entrypoint (uvicorn api.asgi:app) — lifespan opens the DB
main.py app factory + public feed / permalink / feeds / oEmbed
db.py aiosqlite WAL + numbered .sql migrations
models.py Pydantic v2 (strict, size-limited, https-only URLs)
repo.py billet/comment/reaction/author queries (keyset pagination)
ids.py ULID (time-sortable)
routes/ admin.py (auth + CRUD + moderation), public.py (react/comment)
services/ render, sanitize, ssrf, oembed, linkcard, eventlog, revisions,
antispam, security, feeds
templates/ static/ migrations/
manage.py CLI: create-author / set-password / seed
Invariants. Every security-relevant action (publish/edit/delete, comment
moderation, login) appends to an append-only BLAKE2b hash-chained event_log
(services/eventlog, verify_chain). Every billet's content and style are
versioned in a Gitea repo (services/revisions, one front-matter .md per
billet, full git log --follow history).
Install
Debian package
dpkg -i secubox-billets_*.deb
# postinst creates the secubox user, /var/lib/secubox/billets, the signing
# secret, the venv (pip install requirements.txt), and the nginx vhost.
cd /usr/lib/secubox/billets
sudo -u secubox venv/bin/python -m api.manage create-author admin
Set the public origin so feeds/oEmbed emit absolute URLs — add to the unit:
Environment=BILLETS_SITE_URL=https://billets.example.com.
Standalone
sudo deploy/install.sh (idempotent). In SecuBox, billets is fronted by
HAProxy (TLS 1.3) → sbxwaf → nginx → the socket.
Configuration (environment)
| Var | Default | Purpose |
|---|---|---|
BILLETS_DB |
/var/lib/secubox/billets/billets.db |
SQLite WAL file |
BILLETS_REVISIONS_DIR |
/var/lib/secubox/billets/revisions |
Gitea revision repo |
BILLETS_SECRET_FILE |
/etc/secubox/secrets/billets |
signing secret (0600 secubox) |
BILLETS_SECRET |
— | secret override (takes precedence) |
BILLETS_SITE_URL |
derived from request | absolute origin for feeds/oEmbed |
BILLETS_MEDIA_DIR |
/var/lib/secubox/billets/media |
re-encoded media + embed snapshots |
BILLETS_SNAPSHOT_BROWSER |
1 |
set 0 to disable the headless-Chromium capture |
Communiqué style + embed snapshot
Each billet has a style: default (feed look) or communique (a
poster/affiche layout — ROOT→MESH→MIND side-band, a framed "TRANSMISSION · CANAL
PUBLIC" embed, stamp/kicker/slogans/footer). Pick it in the editor.
For communiqué billets with an embed, the module captures a still vignette of the embed (stored like any media file, shown in the poster frame as a click-to-load thumbnail and in the feed as a compact framed image):
- Headless Chromium (a real full-page screenshot) — OPTIONAL. Enable it
with
pip install playwright && playwright install chromium(or install thesnapshotextra). Any missing package / browser / timeout degrades silently. - og:image fallback — the page's
og:imagefetched through the same SSRF guard (https-only, must resolve to a public IP, redirects re-validated, size-capped). This is the default path when Chromium is not installed, so the feature works fully without a browser.
The capture runs OFF the shared event loop (asyncio.to_thread) and only
re-captures when the embed_url changes.
GET /admin/billets/{id}/archive.html (JWT-gated) exports a portable,
fully self-contained single .html of a billet in the communiqué layout: CSS
inlined, system fonts (no network), media + the embed vignette inlined as
data: URIs, and the embed rendered as its snapshot image linking to the
original (no live iframe, so it opens offline). Button: "💾 Archive HTML".
To push revisions to Gitea, add an origin remote in the revisions repo as the
secubox user (else commits stay local — still a full history).
Backup / restore
SQLite is the single source of truth. Take a consistent snapshot:
sudo -u secubox sqlite3 /var/lib/secubox/billets/billets.db ".backup '/var/backups/billets-$(date +%F).db'"
Restore: stop the service, copy the backup over billets.db (and the -wal/-shm
are rebuilt), start again. The Gitea revision repo and /etc/secubox/secrets/billets
should be backed up alongside.
Tests
PYTHONPATH=. python -m pytest -q # from packages/secubox-billets
Covers data layer, feed/pagination, admin auth/CRUD, SSRF + XSS (the critical
step), comments/anti-spam, reactions, feeds/oEmbed. pip-audit runs in CI.
Design decisions (documented per the brief)
- aiosqlite (async), not sync sqlite3. As a SecuBox module, billets shares the aggregator's event loop path; a blocking DB call would stall the box. All DB access is awaited off-loop.
api/layout + Unix socket, following SecuBox module conventions (rather than the standaloneapp/layout).- Own author auth (argon2id + optional TOTP) kept per spec; SecuBox-auth SSO is a possible v2.
- htmx replaced by a tiny vanilla shim (
static/billets.js, CSPscript-src 'self'). The reaction endpoint returns an htmx-compatible fragment, so dropping in a vendoredhtmx.min.jsat deploy works unchanged — this avoids an un-fetchable CDN dependency while keeping graceful no-JS degradation. - Emails hashed (BLAKE2b), not encrypted — never displayed, only used to detect a returning approved commenter (spec allows "chiffré au repos ou hashé").
- oEmbed discovery is same-origin only (Mastodon/PeerTube self-hosted); an unknown host with no same-origin oEmbed link falls back to a local OpenGraph card.