feat(billets): step 7 — ASGI entrypoint, systemd/nginx/debian deploy, README

api/asgi.py (uvicorn api.asgi:app) with a lifespan that opens the WAL DB + runs
migrations; api/manage.py CLI (create-author/set-password/seed). Debian package
(control/rules/postinst: secubox user, venv+pinned requirements, signing secret,
nginx vhost, www-data->secubox group for socket access) + standalone deploy/
(install.sh, billets.service, nginx.conf). README documents install/config/backup
and every 'tranche et documente' decision. 88 tests.

Co-Authored-By: Gerald KERMA <devel@cybermind.fr>
This commit is contained in:
CyberMind-FR 2026-07-11 15:52:08 +02:00
parent 0f3906dbe4
commit 68608a3857
16 changed files with 525 additions and 3 deletions

View File

@ -0,0 +1,99 @@
<!-- SPDX-License-Identifier: LicenseRef-CMSD-1.0 -->
# billets — micro-blog gateway inter-médias sociaux
Self-hosted micro-blog + social-media gateway. Author-published short posts; a
public read feed with comments, emoji reactions and republish share-intents. It
**ingests** links/embeds from social networks (SSRF-guarded oEmbed, sanitized)
and lets each billet be **re-shared** back to them.
Built as a SecuBox module (`secubox-billets`): FastAPI + **aiosqlite (WAL)** +
Jinja2, served on a Unix socket behind nginx.
## Architecture
```
api/
asgi.py runtime entrypoint (uvicorn api.asgi:app) — lifespan opens the DB
main.py app factory + public feed / permalink / feeds / oEmbed
db.py aiosqlite WAL + numbered .sql migrations
models.py Pydantic v2 (strict, size-limited, https-only URLs)
repo.py billet/comment/reaction/author queries (keyset pagination)
ids.py ULID (time-sortable)
routes/ admin.py (auth + CRUD + moderation), public.py (react/comment)
services/ render, sanitize, ssrf, oembed, linkcard, eventlog, revisions,
antispam, security, feeds
templates/ static/ migrations/
manage.py CLI: create-author / set-password / seed
```
**Invariants.** Every security-relevant action (publish/edit/delete, comment
moderation, login) appends to an **append-only BLAKE2b hash-chained** `event_log`
(`services/eventlog`, `verify_chain`). Every billet's content **and style** are
versioned in a **Gitea repo** (`services/revisions`, one front-matter `.md` per
billet, full `git log --follow` history).
## Install
### Debian package
```
dpkg -i secubox-billets_*.deb
# postinst creates the secubox user, /var/lib/secubox/billets, the signing
# secret, the venv (pip install requirements.txt), and the nginx vhost.
cd /usr/lib/secubox/billets
sudo -u secubox venv/bin/python -m api.manage create-author admin
```
Set the public origin so feeds/oEmbed emit absolute URLs — add to the unit:
`Environment=BILLETS_SITE_URL=https://billets.example.com`.
### Standalone
`sudo deploy/install.sh` (idempotent). In SecuBox, billets is fronted by
HAProxy (TLS 1.3) → sbxwaf → nginx → the socket.
## Configuration (environment)
| Var | Default | Purpose |
|-----|---------|---------|
| `BILLETS_DB` | `/var/lib/secubox/billets/billets.db` | SQLite WAL file |
| `BILLETS_REVISIONS_DIR` | `/var/lib/secubox/billets/revisions` | Gitea revision repo |
| `BILLETS_SECRET_FILE` | `/etc/secubox/secrets/billets` | signing secret (0600 secubox) |
| `BILLETS_SECRET` | — | secret override (takes precedence) |
| `BILLETS_SITE_URL` | derived from request | absolute origin for feeds/oEmbed |
To push revisions to Gitea, add an `origin` remote in the revisions repo as the
`secubox` user (else commits stay local — still a full history).
## Backup / restore
SQLite is the single source of truth. Take a consistent snapshot:
```
sudo -u secubox sqlite3 /var/lib/secubox/billets/billets.db ".backup '/var/backups/billets-$(date +%F).db'"
```
Restore: stop the service, copy the backup over `billets.db` (and the `-wal`/`-shm`
are rebuilt), start again. The Gitea revision repo and `/etc/secubox/secrets/billets`
should be backed up alongside.
## Tests
```
PYTHONPATH=. python -m pytest -q # from packages/secubox-billets
```
Covers data layer, feed/pagination, admin auth/CRUD, **SSRF + XSS** (the critical
step), comments/anti-spam, reactions, feeds/oEmbed. `pip-audit` runs in CI.
## Design decisions (documented per the brief)
- **aiosqlite (async), not sync sqlite3.** As a SecuBox module, billets shares
the aggregator's event loop path; a blocking DB call would stall the box. All
DB access is awaited off-loop.
- **`api/` layout + Unix socket**, following SecuBox module conventions (rather
than the standalone `app/` layout).
- **Own author auth** (argon2id + optional TOTP) kept per spec; SecuBox-auth SSO
is a possible v2.
- **htmx replaced by a tiny vanilla shim** (`static/billets.js`, CSP
`script-src 'self'`). The reaction endpoint returns an htmx-compatible fragment,
so dropping in a vendored `htmx.min.js` at deploy works unchanged — this avoids
an un-fetchable CDN dependency while keeping graceful no-JS degradation.
- **Emails hashed** (BLAKE2b), not encrypted — never displayed, only used to
detect a returning approved commenter (spec allows "chiffré au repos ou hashé").
- **oEmbed discovery is same-origin only** (Mastodon/PeerTube self-hosted); an
unknown host with no same-origin oEmbed link falls back to a local OpenGraph card.

View File

@ -0,0 +1,34 @@
# SPDX-License-Identifier: LicenseRef-CMSD-1.0
# Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
"""Runtime ASGI entrypoint: `uvicorn api.asgi:app`.
A lifespan opens the WAL SQLite connection (running migrations) once at startup
and closes it (plus the outbound HTTP client) on shutdown. The DB path, signing
secret, and revision dir come from the environment (see README)."""
from __future__ import annotations
from contextlib import asynccontextmanager
from datetime import datetime, timezone
from . import db
from .main import create_app
def _now() -> str:
return datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
@asynccontextmanager
async def lifespan(app):
conn = await db.connect(now=_now())
app.state.conn = conn
try:
yield
finally:
await conn.close()
client = getattr(app.state, "http_client", None)
if client is not None:
await client.aclose()
app = create_app(conn=None, lifespan=lifespan)

View File

@ -99,9 +99,13 @@ def _billet_view(row: aiosqlite.Row) -> dict:
return d
def create_app(conn: aiosqlite.Connection, *, secret: str | None = None,
revisions_dir: str | None = None) -> FastAPI:
app = FastAPI(title="billets", docs_url=None, redoc_url=None, openapi_url=None)
def create_app(conn: aiosqlite.Connection | None = None, *, secret: str | None = None,
revisions_dir: str | None = None, lifespan=None) -> FastAPI:
# `conn` may be None when a `lifespan` opens the DB at startup (runtime);
# tests pass a live connection and no lifespan. Routes read
# `app.state.conn` at request time, so either wiring works.
app = FastAPI(title="billets", docs_url=None, redoc_url=None, openapi_url=None,
lifespan=lifespan)
app.state.conn = conn
app.state.secret = secret or sec.get_secret()
app.state.revisions_dir = revisions_dir or DEFAULT_REVISIONS_DIR

View File

@ -0,0 +1,87 @@
# SPDX-License-Identifier: LicenseRef-CMSD-1.0
# Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
"""Management CLI: `python -m api.manage <cmd>`.
create-author <username> create the (first) author; prompts for a password
set-password <username> reset an author's password
seed insert sample published billets
"""
from __future__ import annotations
import argparse
import asyncio
import getpass
import sys
from datetime import datetime, timezone
from . import db, repo
from .seed import seed_billets
from .services import security as sec
def _now() -> str:
return datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
async def _create_author(username: str, password: str) -> int:
conn = await db.connect(now=_now())
try:
if await repo.get_author_by_username(conn, username):
print(f"author {username!r} already exists", file=sys.stderr)
return 1
await repo.create_author(conn, username, sec.hash_password(password), now=_now())
print(f"author {username!r} created")
return 0
finally:
await conn.close()
async def _set_password(username: str, password: str) -> int:
conn = await db.connect(now=_now())
try:
row = await repo.get_author_by_username(conn, username)
if not row:
print(f"no such author {username!r}", file=sys.stderr)
return 1
await conn.execute("UPDATE author SET password_hash=? WHERE username=?",
(sec.hash_password(password), username))
await conn.commit()
print(f"password updated for {username!r}")
return 0
finally:
await conn.close()
async def _seed() -> int:
conn = await db.connect(now=_now())
try:
ids = await seed_billets(conn)
print(f"seeded {len(ids)} billets")
return 0
finally:
await conn.close()
def main(argv: list[str] | None = None) -> int:
p = argparse.ArgumentParser(prog="billets-manage")
sub = p.add_subparsers(dest="cmd", required=True)
for name in ("create-author", "set-password"):
sp = sub.add_parser(name)
sp.add_argument("username")
sub.add_parser("seed")
args = p.parse_args(argv)
if args.cmd in ("create-author", "set-password"):
pw = getpass.getpass("Password: ")
if len(pw) < 10:
print("password too short (min 10)", file=sys.stderr)
return 2
fn = _create_author if args.cmd == "create-author" else _set_password
return asyncio.run(fn(args.username, pw))
if args.cmd == "seed":
return asyncio.run(_seed())
return 2
if __name__ == "__main__":
raise SystemExit(main())

View File

@ -0,0 +1,10 @@
secubox-billets (0.1.0-1~bookworm1) bookworm; urgency=medium
* Initial release: billets micro-blog gateway. Public feed (cursor pagination,
permalinks, Atom/JSON), admin (argon2id + optional TOTP, CSRF, billet CRUD),
SSRF-guarded oEmbed + nh3 sanitization + OpenGraph link cards, moderated
comments (honeypot + timed token + rate-limit) and emoji reactions, outbound
oEmbed + Republier share menu. Per-billet Gitea revisioning; BLAKE2b-chained
event log. FastAPI + aiosqlite (WAL) on a Unix socket behind nginx.
-- Gerald KERMA <devel@cybermind.fr> Sat, 11 Jul 2026 15:00:00 +0200

View File

@ -0,0 +1 @@
13

View File

@ -0,0 +1,23 @@
Source: secubox-billets
Section: web
Priority: optional
Maintainer: Gerald KERMA <devel@cybermind.fr>
Build-Depends: debhelper-compat (= 13)
Standards-Version: 4.6.2
Homepage: https://secubox.in
Package: secubox-billets
Architecture: all
Depends: ${misc:Depends},
python3 (>= 3.11),
python3-venv,
adduser,
nginx
Description: billets — micro-blog gateway inter-médias sociaux (SecuBox)
Self-hosted micro-blog with an author admin, a public read feed with comments
and emoji reactions, and inbound/outbound social-media gateway features:
SSRF-guarded oEmbed ingestion (sanitized, iframe-allowlisted), OpenGraph link
cards, Atom/JSON feeds, an outbound oEmbed endpoint and share intents.
FastAPI + aiosqlite (WAL) + Jinja2, served on a Unix socket behind nginx.
Every billet's content and style are versioned in a Gitea repo; every security
decision is written to an append-only BLAKE2b-chained event log.

View File

@ -0,0 +1,48 @@
#!/bin/bash
set -e
PREFIX=/usr/lib/secubox/billets
DATA=/var/lib/secubox/billets
case "$1" in
configure)
id -u secubox >/dev/null 2>&1 || \
adduser --system --group --no-create-home --home /var/lib/secubox \
--shell /usr/sbin/nologin secubox
install -d -o root -g root -m 1777 /run/secubox
install -d -o secubox -g secubox -m 0755 "$DATA" "$DATA/revisions"
# Parent 0755 (traversable), secrets dir 0700 secubox.
install -d -o secubox -g secubox -m 0755 /etc/secubox
install -d -o secubox -g secubox -m 0700 /etc/secubox/secrets
# Signing secret, once, 0600 secubox.
if [ ! -s /etc/secubox/secrets/billets ]; then
( umask 077; head -c 48 /dev/urandom | base64 | tr -d '\n' > /etc/secubox/secrets/billets )
chown secubox:secubox /etc/secubox/secrets/billets
chmod 0600 /etc/secubox/secrets/billets
fi
# Python venv with pinned deps (needs network or a local wheelhouse).
if [ ! -x "$PREFIX/venv/bin/python" ]; then
python3 -m venv "$PREFIX/venv"
fi
"$PREFIX/venv/bin/pip" install --upgrade pip >/dev/null 2>&1 || true
"$PREFIX/venv/bin/pip" install -r "$PREFIX/requirements.txt" \
|| echo "billets: pip install failed — install deps into $PREFIX/venv then 'systemctl restart secubox-billets'" >&2
chown -R secubox:secubox "$PREFIX"
# nginx must read the uvicorn socket (group secubox).
getent group secubox >/dev/null && adduser www-data secubox >/dev/null 2>&1 || true
if [ ! -e /etc/nginx/sites-enabled/billets.conf ] && [ -e /etc/nginx/sites-available/billets.conf ]; then
ln -s ../sites-available/billets.conf /etc/nginx/sites-enabled/billets.conf 2>/dev/null || true
fi
;;
esac
#DEBHELPER#
case "$1" in
configure)
systemctl reload nginx 2>/dev/null || true
echo "billets: create the first author with:" >&2
echo " sudo -u secubox $PREFIX/venv/bin/python -m api.manage create-author admin (run in $PREFIX)" >&2
;;
esac
exit 0

View File

@ -0,0 +1,8 @@
#!/bin/bash
set -e
case "$1" in
remove|deconfigure)
systemctl stop secubox-billets.service 2>/dev/null || true
;;
esac
#DEBHELPER#

View File

@ -0,0 +1,13 @@
#!/usr/bin/make -f
%:
dh $@
override_dh_auto_build:
override_dh_auto_test:
override_dh_auto_install:
install -d debian/secubox-billets/usr/lib/secubox/billets
cp -r api debian/secubox-billets/usr/lib/secubox/billets/
install -m 0644 requirements.txt debian/secubox-billets/usr/lib/secubox/billets/requirements.txt
install -d debian/secubox-billets/etc/nginx/sites-available
install -m 0644 deploy/nginx.conf debian/secubox-billets/etc/nginx/sites-available/billets.conf

View File

@ -0,0 +1,33 @@
# SPDX-License-Identifier: LicenseRef-CMSD-1.0
# Installed as /etc/systemd/system/secubox-billets.service
[Unit]
Description=SecuBox billets — micro-blog gateway inter-médias sociaux
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
User=secubox
Group=secubox
WorkingDirectory=/usr/lib/secubox/billets
Environment=BILLETS_DB=/var/lib/secubox/billets/billets.db
Environment=BILLETS_REVISIONS_DIR=/var/lib/secubox/billets/revisions
Environment=BILLETS_SECRET_FILE=/etc/secubox/secrets/billets
# Set BILLETS_SITE_URL to the public https origin so feeds/oEmbed emit absolute URLs.
# Environment=BILLETS_SITE_URL=https://billets.example.com
ExecStartPre=/usr/bin/install -d -o secubox -g secubox -m 1777 /run/secubox
ExecStart=/usr/lib/secubox/billets/venv/bin/uvicorn api.asgi:app \
--uds /run/secubox/billets.sock --no-access-log --workers 1
Restart=on-failure
RestartSec=2
# Hardening (CSPN posture)
NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=true
PrivateTmp=true
ReadWritePaths=/var/lib/secubox/billets /run/secubox
UMask=0007
[Install]
WantedBy=multi-user.target

View File

@ -0,0 +1,33 @@
# SPDX-License-Identifier: LicenseRef-CMSD-1.0
# Installed as /etc/systemd/system/secubox-billets.service
[Unit]
Description=SecuBox billets — micro-blog gateway inter-médias sociaux
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
User=secubox
Group=secubox
WorkingDirectory=/usr/lib/secubox/billets
Environment=BILLETS_DB=/var/lib/secubox/billets/billets.db
Environment=BILLETS_REVISIONS_DIR=/var/lib/secubox/billets/revisions
Environment=BILLETS_SECRET_FILE=/etc/secubox/secrets/billets
# Set BILLETS_SITE_URL to the public https origin so feeds/oEmbed emit absolute URLs.
# Environment=BILLETS_SITE_URL=https://billets.example.com
ExecStartPre=/usr/bin/install -d -o secubox -g secubox -m 1777 /run/secubox
ExecStart=/usr/lib/secubox/billets/venv/bin/uvicorn api.asgi:app \
--uds /run/secubox/billets.sock --no-access-log --workers 1
Restart=on-failure
RestartSec=2
# Hardening (CSPN posture)
NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=true
PrivateTmp=true
ReadWritePaths=/var/lib/secubox/billets /run/secubox
UMask=0007
[Install]
WantedBy=multi-user.target

View File

@ -0,0 +1,48 @@
#!/usr/bin/env bash
# SPDX-License-Identifier: LicenseRef-CMSD-1.0
# Idempotent standalone install for Debian bookworm (arm64/x86_64).
# For SecuBox, prefer the .deb (debian/). This script does the same by hand.
set -euo pipefail
PREFIX=/usr/lib/secubox/billets
DATA=/var/lib/secubox/billets
SECRETS=/etc/secubox/secrets
SRC="$(cd "$(dirname "$0")/.." && pwd)"
id -u secubox >/dev/null 2>&1 || \
adduser --system --group --no-create-home --home /var/lib/secubox \
--shell /usr/sbin/nologin secubox
install -d -o secubox -g secubox -m 0755 "$DATA" "$DATA/revisions"
install -d -o secubox -g secubox -m 0700 "$SECRETS"
install -d -o root -g root -m 1777 /run/secubox
# Signing secret (once; 0600 secubox).
if [ ! -s "$SECRETS/billets" ]; then
umask 077
head -c 48 /dev/urandom | base64 | tr -d '\n' > "$SECRETS/billets"
chown secubox:secubox "$SECRETS/billets"; chmod 0600 "$SECRETS/billets"
fi
# Code + venv.
install -d "$PREFIX"
cp -r "$SRC/api" "$PREFIX/"
install -m 0644 "$SRC/requirements.txt" "$PREFIX/requirements.txt"
python3 -m venv "$PREFIX/venv"
"$PREFIX/venv/bin/pip" install --upgrade pip >/dev/null
"$PREFIX/venv/bin/pip" install -r "$PREFIX/requirements.txt"
chown -R secubox:secubox "$PREFIX"
# systemd + nginx.
install -m 0644 "$SRC/deploy/billets.service" /etc/systemd/system/secubox-billets.service
install -d /etc/nginx/sites-available
install -m 0644 "$SRC/deploy/nginx.conf" /etc/nginx/sites-available/billets.conf
getent group secubox >/dev/null && adduser www-data secubox >/dev/null 2>&1 || true
systemctl daemon-reload
systemctl enable --now secubox-billets.service
nginx -t && systemctl reload nginx || true
echo "billets installed. Create the author:"
echo " sudo -u secubox $PREFIX/venv/bin/python -m api.manage create-author admin"
echo "(run from $PREFIX). Then browse the vhost and /admin."

View File

@ -0,0 +1,28 @@
# SPDX-License-Identifier: LicenseRef-CMSD-1.0
# billets vhost. In SecuBox this listens on the internal nginx port and is
# fronted by HAProxy (TLS 1.3) -> sbxwaf -> here; standalone, terminate TLS here
# (certbot) and redirect :80 -> :443. Proxies to the uvicorn Unix socket; nginx
# serves /static directly. nginx must be able to read the socket its worker
# user (www-data) is added to the `secubox` group by the package postinst.
server {
listen 8910;
server_name billets.example.com;
# Static assets straight from disk (CSP-safe: same origin).
location /static/ {
alias /usr/lib/secubox/billets/api/static/;
expires 7d;
access_log off;
try_files $uri =404;
}
location / {
proxy_pass http://unix:/run/secubox/billets.sock;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_read_timeout 30s;
client_max_body_size 1m;
}
}

View File

@ -0,0 +1,15 @@
# SPDX-License-Identifier: LicenseRef-CMSD-1.0
# Pinned runtime deps for the billets venv (pip-audit in CI). Wheels for
# argon2-cffi / nh3 (rust) ship for linux aarch64 + x86_64.
fastapi==0.115.6
uvicorn==0.34.0
pydantic==2.10.4
aiosqlite==0.20.0
jinja2==3.1.5
argon2-cffi==23.1.0
nh3==0.2.20
httpx==0.28.1
pyotp==2.9.0
itsdangerous==2.2.0
markdown-it-py==3.0.0
python-multipart==0.0.20

View File

@ -0,0 +1,38 @@
# SPDX-License-Identifier: LicenseRef-CMSD-1.0
# Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
import importlib
async def test_asgi_lifespan_opens_and_migrates(tmp_path, monkeypatch):
monkeypatch.setenv("BILLETS_DB", str(tmp_path / "b.db"))
monkeypatch.setenv("BILLETS_SECRET", "test-secret")
monkeypatch.setenv("BILLETS_REVISIONS_DIR", str(tmp_path / "revs"))
from api import asgi as asgi_mod
importlib.reload(asgi_mod)
app = asgi_mod.app
async with asgi_mod.lifespan(app):
assert app.state.conn is not None
async with app.state.conn.execute(
"SELECT name FROM sqlite_master WHERE type='table'") as cur:
names = {r[0] for r in await cur.fetchall()}
assert {"billet", "comment", "reaction", "author", "event_log"} <= names
async def test_manage_create_author(tmp_path, monkeypatch):
monkeypatch.setenv("BILLETS_DB", str(tmp_path / "m.db"))
monkeypatch.setenv("BILLETS_SECRET", "test-secret")
from api import manage
assert await manage._create_author("gk", "supersecret123") == 0
assert await manage._create_author("gk", "supersecret123") == 1 # duplicate
assert await manage._set_password("gk", "another-secret-9") == 0
assert await manage._set_password("ghost", "another-secret-9") == 1
async def test_manage_seed(tmp_path, monkeypatch):
monkeypatch.setenv("BILLETS_DB", str(tmp_path / "s.db"))
from api import manage, db
assert await manage._seed() == 0
conn = await db.connect(now="2026-07-11T12:00:00Z")
async with conn.execute("SELECT COUNT(*) FROM billet WHERE status='published'") as cur:
assert (await cur.fetchone())[0] >= 1
await conn.close()