docs: MODULE-GUIDELINES + grafana/yacy/rustdesk LXC plan

- docs/MODULE-GUIDELINES.md (new) — canonical reference for authoring a
  secubox-<module> package: file structure, LXC layout, WebUI conventions,
  nginx wiring, Debian packaging, CTL three-fold, FastAPI shape, tests,
  validation checklist. Sister doc to docs/grammar.md and docs/UI-GUIDE.md.
- docs/grammar.md — cross-reference MODULE-GUIDELINES.md in the header
  sister-docs block.
- docs/MODULES.md — point new-module authors at MODULE-GUIDELINES.md.
- docs/superpowers/plans/2026-05-20-grafana-yacy-rustdesk-lxc-modules.md
  (new) — concrete implementation plan for three new LXC-hosted modules:
  secubox-grafana (#229), secubox-yacy (#230), secubox-rustdesk (#231).
  All three follow the new MODULE-GUIDELINES.md. IP allocations 10.100.0.{70,80,90}.
This commit is contained in:
CyberMind-FR 2026-05-20 06:43:04 +02:00
parent 5b11e51698
commit 71f631003d
4 changed files with 1403 additions and 0 deletions

600
docs/MODULE-GUIDELINES.md Normal file
View File

@ -0,0 +1,600 @@
# SecuBox Module Guidelines
> Canonical reference for authoring a new `secubox-<module>` Debian package.
> Read alongside [`docs/grammar.md`](grammar.md) (CTL grammar) and
> [`docs/UI-GUIDE.md`](UI-GUIDE.md) (theme + sidebar conventions).
>
> The HOWTO that walks through adding a new CTL verb is at
> [`HOWTO-grammar.md`](../HOWTO-grammar.md). This document covers the
> module-level scaffolding around a verb (packaging, LXC, FastAPI host
> control plane, web UI, nginx wiring).
---
## 1. When to create a new module
A new `secubox-<module>` package is justified when **all** of these hold:
- It introduces a new noun in the CTL grammar (not a verb on an existing noun).
- It owns at least one of: a long-running daemon, a web UI route under `/<module>/`, a public-facing endpoint under `/api/v1/<module>/`, or a Debian-distributable bundle of templates/config.
- It is independently uninstallable (`apt remove secubox-<module>` must leave the rest of SecuBox functional).
Examples:
| Add a module | Don't add a module |
|---|---|
| Grafana dashboards backed by a Grafana LXC | A new dashboard JSON for the existing `secubox-metrics` module |
| YaCy peer search engine | A YaCy crawl-result viewer that lives in `secubox-hub` |
| RustDesk relay | A `wireguard add peer rustdesk-relay` verb on the existing VPN module |
---
## 2. Module shape (file structure)
The canonical layout. Every directory below is optional except `debian/`, but most modules grow into the full shape.
```text
packages/secubox-<module>/
├── api/
│ ├── __init__.py
│ ├── main.py # FastAPI on /run/secubox/<module>.sock
│ ├── models.py # Pydantic schemas
│ └── lxc_helpers.py # LXC modules only
├── conf/
│ └── <module>.toml.example # installed to /etc/secubox/
├── debian/
│ ├── changelog
│ ├── control # see §6
│ ├── rules # see §6
│ ├── install # or use rules override_dh_auto_install
│ ├── postinst # idempotent — see §6
│ ├── prerm
│ ├── postrm
│ └── secubox-<module>.service # systemd unit — see §6
├── lib/
│ └── <module>/ # LXC modules only — see §3
│ ├── install-lxc.sh
│ ├── update-lxc.sh
│ └── provision/
├── menu.d/
│ └── 50-<module>.json # SecuBox sidebar entry — see §4
├── nginx/
│ └── <module>.conf # /etc/nginx/secubox.d/ snippet — see §5
├── README.md # operator-facing
├── sbin/
│ └── <module>ctl # bash CLI — see §7 + grammar.md
├── tests/
│ ├── helpers.bash # bats helpers
│ └── test-*.bats # CLI tests
└── www/
├── <module>/
│ ├── index.html # SecuBox-themed landing — see UI-GUIDE.md
│ ├── settings.html
│ └── logs.html
└── shared/ # rarely needed — most JS/CSS lives in secubox-hub
```
Mirror an existing module of similar shape rather than starting from scratch:
| Pattern | Reference module |
|---|---|
| LXC-hosted daemon | `packages/secubox-gitea/`, `packages/secubox-mail/` |
| Host-only daemon | `packages/secubox-metrics/`, `packages/secubox-crowdsec/` |
| WAF / proxy add-on | `packages/secubox-mitmproxy/` |
| Pure web UI on existing service | `packages/secubox-soc-web/` |
| Pure CLI / no daemon | `packages/secubox-droplet/` |
---
## 3. LXC layout (modules that run their daemon in a Debian LXC)
### When LXC is the right choice
- Daemon needs a different runtime (JVM, Alpine, custom apt repo) than the host.
- Daemon is high-risk and should run in an isolated rootfs (RustDesk, mitmproxy).
- Daemon needs root-level setup that would conflict with another module's daemon if run on the host.
### IP allocation
The SecuBox LXC bridge is `br-secubox` on `10.100.0.0/24`. Allocations:
| IP | Module |
|---|---|
| `.1` | bridge gateway (host side) |
| `.10` | mail |
| `.11`/`.12` | mail-related (rspamd, sieve) |
| `.30` | (reserved) |
| `.40` | gitea |
| `.70` | grafana (planned, #229) |
| `.80` | yacy (planned, #230) |
| `.90` | rustdesk (planned, #231) |
Pick the next free `.X` above the current high-water mark. Reserve the IP in
`docs/grammar.md` or this doc on submission of the module PR.
### Bootstrap script: `lib/<module>/install-lxc.sh`
Idempotent shell script that creates the LXC if absent, debootstraps it,
installs the daemon, provisions defaults, and exits 0. Must be safely
re-runnable (the operator may run it from `<module>ctl install` repeatedly).
Anatomy:
```bash
#!/usr/bin/env bash
set -euo pipefail
readonly LXC_NAME="${SECUBOX_LXC_NAME:-<module>}"
readonly LXC_IP="${SECUBOX_LXC_IP:-10.100.0.XX}"
readonly LXC_PATH="${SECUBOX_LXC_PATH:-/data/lxc}"
readonly DEBIAN_SUITE="bookworm"
# 1. Idempotency short-circuit
lxc-info -n "$LXC_NAME" -P "$LXC_PATH" 2>/dev/null | grep -q '^State: *RUNNING' && exit 0
# 2. Create if missing
[ -d "$LXC_PATH/$LXC_NAME" ] || lxc-create -n "$LXC_NAME" -t debian -P "$LXC_PATH" -- -r "$DEBIAN_SUITE"
# 3. Pin static IP, AppArmor profile, autostart
cat > "$LXC_PATH/$LXC_NAME/config" <<EOF
lxc.uts.name = $LXC_NAME
lxc.net.0.type = veth
lxc.net.0.link = br-secubox
lxc.net.0.flags = up
lxc.net.0.ipv4.address = $LXC_IP/24
lxc.net.0.ipv4.gateway = 10.100.0.1
lxc.rootfs.path = dir:$LXC_PATH/$LXC_NAME/rootfs
lxc.include = /usr/share/lxc/config/common.conf
lxc.apparmor.profile = generated
lxc.start.auto = 1
EOF
# 4. Start + wait for network
lxc-start -n "$LXC_NAME" -P "$LXC_PATH"
for i in $(seq 1 30); do
lxc-attach -n "$LXC_NAME" -P "$LXC_PATH" -- ping -c1 -W1 10.100.0.1 >/dev/null 2>&1 && break
sleep 1
done
# 5. Install daemon + dependencies
lxc-attach -n "$LXC_NAME" -P "$LXC_PATH" -- bash -c '
apt-get update -q
DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends <packages>
'
# 6. Provision (dashboards / peer lists / keys)
"$(dirname "$0")/provision/run.sh"
# 7. Sentinel
touch /var/lib/secubox/<module>/.lxc-provisioned
```
### Provisioning (`lib/<module>/provision/`)
Ship configuration that the operator expects out-of-the-box:
- Grafana dashboards (`.json`), datasource yaml.
- YaCy peer seed list, blacklist, default crawl profile.
- RustDesk key material is generated, not shipped — but a `key generate` wrapper goes here.
Provisioned content is read-only after install; user customisation lives in
`/etc/secubox/<module>.toml` and overrides the provisioned defaults.
### Not in the image — installed on demand
Modules whose LXC rootfs is >100 MB should **not** be pre-built into the
SecuBox system image. The package ships only the install script; the
operator runs `<module>ctl install` post-firstboot.
This keeps the v2.10.x image at ~8 GB. Modules requiring large daemons
(grafana, yacy, rustdesk) follow this rule. Compact LXCs (mitmproxy ~50 MB)
can pre-build during image construction.
---
## 4. WebUI conventions
### Theme
Mandatory. See [`docs/UI-GUIDE.md`](UI-GUIDE.md):
- CRT P31 phosphor palette via `/shared/crt-{light,dark}.css`
- Sidebar via `/shared/sidebar-{light,dark}.css` populated by `/shared/sidebar.js`
- Theme toggle persists to `localStorage.sbx_theme`
### Auth wrapper
Every page (except `/login.html`) must redirect to `/login.html?redirect=<current-path>`
when no `localStorage.sbx_token` is present. The hub's `index.html` shows the
canonical `checkAuth()` implementation:
```js
function checkAuth() {
if (!localStorage.getItem('sbx_token')) {
window.location.href = '/login.html?redirect=' + encodeURIComponent(window.location.pathname);
return false;
}
return true;
}
if (!checkAuth()) {
document.body.innerHTML = '<div style="padding:2rem;color:var(--root-main);">Redirecting to login...</div>';
}
```
Path is `/login.html`, **never** `/portal/login.html` — see #222.
### Menu integration (`menu.d/<module>.json`)
```json
{
"title": "Module Name",
"subtitle": "One-line purpose",
"icon": "fa-icon-name",
"url": "/<module>/",
"section": "monitoring|security|network|publishing|hosting|search|admin",
"order": 50,
"module": "<module>"
}
```
`section` slots into the existing sidebar grouping. `order` controls
position inside the section (10/20/30/.../90).
### Three-fold landing
The module's `index.html` should surface the same three sections the CTL
exposes — components / status / access — at the top of the page:
```html
<section class="threefold">
<article id="components">...</article> <!-- LXC state, daemon state, host API state -->
<article id="status">...</article> <!-- overall green/yellow/red + the last event -->
<article id="access">...</article> <!-- public hostname + auth method -->
</section>
```
Pulls live data from `/api/v1/<module>/components`, `/status`, `/access`.
### Iframe pattern for LXC web UIs
When the LXC daemon ships its own admin UI (Grafana, YaCy, RustDesk web
console), embed it inside a SecuBox-themed iframe under `/<module>/`. The
iframe target proxies through nginx to the LXC IP (see §5). Operator
navigates dashboards via the SecuBox sidebar (`menu.d`), not the native
chrome.
---
## 5. nginx wiring (`nginx/<module>.conf`)
Installed to `/etc/nginx/secubox.d/<module>.conf` and auto-included by
`/etc/nginx/sites-available/secubox`.
### Standard pattern (host FastAPI only)
```nginx
# /etc/nginx/secubox.d/<module>.conf
location /api/v1/<module>/ {
proxy_pass http://unix:/run/secubox/<module>.sock:/;
include /etc/nginx/snippets/secubox-proxy.conf;
}
```
### LXC daemon iframe target
```nginx
location /api/v1/<module>/ {
proxy_pass http://unix:/run/secubox/<module>.sock:/;
include /etc/nginx/snippets/secubox-proxy.conf;
}
# Native admin UI of the LXC daemon (iframe target)
location /<module>/ {
proxy_pass http://10.100.0.XX:PORT/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
```
Daemon must be configured to serve from the `/<module>/` subpath
(e.g. Grafana `serve_from_sub_path=true`).
### Non-HTTP daemons (rustdesk, raw TCP/UDP)
Don't proxy through nginx — proxy through HAProxy stream mode. See
`packages/secubox-haproxy/conf/` for the stream-backend pattern.
---
## 6. Debian packaging conventions
### `debian/control`
```text
Source: secubox-<module>
Section: admin
Priority: optional
Maintainer: Gerald KERMA <devel@cybermind.fr>
Build-Depends: debhelper-compat (= 13), dh-python, python3-all
Standards-Version: 4.6.2
Homepage: https://cybermind.fr/secubox
Package: secubox-<module>
Architecture: all
Depends: ${misc:Depends},
secubox-core (>= 1.0),
<module-specific deps>
Recommends: <optional companions>
Description: SecuBox <Module><one-line purpose>
<Longer description, 3-5 lines.>
```
**Do not** ship a `debian/compat` file. The `Build-Depends: debhelper-compat (= 13)` is the single source of truth (see #220).
### `debian/rules`
```makefile
#!/usr/bin/make -f
%:
dh $@ --with python3
override_dh_auto_install:
install -d debian/secubox-<module>/usr/lib/secubox/<module>
cp -r api debian/secubox-<module>/usr/lib/secubox/<module>/
install -d debian/secubox-<module>/usr/sbin
install -m 755 sbin/<module>ctl debian/secubox-<module>/usr/sbin/<module>ctl
install -d debian/secubox-<module>/usr/share/secubox/www
[ -d www ] && cp -r www/. debian/secubox-<module>/usr/share/secubox/www/ || true
install -d debian/secubox-<module>/usr/share/secubox/menu.d
[ -d menu.d ] && cp -r menu.d/. debian/secubox-<module>/usr/share/secubox/menu.d/ || true
install -d debian/secubox-<module>/etc/nginx/secubox.d
[ -f nginx/<module>.conf ] && cp nginx/<module>.conf debian/secubox-<module>/etc/nginx/secubox.d/ || true
# LXC modules only:
install -d debian/secubox-<module>/usr/share/secubox/lib/<module>
[ -d lib/<module> ] && cp -r lib/<module>/. debian/secubox-<module>/usr/share/secubox/lib/<module>/ || true
```
### `debian/postinst`
Idempotent system-user creation, directory provisioning, conditional
nginx reload, service enable. **Never start the FastAPI before its
prerequisites are present** — for LXC modules, defer the first start
to `<module>ctl install` (which fires after the LXC is up).
```sh
#!/bin/sh
set -e
case "$1" in
configure)
getent group secubox >/dev/null || groupadd --system secubox
getent passwd secubox >/dev/null || useradd --system --gid secubox \
--home /var/lib/secubox --no-create-home --shell /usr/sbin/nologin secubox
install -d -m 0770 -o root -g secubox /etc/secubox
install -d -m 0755 -o secubox -g secubox /var/lib/secubox/<module>
if [ ! -f /etc/secubox/<module>.toml ] && [ -f /etc/secubox/<module>.toml.example ]; then
cp /etc/secubox/<module>.toml.example /etc/secubox/<module>.toml
chmod 640 /etc/secubox/<module>.toml
chown root:secubox /etc/secubox/<module>.toml
fi
if systemctl is-active --quiet nginx 2>/dev/null; then
nginx -t >/dev/null 2>&1 && systemctl reload nginx 2>/dev/null || true
fi
systemctl daemon-reload 2>/dev/null || true
systemctl enable secubox-<module>.service 2>/dev/null || true
# LXC modules: do not start yet; ctl install will start after LXC up.
# Host-only modules: uncomment the next line:
# systemctl start secubox-<module>.service 2>/dev/null || true
;;
esac
#DEBHELPER#
exit 0
```
### `debian/secubox-<module>.service`
```ini
[Unit]
Description=SecuBox <Module> API
After=network.target secubox-core.service
Wants=secubox-core.service
[Service]
UMask=0007
Type=simple
User=secubox
Group=secubox
WorkingDirectory=/usr/lib/secubox/<module>
RuntimeDirectory=secubox
RuntimeDirectoryMode=0755
RuntimeDirectoryPreserve=yes
ExecStartPre=/bin/mkdir -p /etc/secubox
ExecStartPre=/bin/chown secubox:secubox /etc/secubox
ExecStart=/usr/bin/python3 -m uvicorn api.main:app --uds /run/secubox/<module>.sock --log-level warning
Restart=on-failure
RestartSec=5
NoNewPrivileges=true
LogsDirectory=secubox
LogsDirectoryMode=0755
ReadWritePaths=/run/secubox /var/lib/secubox /etc/secubox /var/log/secubox
[Install]
WantedBy=multi-user.target
```
Board-specific units (Pi/MOCHAbin I2C/LED) **must** gate on `ConditionArchitecture=arm64` to skip silently on x86_64 (see #226).
### Slipstream into the system image
The `build-packages.yml` workflow auto-discovers any new `packages/secubox-*/debian/control` on the next push. No workflow edit is required.
The `build-image.sh` slipstream loop (`cp /tmp/secubox-debs/secubox-*.deb`) picks up the new `_all.deb` automatically. New module appears in `dpkg -l 'secubox-*'` on the next image build.
`apt-get install -f -y` after the slipstream `dpkg -i --force-depends` step (since #218) pulls any new declared Debian deps — no need to add them to the debootstrap `INCLUDE_PKGS`.
---
## 7. CTL conventions
Read first: [`docs/grammar.md`](grammar.md) (canonical verbs table) +
[`HOWTO-grammar.md`](../HOWTO-grammar.md) (recipe for adding a verb).
### Mandatory three-fold
Every `<module>ctl` exposes:
| Noun | Verb | Output |
|---|---|---|
| `components` | (no verb) `list` `status` | one line per managed component (LXC, daemon, host-API, sub-units) |
| `status` | (no verb) | overall green/yellow/red + last event line |
| `access` | `list` `show <name>` | every URL/socket the module exposes + the auth method |
Plain stdout for humans, `--json` for machines. Schema:
```json
{
"module": "<module>",
"version": "<semver>",
"components": [
{"name": "lxc", "state": "running|stopped|absent", "detail": "..."},
{"name": "daemon", "state": "running|stopped|absent", "detail": "..."},
{"name": "host-api", "state": "running|stopped|absent", "detail": "..."}
]
}
```
### Module-specific noun-verb pairs
Per the grammar table — one verb closes one previously-implicit operation.
Examples in flight:
```text
grafanactl dashboard list/add/remove/export # OPS MONITORING (#229)
yacyctl index status/build/clear/optimize # SEARCH (#230)
rustdeskctl peer add/remove/list # REMOTE-ACCESS (#231)
```
### Style
- Imperative present tense: `add`, `remove`, `list`, `status`. Not `creating`, `deletion`, `listed`.
- Singular nouns: `route`, `repo`, `app`, `peer`, `dashboard`, `vhost`.
- Aliases accepted but not advertised in `--help`: `rm`/`del` for `remove`, `ls` for `list`.
- Exit codes: `0` success, `1` user error, `2` runtime error, `3` lxc/daemon down.
### Help
`<module>ctl --help` prints the noun-verb table; `<module>ctl <noun>` prints
the verbs for that noun; `<module>ctl <noun> <verb> --help` prints flags.
---
## 8. API conventions
### Transport
FastAPI + Uvicorn on a Unix socket at `/run/secubox/<module>.sock`. **Never** a TCP port (avoids LAN-side enumeration).
### Routing
nginx proxies `/api/v1/<module>/*` → the Unix socket. The FastAPI app
mounts its routes at the root (no `/api/v1/<module>` prefix in the app
code — nginx strips it).
### Mandatory endpoints
| Endpoint | Purpose |
|---|---|
| `GET /status` | Same JSON as `<module>ctl status --json` |
| `GET /components` | Same JSON as `<module>ctl components --json` |
| `GET /access` | Same JSON as `<module>ctl access --json` |
| `GET /healthz` | `{"ok": true}` if the API process is alive — no shelling out |
| `GET /version` | `{"version": "<semver>", "build": "<git-sha>"}` |
These five let `healthctl` watch the module without module-specific code.
### Module-specific endpoints
One endpoint per noun-verb pair the CTL exposes. The API can either
re-implement the logic in Python (fast) or shell out to `<module>ctl <noun> <verb> --json`
(simple, single source of truth). The latter is the SecuBox default.
### Auth
All non-trivial endpoints (anything mutating state, or returning data
beyond status/components/access/healthz/version) require JWT via
`Depends(auth.require_jwt)`. The JWT secret is generated at firstboot
into `/etc/secubox/secubox.conf`.
### Pydantic models
Define in `api/models.py`. Re-use shared models from `secubox-core` when
they fit (`secubox_core.models.JobStatus`, etc.).
---
## 9. Tests
### CTL tests
Use [bats-core](https://github.com/bats-core/bats-core). One file per noun,
or one file per coherent flow (`test-<module>ctl-install.bats`,
`test-<module>ctl-dashboards.bats`).
PATH-shim external commands (`lxc-create`, `lxc-attach`, `systemctl`,
`curl`) in `tests/helpers.bash` so the suite runs offline:
```bash
# tests/helpers.bash
setup() {
export PATH="$BATS_TEST_DIRNAME/stubs:$PATH"
# Each stub is a shell script that echoes a canned response.
}
```
### API tests
pytest + httpx async client. Spin up uvicorn on a tempfile socket per
test session; assert JSON schema.
### CI
Both run automatically as part of `Build SecuBox .deb packages` when
`debian/rules` includes them via `dh_auto_test` or `debian/tests/`.
---
## 10. Validation checklist (run before opening the PR)
- [ ] Package builds locally: `cd packages/secubox-<module> && dpkg-buildpackage -us -uc -b` succeeds with no warnings other than the standard `dpkg-source: warning: source format` line.
- [ ] `lintian --no-tag-display-limit ../secubox-<module>_*.deb` reports zero `E:` and zero `W:` (info lines OK).
- [ ] `<module>ctl --help` lists every noun.
- [ ] `<module>ctl <noun> --help` lists every verb on that noun.
- [ ] Three-fold JSON schemas match §7 and §8.
- [ ] `bats tests/` runs green offline (no internet, no LXC).
- [ ] WebUI page loads with `localStorage.sbx_token` set; redirects to `/login.html` when not.
- [ ] nginx vhost includes both `/api/v1/<module>/` and `/<module>/` (if iframe target).
- [ ] No `debian/compat` file (compat 13 declared via `Build-Depends` only).
- [ ] No Pi-only services without `ConditionArchitecture=arm64` (see #226).
- [ ] No reference to `/portal/login.html` (use `/login.html` — see #222).
- [ ] `apt-get install -y ../secubox-<module>_*.deb` on a fresh VM completes; `systemctl status secubox-<module>` is `loaded` (host modules: also `active`); LXC modules: `<module>ctl install` then `<module>ctl status` returns green.
- [ ] Module added to `docs/MODULES.md` catalog.
- [ ] CTL verbs added to `docs/grammar.md` canonical table.
- [ ] `.claude/MIGRATION-MAP.md` row added.
---
## References
- [`docs/grammar.md`](grammar.md) — canonical CTL grammar
- [`HOWTO-grammar.md`](../HOWTO-grammar.md) — recipe for adding a verb
- [`docs/UI-GUIDE.md`](UI-GUIDE.md) — UI theme and sidebar
- [`docs/MODULES.md`](MODULES.md) — module catalog
- [`docs/SECUBOX-DEV-METHODOLOGY.md`](SECUBOX-DEV-METHODOLOGY.md) — overall development workflow
- [`.claude/PATTERNS.md`](../.claude/PATTERNS.md) — RPCD → FastAPI porting patterns
- Plan example: [`docs/superpowers/plans/2026-05-20-grafana-yacy-rustdesk-lxc-modules.md`](superpowers/plans/2026-05-20-grafana-yacy-rustdesk-lxc-modules.md) — three modules following this guideline end-to-end.

View File

@ -6,6 +6,11 @@
This document catalogs all SecuBox Debian modules, their features, and UI screenshots locations.
> To **author** a new `secubox-<module>` package, read
> [`MODULE-GUIDELINES.md`](MODULE-GUIDELINES.md) first — it codifies the
> file layout, LXC pattern, WebUI conventions, CTL grammar fit, FastAPI
> shape, and the validation checklist.
---
## Module Overview by Category

View File

@ -9,6 +9,12 @@
> matured through SecuBox-OpenWrt and incarnates here as SecuBox-Deb on
> Debian bookworm arm64/amd64. This document codifies the CLI grammar that
> makes the modular concept expressible from the shell.
>
> Sister documents:
>
> - [`MODULE-GUIDELINES.md`](MODULE-GUIDELINES.md) — package/LXC/WebUI/API scaffolding around a verb
> - [`../HOWTO-grammar.md`](../HOWTO-grammar.md) — recipe for adding a new verb
> - [`UI-GUIDE.md`](UI-GUIDE.md) — theme and sidebar conventions
---

View File

@ -0,0 +1,792 @@
# Three new LXC SecuBox modules: secubox-grafana, secubox-yacy, secubox-rustdesk
> **For agentic workers:** REQUIRED SUB-SKILL: `superpowers:subagent-driven-development` (recommended) or `superpowers:executing-plans`. Steps use checkbox (`- [ ]`) syntax.
**Goal:** Add three new LXC-hosted SecuBox modules — Grafana (security metrics dashboards), YaCy (peer-to-peer search engine), RustDesk (self-hosted remote-desktop relay) — each shipped as a `secubox-<name>` Debian package with the standard SecuBox layout (CTL tool, FastAPI control plane, web UI, nginx vhost, menu integration, LXC bootstrap, three-fold introspection). All three slipstreamed into the next release artifact.
**Architecture:** One LXC container per module, debootstrap-based, IP-allocated on the existing `10.100.0.0/24` SecuBox LXC bridge. The host runs only the FastAPI control plane (Unix socket → nginx `/api/v1/<module>/`), the WebUI shell, the CTL tool, and the bootstrap/management scripts. The LXC runs the actual service daemon. Public exposure goes through `haproxyctl vhost add``mitmproxyctl route add` for HTTP/HTTPS modules (grafana, yacy); RustDesk uses HAProxy stream-mode (TCP+UDP) directly because mitmproxy doesn't inspect non-HTTP traffic.
**Tech stack:**
- Debian 12 bookworm LXC (consistent with mail, gitea, mitmproxy LXCs already in production)
- FastAPI + Uvicorn on `/run/secubox/<module>.sock` for the host control plane
- Bash CTL tool following the canonical SecuBox grammar (`<module>ctl <noun> <verb>`, three-fold `components|status|access`)
- Vanilla HTML/CSS/JS web UI under the SecuBox 6-module palette (DESIGN-CHARTER.md)
- nginx reverse proxy snippet in `/etc/nginx/secubox.d/<module>.conf` (auto-included by sites-available/secubox)
---
## 0. Pre-flight: IP and port allocations
| Module | LXC IP | LXC name | Internal ports | Public hostname (HAProxy vhost) |
|---|---|---|---|---|
| grafana | `10.100.0.70` | `grafana` | 3000/tcp (HTTP) | `grafana.gk2.secubox.in` |
| yacy | `10.100.0.80` | `yacy` | 8090/tcp (HTTP, admin), 8443/tcp (HTTPS) | `yacy.gk2.secubox.in` |
| rustdesk | `10.100.0.90` | `rustdesk` | 21114/tcp (web), 21115/tcp (NAT test), 21116/tcp+udp (hbbs signal), 21117/tcp (hbbr relay), 21118/tcp (web client), 21119/tcp (legacy) | `rustdesk.gk2.secubox.in` (web) + raw `21115-21119` TCP/UDP on bare host IP |
Confirm against `grep -rohE '10\.100\.0\.[0-9]+' packages/ image/ | sort -u` before T0; current high-water mark observed is `.60`. If `.70/.80/.90` collide, bump to the next free `.7X` range.
---
## 1. File structure (identical skeleton for all three)
Mirrored from `packages/secubox-gitea` + `packages/secubox-mail` (the two closest LXC references).
```text
packages/secubox-<module>/
├── api/
│ ├── __init__.py
│ ├── main.py # FastAPI: status, restart, logs, module-specific endpoints
│ ├── models.py # Pydantic schemas
│ └── lxc_helpers.py # shared lxc-attach / lxc-info wrappers (consider moving to secubox-core)
├── conf/
│ └── <module>.toml.example # written to /etc/secubox/<module>.toml.example
├── debian/
│ ├── changelog # 1.0.0-1~bookworm1
│ ├── control # Depends below
│ ├── install # path -> rootfs mapping (or use rules override_dh_auto_install)
│ ├── postinst # idempotent user/dir/service setup, NO LXC creation (defer to ctl)
│ ├── prerm # stop service, do not destroy LXC
│ ├── postrm # purge: rm /etc/secubox/<module>.toml, /var/lib/secubox/<module>
│ ├── rules # %: dh $@ --with python3 ; override_dh_auto_install
│ └── secubox-<module>.service # systemd unit for the host FastAPI
├── lib/
│ └── <module>/
│ ├── install-lxc.sh # idempotent: create LXC if missing, debootstrap, install daemon
│ ├── update-lxc.sh # in-place upgrade
│ ├── provision/ # per-module provisioning (grafana dashboards, yacy peer seeds, rustdesk keys)
│ └── README.md
├── menu.d/
│ └── 50-<module>.json # SecuBox web UI menu entry, see existing menu.d schema
├── nginx/
│ └── <module>.conf # location /api/v1/<module>/ + /<module>/ (proxy_pass unix socket / LXC IP)
├── README.md # operator-facing: install/upgrade/ctl reference/troubleshooting
├── sbin/
│ └── <module>ctl # bash CLI, three-fold + module-specific verbs
├── tests/
│ ├── helpers.bash # bats helpers (mock lxc-attach, PATH shims)
│ └── test-<module>ctl.bats # bats-core integration tests
└── www/
├── <module>/
│ ├── index.html # SecuBox-wrapped landing page (iframe to LXC web UI for grafana/yacy)
│ ├── settings.html
│ └── logs.html
└── shared/ # if any module-specific shared JS/CSS
```
### `debian/control` template
```text
Source: secubox-<module>
Section: admin
Priority: optional
Maintainer: Gerald KERMA <devel@cybermind.fr>
Build-Depends: debhelper-compat (= 13), dh-python, python3-all
Standards-Version: 4.6.2
Homepage: https://cybermind.fr/secubox
Package: secubox-<module>
Architecture: all
Depends: ${misc:Depends},
secubox-core (>= 1.0),
secubox-haproxy,
lxc,
lxc-templates,
debootstrap,
python3-uvicorn,
python3-fastapi,
python3-toml,
openssl,
curl,
<module-specific runtime deps below>
Recommends: secubox-mitmproxy
Description: SecuBox <Module><short purpose>
<Longer description, 3-5 lines, includes the three-fold verbs.>
```
Module-specific deps:
- **grafana**: nothing extra on host (the grafana daemon lives in LXC; the host only needs `curl` for health probes and `jq` for dashboard JSON munging — already pulled by other modules).
- **yacy**: same; the JVM lives in the LXC.
- **rustdesk**: `iproute2` (for `ss` to inspect UDP listeners during health probe).
### `debian/secubox-<module>.service`
```ini
[Unit]
Description=SecuBox <Module> API
After=network.target secubox-core.service
Wants=secubox-core.service
[Service]
UMask=0007
Type=simple
User=secubox
Group=secubox
WorkingDirectory=/usr/lib/secubox/<module>
RuntimeDirectory=secubox
RuntimeDirectoryMode=0755
RuntimeDirectoryPreserve=yes
ExecStartPre=/bin/mkdir -p /etc/secubox
ExecStartPre=/bin/chown secubox:secubox /etc/secubox
ExecStart=/usr/bin/python3 -m uvicorn api.main:app --uds /run/secubox/<module>.sock --log-level warning
Restart=on-failure
RestartSec=5
NoNewPrivileges=true
LogsDirectory=secubox
LogsDirectoryMode=0755
ReadWritePaths=/run/secubox /var/lib/secubox /etc/secubox /var/log/secubox
[Install]
WantedBy=multi-user.target
```
### `debian/postinst` template
```bash
#!/bin/sh
set -e
case "$1" in
configure)
getent group secubox >/dev/null || groupadd --system secubox
getent passwd secubox >/dev/null || useradd --system --gid secubox \
--home /var/lib/secubox --no-create-home --shell /usr/sbin/nologin secubox
install -d -m 0770 -o root -g secubox /etc/secubox
install -d -m 0755 -o secubox -g secubox /var/lib/secubox/<module>
install -d -m 0755 -o secubox -g secubox /var/log/secubox
# Copy example config if no live config exists
if [ ! -f /etc/secubox/<module>.toml ] && [ -f /etc/secubox/<module>.toml.example ]; then
cp /etc/secubox/<module>.toml.example /etc/secubox/<module>.toml
chmod 640 /etc/secubox/<module>.toml
chown root:secubox /etc/secubox/<module>.toml
fi
# Reload nginx if running (vhost snippet was just installed)
if systemctl is-active --quiet nginx 2>/dev/null; then
nginx -t >/dev/null 2>&1 && systemctl reload nginx 2>/dev/null || true
fi
systemctl daemon-reload 2>/dev/null || true
systemctl enable secubox-<module>.service 2>/dev/null || true
# Do not start the FastAPI before the LXC is provisioned; ctl will start it
# at the end of `<module>ctl install`.
;;
esac
#DEBHELPER#
exit 0
```
### `lib/<module>/install-lxc.sh` template
Idempotent: safe to re-run. Mirrors the structure of `packages/secubox-mail/lib/mail/install-mail-lxc.sh` (existing reference).
```bash
#!/usr/bin/env bash
set -euo pipefail
readonly LXC_NAME="${SECUBOX_LXC_NAME:-<module>}"
readonly LXC_IP="${SECUBOX_LXC_IP:-10.100.0.XX}"
readonly LXC_PATH="${SECUBOX_LXC_PATH:-/data/lxc}"
readonly DEBIAN_SUITE="bookworm"
# 1. Bail out if LXC already exists and is healthy
if lxc-info -n "$LXC_NAME" -P "$LXC_PATH" 2>/dev/null | grep -q '^State: *RUNNING'; then
echo "[install-lxc] $LXC_NAME already running — nothing to do"
exit 0
fi
# 2. Create the LXC if missing
if [ ! -d "$LXC_PATH/$LXC_NAME" ]; then
lxc-create -n "$LXC_NAME" -t debian -P "$LXC_PATH" -- -r "$DEBIAN_SUITE"
fi
# 3. Wire static IP into LXC config (10.100.0.X on br-secubox)
cat > "$LXC_PATH/$LXC_NAME/config" <<EOF
lxc.uts.name = $LXC_NAME
lxc.net.0.type = veth
lxc.net.0.link = br-secubox
lxc.net.0.flags = up
lxc.net.0.ipv4.address = $LXC_IP/24
lxc.net.0.ipv4.gateway = 10.100.0.1
lxc.rootfs.path = dir:$LXC_PATH/$LXC_NAME/rootfs
lxc.include = /usr/share/lxc/config/common.conf
lxc.apparmor.profile = generated
lxc.start.auto = 1
EOF
# 4. Start, wait for network
lxc-start -n "$LXC_NAME" -P "$LXC_PATH"
for i in $(seq 1 30); do
lxc-attach -n "$LXC_NAME" -P "$LXC_PATH" -- ping -c1 -W1 10.100.0.1 >/dev/null 2>&1 && break
sleep 1
done
# 5. Install the daemon (module-specific section — see per-module sections below)
lxc-attach -n "$LXC_NAME" -P "$LXC_PATH" -- bash -c '
apt-get update -q
DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
<module-specific packages>
'
# 6. Provision (dashboards / peer lists / keys)
"$(dirname "$0")/provision/run.sh"
# 7. Mark provisioned
touch /var/lib/secubox/<module>/.lxc-provisioned
```
---
## 2. CTL grammar fit
Per `docs/grammar.md`, all three modules get a new entry in the verbs table. Each CTL exposes the three-fold introspection (`components`, `status`, `access`) **plus** module-specific noun/verb pairs.
### `grafanactl` — OPS MONITORING layer (sister to `healthctl`)
| Noun | Verb |
|---|---|
| `components` | `status`, `list` |
| `status` | (no verb — prints overall state) |
| `access` | `list`, `show <name>` |
| `dashboard` | `list`, `add <file.json>`, `remove <uid>`, `export <uid>` |
| `datasource` | `list`, `add <toml>`, `remove <name>`, `test <name>` |
| `alert` | `list`, `mute <id>`, `unmute <id>` |
| `user` | `list`, `add <name> <role>`, `remove <name>`, `passwd <name>` |
| `api-key` | `list`, `create <name> <role>`, `revoke <id>` |
Help line for the grammar table addition:
```text
| OPS MONITORING | `grafanactl dashboard list/add/remove/export` | #229 (this) |
```
### `yacyctl` — new layer: SEARCH (independent search-engine sovereignty, fits Punk Exposure ethos)
| Noun | Verb |
|---|---|
| `components` | `status`, `list` |
| `status` | (no verb) |
| `access` | `list`, `show` |
| `peer` | `list`, `add <url>`, `remove <hash>`, `status <hash>` |
| `index` | `status`, `build <urls.txt>`, `clear`, `optimize` |
| `query` | `test <terms>`, `count <terms>` |
| `blacklist` | `list`, `add <pattern>`, `remove <pattern>` |
| `crawler` | `list`, `start <profile>`, `stop <id>`, `schedule <profile> <cron>` |
Help line:
```text
| SEARCH | `yacyctl index status/build/clear/optimize` | #230 (this) |
```
### `rustdeskctl` — new layer: REMOTE-ACCESS (peer-mediated remote desktop, also Punk Exposure)
| Noun | Verb |
|---|---|
| `components` | `status`, `list` |
| `status` | (no verb) |
| `access` | `list`, `show` |
| `peer` | `list`, `add <id> <pubkey>`, `remove <id>` |
| `relay` | `status`, `restart`, `log tail` |
| `key` | `show`, `rotate` (generates a new keypair, broadcasts to peers) |
| `session` | `list`, `kill <id>` |
Help line:
```text
| REMOTE-ACCESS | `rustdeskctl peer add/remove/list` | #231 (this) |
```
### Three-fold output (all modules)
Each `<module>ctl components`, `status`, and `access` returns JSON to stdout when `--json` is passed, formatted text otherwise. Schema mirrors existing `giteactl`/`healthctl`:
```json
{
"module": "grafana",
"version": "1.0.0",
"components": [
{"name": "lxc", "state": "running", "detail": "10.100.0.70 — uptime 3d12h"},
{"name": "daemon", "state": "running", "detail": "grafana-server 10.4.2, port 3000"},
{"name": "host-api", "state": "running", "detail": "uvicorn on /run/secubox/grafana.sock"}
]
}
```
---
## 3. Per-module specifics
### 3a. secubox-grafana
**LXC daemon install (inside LXC):**
```bash
apt-get install -y --no-install-recommends \
apt-transport-https software-properties-common wget gnupg ca-certificates
wget -qO - https://apt.grafana.com/gpg.key | apt-key add -
echo "deb https://apt.grafana.com stable main" > /etc/apt/sources.list.d/grafana.list
apt-get update -q
apt-get install -y --no-install-recommends grafana
systemctl enable grafana-server
```
**Provisioning (from `lib/grafana/provision/`):**
- Datasources: `secubox-metrics` (prometheus on `127.0.0.1:9090` exposed to LXC via host bridge), `crowdsec-metrics`, `journald-loki` (if loki later).
- Dashboards (JSON, ship in package):
- `nftables.json` — drops/accepts per chain/rule, geo overlay
- `crowdsec.json` — alerts per scenario, decisions over time
- `suricata.json` — alerts by severity, top src IPs
- `cookie-audit.json` — RGPD/ePrivacy violations from cookie-audit ledger
- `mitmproxy-waf.json` — Set-Cookie inspections + block decisions
- `secubox-services.json` — secubox-* systemd unit health (vital + non-vital)
**Web UI (`www/grafana/index.html`):**
SecuBox-themed wrapper page that opens the grafana UI inside an iframe sandbox (no chrome). Top bar: SecuBox logo, links back to `/`, mode switcher. Below: full-bleed iframe at `https://<host>/grafana/d/...`. Native Grafana is hidden under the SecuBox menu so the user navigates dashboards via SecuBox menu.d.
**nginx vhost (`nginx/grafana.conf`):**
```nginx
# /etc/nginx/secubox.d/grafana.conf
location /api/v1/grafana/ {
proxy_pass http://unix:/run/secubox/grafana.sock:/;
include /etc/nginx/snippets/secubox-proxy.conf;
}
# Grafana native UI (iframe target)
location /grafana/ {
proxy_pass http://10.100.0.70:3000/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# Grafana live websocket
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
```
Grafana `serve_from_sub_path` must be set true in `/etc/grafana/grafana.ini`:
```ini
[server]
domain = <vhost>
root_url = https://<vhost>/grafana/
serve_from_sub_path = true
```
### 3b. secubox-yacy
**LXC daemon install:**
```bash
apt-get install -y --no-install-recommends \
default-jre-headless wget ca-certificates unzip
mkdir -p /opt/yacy && cd /opt/yacy
wget -q https://release.yacy.net/yacy_v1.940_20240319_10001.tar.gz
tar xzf yacy_v1.940_*.tar.gz --strip-components=1
useradd -r -s /bin/false yacy
chown -R yacy:yacy /opt/yacy
cat > /etc/systemd/system/yacy.service <<'EOF'
[Unit]
Description=YaCy P2P Search
After=network.target
[Service]
Type=simple
User=yacy
WorkingDirectory=/opt/yacy
ExecStart=/opt/yacy/startYACY.sh -d
ExecStop=/opt/yacy/stopYACY.sh
Restart=on-failure
[Install]
WantedBy=multi-user.target
EOF
systemctl enable yacy
```
**Provisioning:**
- Pre-seed peer list from `lib/yacy/provision/peers.seed.txt`
- Default crawl profile: `secubox-*.in` domains, depth 2
- Blacklist seed: `lib/yacy/provision/blacklist.txt` (CrowdSec banlist excerpt)
- Admin password generated by `<module>ctl install` (firstboot-style argon2-hash to `/etc/secubox/yacy.toml`)
**WebUI wrapper:** same iframe pattern as grafana.
**nginx vhost:**
```nginx
location /api/v1/yacy/ {
proxy_pass http://unix:/run/secubox/yacy.sock:/;
include /etc/nginx/snippets/secubox-proxy.conf;
}
location /yacy/ {
proxy_pass http://10.100.0.80:8090/;
proxy_set_header Host $host;
proxy_http_version 1.1;
}
```
### 3c. secubox-rustdesk
**LXC daemon install:**
```bash
apt-get install -y --no-install-recommends wget ca-certificates
cd /opt
wget -q https://github.com/rustdesk/rustdesk-server/releases/download/1.1.11/rustdesk-server-linux-amd64.zip
unzip rustdesk-server-linux-amd64.zip
useradd -r -s /bin/false rustdesk
mkdir /var/lib/rustdesk
chown rustdesk:rustdesk /var/lib/rustdesk /opt/amd64
# Two systemd units:
cat > /etc/systemd/system/rustdesk-hbbs.service <<'EOF'
[Unit]
Description=RustDesk Signal Server (hbbs)
After=network.target
[Service]
Type=simple
User=rustdesk
WorkingDirectory=/var/lib/rustdesk
ExecStart=/opt/amd64/hbbs
Restart=on-failure
[Install]
WantedBy=multi-user.target
EOF
cat > /etc/systemd/system/rustdesk-hbbr.service <<'EOF'
[Unit]
Description=RustDesk Relay Server (hbbr)
After=network.target rustdesk-hbbs.service
[Service]
Type=simple
User=rustdesk
WorkingDirectory=/var/lib/rustdesk
ExecStart=/opt/amd64/hbbr
Restart=on-failure
[Install]
WantedBy=multi-user.target
EOF
systemctl enable rustdesk-hbbs rustdesk-hbbr
```
**Exposure** — *deviates from grafana/yacy*: RustDesk uses UDP (21116/udp) which mitmproxy can't inspect. Use HAProxy stream-mode TCP+UDP on the host edge:
```text
frontend rustdesk-stream
mode tcp
bind :21115
bind :21116
bind :21117
bind :21118
bind :21119
default_backend rustdesk-lxc
backend rustdesk-lxc
mode tcp
server rustdesk 10.100.0.90 check
```
Plus a separate UDP socket forwarded via `socat` or `iptables -t nat -A PREROUTING ... -p udp --dport 21116 -j DNAT --to 10.100.0.90:21116` (use whichever is already idiomatic in `secubox-haproxy`; check `packages/secubox-haproxy/lib/` first).
Web admin UI (port 21118) still routes through nginx like the others:
```nginx
location /api/v1/rustdesk/ {
proxy_pass http://unix:/run/secubox/rustdesk.sock:/;
include /etc/nginx/snippets/secubox-proxy.conf;
}
location /rustdesk/ {
proxy_pass http://10.100.0.90:21114/;
proxy_http_version 1.1;
}
```
---
## 4. Slipstream into the next release
Once each package's `debian/control` is present and `dpkg-buildpackage -us -uc -b` succeeds locally:
- `.github/workflows/build-packages.yml` auto-discovers via `find packages/secubox-*/debian/control` and adds the new packages to the matrix on next push. No workflow edit required.
- `image/build-image.sh` slipstream loop (`cp /tmp/secubox-debs/secubox-*.deb`) picks up the new `_all.deb` files automatically.
- Verify: after the next `gh workflow run build-packages.yml --ref master`, the run summary should show three new `build (secubox-{grafana,yacy,rustdesk}, amd64)` jobs ending in `success`.
- Next image build (`build-image.yml`) then includes the three new packages in `dpkg -l 'secubox-*'`.
The new modules **do not need their LXC pre-built** in the image. Operator installs them post-firstboot via:
```bash
grafanactl install # idempotent — runs lib/grafana/install-lxc.sh
yacyctl install
rustdeskctl install
```
This avoids inflating the image with three LXC rootfs trees (~600 MB total) when most operators won't need all three.
---
## 5. Tasks (TDD, frequent commits, one PR per module)
Each module gets its own issue + worktree + PR (per `feedback_no_unprompted_prs` memory — opening PRs unprompted needs per-PR approval; this plan is the prompt). Strongly recommended order:
1. **#229 — secubox-grafana** (most directly useful, drives operator adoption of the SecuBox stack)
2. **#230 — secubox-yacy** (independent of any other module)
3. **#231 — secubox-rustdesk** (most divergent due to UDP/HAProxy-stream — leave for last so the grafana+yacy patterns are settled)
Per-module task breakdown follows. Each `[ ]` is a single 2-5 minute action.
---
### Module 1: secubox-grafana
#### Task G1: package skeleton + debian/control
**Files:**
- Create: `packages/secubox-grafana/debian/{changelog,control,rules,install,postinst,prerm,secubox-grafana.service}`
- Create: `packages/secubox-grafana/conf/grafana.toml.example`
- Create: `packages/secubox-grafana/README.md`
- [ ] **Step 1: Write `debian/changelog`**
```text
secubox-grafana (1.0.0-1~bookworm1) bookworm; urgency=medium
* Initial release: Grafana OSS in LXC for SecuBox security dashboards.
* Three-fold CTL grammar (grafanactl: components, status, access).
* Dashboard verbs: list/add/remove/export.
* Datasource verbs: list/add/remove/test.
* Pre-provisioned dashboards: nftables, crowdsec, suricata, cookie-audit,
mitmproxy-waf, secubox-services.
* Closes: #229
-- Gerald KERMA <devel@cybermind.fr> Wed, 20 May 2026 10:00:00 +0200
```
- [ ] **Step 2: Write `debian/control`** — use the template from §1, module-specific `Depends` from §1.
- [ ] **Step 3: Write `debian/rules`**:
```makefile
#!/usr/bin/make -f
%:
dh $@ --with python3
override_dh_auto_install:
install -d debian/secubox-grafana/usr/lib/secubox/grafana
cp -r api debian/secubox-grafana/usr/lib/secubox/grafana/
install -d debian/secubox-grafana/usr/share/secubox/grafana/provision
[ -d lib/grafana/provision ] && cp -r lib/grafana/provision/. debian/secubox-grafana/usr/share/secubox/grafana/provision/ || true
install -d debian/secubox-grafana/usr/sbin
install -m 755 sbin/grafanactl debian/secubox-grafana/usr/sbin/grafanactl
install -d debian/secubox-grafana/usr/share/secubox/www
[ -d www ] && cp -r www/. debian/secubox-grafana/usr/share/secubox/www/ || true
install -d debian/secubox-grafana/usr/share/secubox/menu.d
[ -d menu.d ] && cp -r menu.d/. debian/secubox-grafana/usr/share/secubox/menu.d/ || true
install -d debian/secubox-grafana/etc/nginx/secubox.d
[ -f nginx/grafana.conf ] && cp nginx/grafana.conf debian/secubox-grafana/etc/nginx/secubox.d/ || true
install -d debian/secubox-grafana/etc/secubox
[ -f conf/grafana.toml.example ] && cp conf/grafana.toml.example debian/secubox-grafana/etc/secubox/grafana.toml.example || true
install -d debian/secubox-grafana/usr/share/secubox/lib/grafana
[ -d lib/grafana ] && cp -r lib/grafana/. debian/secubox-grafana/usr/share/secubox/lib/grafana/ || true
```
- [ ] **Step 4: Write `debian/postinst`** — copy from §1 template, substitute `<module>=grafana`.
- [ ] **Step 5: Write `debian/prerm`**:
```sh
#!/bin/sh
set -e
case "$1" in
remove|deconfigure|upgrade)
systemctl stop secubox-grafana.service 2>/dev/null || true
systemctl disable secubox-grafana.service 2>/dev/null || true
;;
esac
#DEBHELPER#
exit 0
```
- [ ] **Step 6: Write `debian/secubox-grafana.service`** — use the template from §1.
- [ ] **Step 7: Local build smoke test**
Run: `cd packages/secubox-grafana && dpkg-buildpackage -us -uc -b`
Expected: `secubox-grafana_1.0.0-1~bookworm1_all.deb` produced under `..` (the parent dir), no errors.
- [ ] **Step 8: Commit**
```bash
git add packages/secubox-grafana/{debian,conf,README.md}
git commit -m "feat(secubox-grafana): package skeleton (ref #229)"
```
#### Task G2: CTL tool with three-fold introspection
**Files:**
- Create: `packages/secubox-grafana/sbin/grafanactl`
- Create: `packages/secubox-grafana/tests/helpers.bash`
- Create: `packages/secubox-grafana/tests/test-grafanactl-threefold.bats`
- [ ] **Step 1: Write the failing test**
```bats
@test "grafanactl components prints lxc/daemon/host-api with JSON output" {
run grafanactl components --json
[ "$status" -eq 0 ]
echo "$output" | jq -e '.components | map(.name) | contains(["lxc","daemon","host-api"])'
}
@test "grafanactl status returns module=grafana and version" {
run grafanactl status --json
[ "$status" -eq 0 ]
echo "$output" | jq -e '.module == "grafana"'
echo "$output" | jq -e '.version != ""'
}
@test "grafanactl access list shows the public hostname when configured" {
HOSTNAME=grafana.gk2.secubox.in
echo 'public_hostname = "grafana.gk2.secubox.in"' > /etc/secubox/grafana.toml
run grafanactl access list --json
[ "$status" -eq 0 ]
echo "$output" | jq -e '.access[] | select(.url | contains("grafana.gk2.secubox.in"))'
}
```
- [ ] **Step 2: Run the test, expect failure with `command not found: grafanactl`**.
- [ ] **Step 3: Write the minimal `sbin/grafanactl`** with `components`, `status`, `access` verbs only. Pattern: mirror `packages/secubox-gitea/sbin/giteactl`'s three-fold section.
- [ ] **Step 4: Re-run test, expect PASS**.
- [ ] **Step 5: Commit**.
#### Task G3: `grafanactl install` — provision the LXC
**Files:**
- Modify: `packages/secubox-grafana/sbin/grafanactl` (add `install` verb)
- Create: `packages/secubox-grafana/lib/grafana/install-lxc.sh`
- Create: `packages/secubox-grafana/tests/test-grafanactl-install.bats` (with stubbed `lxc-create`/`lxc-attach` via PATH shims)
- [ ] **Step 1: Write the failing test**`grafanactl install` should call `lxc-create` and `lxc-attach` with the right args, and write `/var/lib/secubox/grafana/.lxc-provisioned` on success.
- [ ] **Step 2: Implement `install-lxc.sh`** using the §1 template with `<module>=grafana`, `IP=10.100.0.70`, `PORTS=3000`.
- [ ] **Step 3: Add `cmd_install()` to `grafanactl`** that just execs the script with sensible env.
- [ ] **Step 4: Run test, expect PASS**.
- [ ] **Step 5: Commit**.
#### Task G4: FastAPI host control plane
**Files:**
- Create: `packages/secubox-grafana/api/{__init__.py,main.py,models.py}`
- Create: `packages/secubox-grafana/tests/test-api.py` (pytest)
- [ ] **Step 1: Write the failing test** — pytest fixture spawns uvicorn on a temp socket, asserts `GET /status` returns `{"module": "grafana", "version": ..., "components": [...]}`.
- [ ] **Step 2: Write minimal `api/main.py`** — three endpoints: `GET /status`, `GET /components`, `GET /access`. Each shells out to `grafanactl <verb> --json` and returns the parsed JSON. (Same trick as `secubox-mail`'s API.)
- [ ] **Step 3: Run test, expect PASS**.
- [ ] **Step 4: Commit**.
#### Task G5: dashboard / datasource verbs
- [ ] **Step 1: Write failing bats** for `grafanactl dashboard list/add/remove/export`.
- [ ] **Step 2: Implement against the Grafana HTTP API** (`http://10.100.0.70:3000/api/dashboards/db` etc.) inside `grafanactl`. Use a stored API key from `/etc/secubox/grafana.toml` (created by `install` step).
- [ ] **Step 3: Same for `datasource list/add/remove/test`**.
- [ ] **Step 4: Pass tests, commit per noun**.
#### Task G6: pre-provisioned dashboards
**Files:**
- Create: `packages/secubox-grafana/lib/grafana/provision/dashboards/{nftables,crowdsec,suricata,cookie-audit,mitmproxy-waf,secubox-services}.json`
- Create: `packages/secubox-grafana/lib/grafana/provision/datasources/secubox-metrics.yaml`
- Modify: `lib/grafana/install-lxc.sh` to copy provisioning into `/etc/grafana/provisioning/`
- [ ] **Step 1**: write a minimal `secubox-services.json` dashboard JSON (single time-series panel: `systemd_unit_state{name=~"secubox-.*"}`).
- [ ] **Step 2**: provisioning yaml for the secubox-metrics datasource (prometheus on `http://10.100.0.X:9090` — verify port from `secubox-metrics.service`).
- [ ] **Step 3**: wire provisioning into `install-lxc.sh` (copies during step 5 of the template).
- [ ] **Step 4**: end-to-end manual test on a dev VM — `grafanactl install`, then `curl https://localhost/grafana/login` → grafana page; `grafanactl dashboard list` → shows the provisioned ones.
- [ ] **Step 5: Commit**.
#### Task G7: web UI + nginx + menu
**Files:**
- Create: `packages/secubox-grafana/www/grafana/{index.html,settings.html}`
- Create: `packages/secubox-grafana/nginx/grafana.conf`
- Create: `packages/secubox-grafana/menu.d/50-grafana.json`
- [ ] **Step 1**: `www/grafana/index.html` — SecuBox-wrapped iframe per the DESIGN-CHARTER palette. Loads `/grafana/d/secubox-services/secubox-services-overview` in iframe.
- [ ] **Step 2**: `nginx/grafana.conf` — see §3a snippet.
- [ ] **Step 3**: `menu.d/50-grafana.json`:
```json
{
"title": "Grafana",
"subtitle": "Security metrics",
"icon": "fa-chart-line",
"url": "/grafana/",
"section": "monitoring",
"order": 50,
"module": "grafana"
}
```
- [ ] **Step 4**: install on dev VM, click through SecuBox menu → Grafana iframe loads.
- [ ] **Step 5: Commit**.
#### Task G8: PR + CI
- [ ] **Step 1**: open PR `feat(secubox-grafana): security-metrics dashboard module (closes #229)`.
- [ ] **Step 2**: build-packages.yml auto-runs on PR open; verify `build (secubox-grafana, amd64)` passes.
- [ ] **Step 3**: request user review.
---
### Module 2: secubox-yacy
Same structure as Grafana with the YaCy-specific deltas from §3b. Tasks mirror G1G8 with `<module>=yacy`, `IP=10.100.0.80`, daemon=YaCy + JVM, provisioning = peer list + crawl profile + blacklist.
**Key deviation from Grafana:** YaCy doesn't expose a Prometheus-friendly metrics endpoint by default; the `grafanactl datasource add` integration with YaCy is OUT-OF-SCOPE for this plan (would require a YaCy → Prometheus exporter, separate issue).
---
### Module 3: secubox-rustdesk
Same structure with the RustDesk-specific deltas from §3c. Major deviations:
- **HAProxy stream-mode config** instead of mitmproxy route. Add to `packages/secubox-haproxy/conf/streams/rustdesk.cfg` (or wherever `haproxyctl vhost add` writes — verify). Use `haproxyctl stream add rustdesk-stream` if such a verb exists; else extend `haproxyctl` first in a separate prereq PR.
- **UDP forwarding:** decide between `nftables` (preferred, matches the project's stack) or `socat`. Add to the rustdesk `install-lxc.sh`.
- **Two systemd units** in the LXC (`rustdesk-hbbs`, `rustdesk-hbbr`) — `rustdeskctl components` must list both as separate components, not aggregated.
---
## 6. Documentation updates after merge
- [ ] `docs/grammar.md` — append three new rows to the canonical verbs table:
- `OPS MONITORING / grafanactl dashboard list/add/remove/export / #229`
- `SEARCH / yacyctl index status/build/clear/optimize / #230`
- `REMOTE-ACCESS / rustdeskctl peer add/remove/list / #231`
- [ ] `HOWTO-grammar.md` — no change (recipe is unchanged; the three new modules just demonstrate it).
- [ ] `wiki/CTL-Grammar.md` — sync with `docs/grammar.md`.
- [ ] `.claude/MIGRATION-MAP.md` — add three new rows (✅ when delivered).
- [ ] `.claude/HISTORY.md` — dated entry summarising all three.
## 7. Self-review checklist (run before declaring the plan ready to execute)
- [ ] **Spec coverage:** every requirement in the user prompt (Grafana for security metrics, YaCy search, both in LXC, web UI, CTL tool, API, deps, slipstream in next release) is addressed by at least one task.
- [ ] **Rustdesk addition (mid-flight scope add):** all rustdesk-specific deltas (UDP, HAProxy stream mode, two daemons) are explicitly called out in §3c and Module 3 task list.
- [ ] **No placeholders:** every CTL noun/verb has a real spec (no "TODO" or "to be decided").
- [ ] **Type consistency:** the JSON schema in §2 ("three-fold output") matches what `grafanactl`, `yacyctl`, `rustdeskctl` will produce.
- [ ] **IP allocation:** `.70`, `.80`, `.90` confirmed free against current `grep -rohE '10\.100\.0\.[0-9]+' packages/ image/` output (verified at plan-write time; re-verify at T0 of each module).
---
## Execution handoff
Plan complete and saved to `docs/superpowers/plans/2026-05-20-grafana-yacy-rustdesk-lxc-modules.md`.
Two execution options:
1. **Subagent-driven** (recommended) — dispatch a fresh subagent per task, two-stage review (spec compliance + code quality) per task, three independent PRs in sequence.
2. **Inline execution** — drive each task in the main session via `superpowers:executing-plans`, batch the trivial tasks (skeleton scaffolding) with checkpoints between modules.
Strongly recommend **(1)** because each module is ~8 TDD tasks × 5 commits and benefits from independent reviewer-style checks per task. Total estimated cycle: ~3 working days end-to-end, with build-packages + build-image CI runs gating each merge.
Awaiting user decision on:
- File three separate issues (#229, #230, #231) now, or just #229 first?
- Subagent-driven vs inline execution?
- LXC IP scheme — `10.100.0.{70,80,90}` confirmed, or different range preferred?