mirror of
https://github.com/CyberMind-FR/secubox-deb.git
synced 2026-07-28 21:17:36 +00:00
Merge #919: secubox-torrent v2.0 WebTorrent streaming (Closes #917)
Some checks are pending
License Headers / check (push) Waiting to run
Some checks are pending
License Headers / check (push) Waiting to run
secubox-torrent v2.0 — WebTorrent streaming pivot (Closes #917)
This commit is contained in:
CyberMind
GitHub
commit
c6f2742d96
939
docs/superpowers/plans/2026-07-28-secubox-torrent-webtorrent.md
Normal file
939
docs/superpowers/plans/2026-07-28-secubox-torrent-webtorrent.md
Normal file
|
|
@ -0,0 +1,939 @@
|
|||
# secubox-torrent v2.0 — WebTorrent Streaming — Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Pivot the dormant `secubox-torrent` module from Transmission to a WebTorrent streaming service (paste magnet → watch in browser, ephemeral-by-default + Keep), running Node.js in an isolated LXC.
|
||||
|
||||
**Architecture:** A Node.js service inside a dedicated `torrent` LXC runs the WebTorrent engine (hybrid WebRTC+BitTorrent), an HTTP Range streaming server, a SQLite library with a purge sweep, and a Fastify API serving the player webui. The host only provisions the LXC and proxies/authenticates a vhost through HAProxy→sbxwaf→nginx.
|
||||
|
||||
**Tech Stack:** Node.js LTS (bookworm), `webtorrent` + `@roamhq/wrtc` (WebRTC hybrid), `fastify`, `better-sqlite3`; tests via Node built-in `node --test` (no test-framework dependency); Debian packaging (arch: all, LXC provisioned at postinst).
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- Package name stays `secubox-torrent`; version `2.0.0-1~bookworm1`; the 1.0.0 Transmission code (api/main.py Python) is removed.
|
||||
- Public vhost host `torrent.gk2.secubox.in` is preserved.
|
||||
- Platform: Debian bookworm **arm64**. Media/state on the USB SSD at `/data` — never eMMC.
|
||||
- LXC pattern matches peertube/podcaster: `LXC_PATH=/data/lxc`, bridge `br-lxc`, gateway `10.100.0.1`, sentinel-file idempotency, `la()` = `lxc-attach -n <name> -P <path> --`.
|
||||
- Torrent engine runs ONLY inside the LXC (untrusted peers). The host runs no torrent logic.
|
||||
- Node service source lives under `lxc/app/` in the package and is copied into the LXC at provision time.
|
||||
- Tests are `node --test` files named `*.test.js`, run from `lxc/app/`, use fakes — NEVER real BitTorrent/WebRTC/network.
|
||||
- Auth: the LXC service trusts the proxied request; the host vhost enforces `sbx_token` via `auth_request` (master users gk2 + admin). No auth logic inside the Node service.
|
||||
- Config: `/etc/secubox/torrent.toml` on the host; the relevant values are passed into the LXC env at service start.
|
||||
- SecuBox webui look per `.claude/WEBUI-PANEL-GUIDELINES.md` (cyan hybrid-dark, reads `localStorage.sbx_token`).
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
packages/secubox-torrent/
|
||||
├── lxc/
|
||||
│ ├── app/ ← Node service (copied into LXC)
|
||||
│ │ ├── package.json ← deps + "test": "node --test"
|
||||
│ │ ├── engine.js ← WebTorrent wrap
|
||||
│ │ ├── engine.test.js
|
||||
│ │ ├── stream.js ← HTTP Range handler
|
||||
│ │ ├── stream.test.js
|
||||
│ │ ├── library.js ← better-sqlite3 + purge
|
||||
│ │ ├── library.test.js
|
||||
│ │ ├── api.js ← Fastify app factory
|
||||
│ │ ├── api.test.js
|
||||
│ │ ├── server.js ← wires engine+stream+library+api, reads env
|
||||
│ │ └── fakes.js ← FakeWebTorrent/FakeTorrent/FakeFile
|
||||
│ ├── install-lxc.sh ← idempotent LXC provisioner
|
||||
│ └── secubox-torrent.service ← systemd unit INSIDE the LXC
|
||||
├── www/torrent/index.html ← player webui (replaces old panel)
|
||||
├── nginx/torrent.conf ← host vhost → LXC
|
||||
├── nft/torrent-egress.nft ← host egress drop-in for the LXC veth
|
||||
├── menu.d/…-torrent.json ← dashboard entry (kept)
|
||||
├── conf/torrent.toml ← default config
|
||||
├── README.md
|
||||
└── debian/ ← control/rules/postinst/postrm/changelog
|
||||
```
|
||||
|
||||
Removed: `packages/secubox-torrent/api/main.py`, `systemd/secubox-torrent.service` (old host FastAPI), and Transmission references.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: LXC skeleton + wrtc-arm64 spike
|
||||
|
||||
**Files:**
|
||||
- Create: `packages/secubox-torrent/lxc/app/package.json`
|
||||
- Create: `packages/secubox-torrent/lxc/app/wrtc-probe.js`
|
||||
- Create: `packages/secubox-torrent/lxc/install-lxc.sh` (skeleton: create LXC + Node)
|
||||
- Create: `packages/secubox-torrent/lxc/SPIKE-RESULT.md`
|
||||
|
||||
**Interfaces:**
|
||||
- Produces: the pinned npm dependency set (whether `@roamhq/wrtc` is included) and `WEBRTC_AVAILABLE` (bool) recorded in `SPIKE-RESULT.md`, consumed by every later task's `package.json` + `engine.js`.
|
||||
|
||||
- [ ] **Step 1: Write package.json**
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "secubox-torrent",
|
||||
"version": "2.0.0",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"engines": { "node": ">=18" },
|
||||
"scripts": { "test": "node --test" },
|
||||
"dependencies": {
|
||||
"webtorrent": "^2.5.1",
|
||||
"@roamhq/wrtc": "^0.8.0",
|
||||
"fastify": "^4.28.1",
|
||||
"better-sqlite3": "^11.3.0"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Write wrtc-probe.js** (decides WebRTC availability)
|
||||
|
||||
```javascript
|
||||
// Exits 0 and prints WEBRTC_AVAILABLE=true if @roamhq/wrtc loads a
|
||||
// RTCPeerConnection on this arch; exits 0 with =false otherwise (fallback).
|
||||
try {
|
||||
const wrtc = await import('@roamhq/wrtc');
|
||||
const pc = new wrtc.default.RTCPeerConnection();
|
||||
pc.close();
|
||||
console.log('WEBRTC_AVAILABLE=true');
|
||||
} catch (e) {
|
||||
console.log('WEBRTC_AVAILABLE=false');
|
||||
console.error('wrtc load failed:', e.message);
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Write install-lxc.sh skeleton** (create LXC + install Node, idempotent)
|
||||
|
||||
```bash
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
readonly LXC_NAME="${SECUBOX_LXC_NAME:-torrent}"
|
||||
readonly LXC_IP="${SECUBOX_LXC_IP:-10.100.0.130}"
|
||||
readonly LXC_PATH="${SECUBOX_LXC_PATH:-/data/lxc}"
|
||||
readonly LXC_BRIDGE="${SECUBOX_LXC_BRIDGE:-br-lxc}"
|
||||
readonly LXC_GW="${SECUBOX_LXC_GW:-10.100.0.1}"
|
||||
readonly DATA_DIR="${SECUBOX_DATA_DIR:-/data/torrent}"
|
||||
readonly STATE_DIR="${SECUBOX_STATE_DIR:-/var/lib/secubox/torrent}"
|
||||
readonly SENTINEL="$STATE_DIR/.lxc-provisioned"
|
||||
log() { printf '[torrent-install] %s\n' "$*"; }
|
||||
la() { lxc-attach -n "$LXC_NAME" -P "$LXC_PATH" -- "$@"; }
|
||||
|
||||
mkdir -p "$STATE_DIR" "$DATA_DIR"
|
||||
if [ -f "$SENTINEL" ]; then log "already provisioned; skipping create"; else
|
||||
lxc-create -n "$LXC_NAME" -P "$LXC_PATH" -t debian -- -r bookworm -a arm64
|
||||
# network: static IP on br-lxc (config appended to the container config)
|
||||
cat >> "$LXC_PATH/$LXC_NAME/config" <<EOF
|
||||
lxc.net.0.type = veth
|
||||
lxc.net.0.link = $LXC_BRIDGE
|
||||
lxc.net.0.flags = up
|
||||
lxc.net.0.ipv4.address = $LXC_IP/24
|
||||
lxc.net.0.ipv4.gateway = $LXC_GW
|
||||
EOF
|
||||
lxc-start -n "$LXC_NAME" -P "$LXC_PATH"
|
||||
sleep 5
|
||||
la apt-get update
|
||||
la apt-get install -y --no-install-recommends nodejs npm ca-certificates
|
||||
touch "$SENTINEL"
|
||||
fi
|
||||
log "node version: $(la node --version 2>/dev/null || echo MISSING)"
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run the spike (manual/CI, on gk2 or an arm64 runner)**
|
||||
|
||||
Run: provision the LXC, then inside it `npm ci` in the app dir and `node wrtc-probe.js`.
|
||||
Expected: prints `WEBRTC_AVAILABLE=true` OR `=false` without crashing npm install.
|
||||
|
||||
- [ ] **Step 5: Record SPIKE-RESULT.md + adjust package.json**
|
||||
|
||||
Write `SPIKE-RESULT.md` with the observed `WEBRTC_AVAILABLE` value, the Node version, and the arm64 npm-install outcome. **If false:** remove `@roamhq/wrtc` from `package.json` dependencies (BitTorrent-only build) and note it. This value is the `webrtc` default for `engine.js`.
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add packages/secubox-torrent/lxc/
|
||||
git commit -m "feat(torrent): LXC skeleton + wrtc-arm64 spike (ref #917)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: engine.js — WebTorrent wrap
|
||||
|
||||
**Files:**
|
||||
- Create: `packages/secubox-torrent/lxc/app/engine.js`
|
||||
- Create: `packages/secubox-torrent/lxc/app/fakes.js`
|
||||
- Test: `packages/secubox-torrent/lxc/app/engine.test.js`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `WEBRTC_AVAILABLE` from Task 1 (the `webrtc` option default).
|
||||
- Produces:
|
||||
- `class Engine { constructor({ WebTorrentCtor, downloadDir, maxActive, webrtc }) }`
|
||||
- `await engine.add(magnet) → { infohash, name, files: [{ idx, name, length, type }] }`
|
||||
- `engine.get(infohash) → torrent | null`
|
||||
- `engine.stats(infohash) → { progress, downloadSpeed, uploadSpeed, numPeers, wires: [{ type }] }`
|
||||
- `engine.remove(infohash, { deleteData }) → void`
|
||||
- `type` is the lowercased file extension without the dot (e.g. `"mp4"`).
|
||||
|
||||
- [ ] **Step 1: Write fakes.js**
|
||||
|
||||
```javascript
|
||||
export class FakeFile {
|
||||
constructor(name, length) { this.name = name; this.length = length; }
|
||||
}
|
||||
export class FakeTorrent {
|
||||
constructor(infoHash, name, files, { failMeta = false } = {}) {
|
||||
this.infoHash = infoHash; this.name = name;
|
||||
this.files = files.map((f, i) => Object.assign(new FakeFile(f.name, f.length), { idx: i }));
|
||||
this.progress = 0.1; this.downloadSpeed = 1000; this.uploadSpeed = 500;
|
||||
this.numPeers = 3; this.wires = [{ type: 'webrtc' }, { type: 'tcp' }];
|
||||
this._failMeta = failMeta; this._handlers = {};
|
||||
}
|
||||
on(ev, cb) { this._handlers[ev] = cb; if (ev === 'metadata' && !this._failMeta) queueMicrotask(cb); }
|
||||
destroy(_opts, cb) { if (cb) cb(); }
|
||||
}
|
||||
export class FakeWebTorrent {
|
||||
constructor() { this.torrents = []; }
|
||||
add(magnet, _opts, cb) {
|
||||
const t = this._next || new FakeTorrent('a'.repeat(40), 'Fake', [{ name: 'movie.mp4', length: 100 }]);
|
||||
this.torrents.push(t); if (cb) queueMicrotask(() => cb(t)); return t;
|
||||
}
|
||||
get(infohash) { return this.torrents.find(t => t.infoHash === infohash) || null; }
|
||||
destroy(cb) { if (cb) cb(); }
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Write the failing test**
|
||||
|
||||
```javascript
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { Engine } from './engine.js';
|
||||
import { FakeWebTorrent } from './fakes.js';
|
||||
|
||||
test('add returns infohash, name and typed file list', async () => {
|
||||
const eng = new Engine({ WebTorrentCtor: FakeWebTorrent, downloadDir: '/tmp/x', maxActive: 5, webrtc: true });
|
||||
const r = await eng.add('magnet:?xt=urn:btih:' + 'a'.repeat(40));
|
||||
assert.equal(r.infohash, 'a'.repeat(40));
|
||||
assert.equal(r.files[0].type, 'mp4');
|
||||
assert.equal(r.files[0].idx, 0);
|
||||
});
|
||||
|
||||
test('stats returns wire types', () => {
|
||||
const eng = new Engine({ WebTorrentCtor: FakeWebTorrent, downloadDir: '/tmp/x', maxActive: 5, webrtc: true });
|
||||
return eng.add('magnet:?xt=urn:btih:' + 'a'.repeat(40)).then(r => {
|
||||
const s = eng.stats(r.infohash);
|
||||
assert.deepEqual(s.wires.map(w => w.type).sort(), ['tcp', 'webrtc']);
|
||||
assert.equal(s.numPeers, 3);
|
||||
});
|
||||
});
|
||||
|
||||
test('maxActive cap rejects beyond limit', async () => {
|
||||
const eng = new Engine({ WebTorrentCtor: FakeWebTorrent, downloadDir: '/tmp/x', maxActive: 1, webrtc: true });
|
||||
await eng.add('magnet:?xt=urn:btih:' + 'a'.repeat(40));
|
||||
eng.client.torrents.push({ infoHash: 'b'.repeat(40) }); // simulate a second active
|
||||
await assert.rejects(() => eng.add('magnet:?xt=urn:btih:' + 'c'.repeat(40)), /max active/);
|
||||
});
|
||||
```
|
||||
|
||||
- [ ] **Step 2b: Run test to verify it fails** — `cd lxc/app && node --test engine.test.js` → FAIL (Engine not defined).
|
||||
|
||||
- [ ] **Step 3: Write engine.js**
|
||||
|
||||
```javascript
|
||||
import path from 'node:path';
|
||||
|
||||
export class Engine {
|
||||
constructor({ WebTorrentCtor, downloadDir, maxActive, webrtc }) {
|
||||
this.downloadDir = downloadDir; this.maxActive = maxActive;
|
||||
// webrtc=false (spike fallback) tells WebTorrent to skip the WebRTC transport.
|
||||
this.client = new WebTorrentCtor(webrtc ? {} : { tracker: { wrtc: false } });
|
||||
}
|
||||
add(magnet) {
|
||||
if (this.client.torrents.length >= this.maxActive) {
|
||||
return Promise.reject(new Error('max active torrents reached'));
|
||||
}
|
||||
return new Promise((resolve, reject) => {
|
||||
const to = setTimeout(() => reject(new Error('metadata timeout')), 60000);
|
||||
const t = this.client.add(magnet, { path: this.downloadDir }, () => {});
|
||||
const done = () => {
|
||||
clearTimeout(to);
|
||||
resolve({
|
||||
infohash: t.infoHash, name: t.name,
|
||||
files: t.files.map((f, i) => ({ idx: i, name: f.name, length: f.length, type: ext(f.name) })),
|
||||
});
|
||||
};
|
||||
t.on('metadata', done);
|
||||
if (t.files && t.files.length) done();
|
||||
});
|
||||
}
|
||||
get(infohash) { return this.client.get(infohash); }
|
||||
stats(infohash) {
|
||||
const t = this.get(infohash); if (!t) return null;
|
||||
return { progress: t.progress, downloadSpeed: t.downloadSpeed, uploadSpeed: t.uploadSpeed,
|
||||
numPeers: t.numPeers, wires: (t.wires || []).map(w => ({ type: w.type })) };
|
||||
}
|
||||
remove(infohash, { deleteData } = {}) {
|
||||
const t = this.get(infohash); if (t) t.destroy({ destroyStore: !!deleteData }, () => {});
|
||||
}
|
||||
}
|
||||
function ext(name) { const m = /\.([A-Za-z0-9]+)$/.exec(name); return m ? m[1].toLowerCase() : ''; }
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run test to verify it passes** — `node --test engine.test.js` → PASS.
|
||||
|
||||
- [ ] **Step 5: Commit** — `git commit -m "feat(torrent): engine.js WebTorrent wrap + fakes (ref #917)"`
|
||||
|
||||
---
|
||||
|
||||
### Task 3: stream.js — HTTP Range handler
|
||||
|
||||
**Files:**
|
||||
- Create: `packages/secubox-torrent/lxc/app/stream.js`
|
||||
- Test: `packages/secubox-torrent/lxc/app/stream.test.js`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `Engine.get(infohash)` (returns a torrent whose `.files[idx]` has `.length`, `.name`, and `.stream({ start, end }) → ReadableStream`).
|
||||
- Produces: `handleStream(engine, req, res)` where `req = { params: { infohash, fileIdx }, headers }` and `res` is a Node ServerResponse-like object. Sets status + headers, pipes the file stream. Returns nothing.
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
```javascript
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { Readable } from 'node:stream';
|
||||
import { handleStream } from './stream.js';
|
||||
|
||||
function fakeRes() {
|
||||
return { statusCode: 200, headers: {}, ended: false, body: '',
|
||||
setHeader(k, v) { this.headers[k.toLowerCase()] = v; },
|
||||
writeHead(c, h) { this.statusCode = c; Object.assign(this.headers, lower(h || {})); },
|
||||
end(s) { if (s) this.body += s; this.ended = true; } };
|
||||
}
|
||||
const lower = o => Object.fromEntries(Object.entries(o).map(([k, v]) => [k.toLowerCase(), v]));
|
||||
function fakeEngine(len = 100) {
|
||||
const file = { name: 'movie.mp4', length: len,
|
||||
stream: ({ start, end }) => Readable.from([`bytes[${start}-${end}]`]) };
|
||||
return { get: () => ({ files: [file] }) };
|
||||
}
|
||||
|
||||
test('full request returns 200 with content-length and type', async () => {
|
||||
const res = fakeRes();
|
||||
await handleStream(fakeEngine(100), { params: { infohash: 'a', fileIdx: '0' }, headers: {} }, res);
|
||||
assert.equal(res.statusCode, 200);
|
||||
assert.equal(res.headers['content-length'], 100);
|
||||
assert.match(res.headers['content-type'], /mp4/);
|
||||
});
|
||||
|
||||
test('range request returns 206 with content-range', async () => {
|
||||
const res = fakeRes();
|
||||
await handleStream(fakeEngine(100), { params: { infohash: 'a', fileIdx: '0' }, headers: { range: 'bytes=10-19' } }, res);
|
||||
assert.equal(res.statusCode, 206);
|
||||
assert.equal(res.headers['content-range'], 'bytes 10-19/100');
|
||||
assert.equal(res.headers['content-length'], 10);
|
||||
});
|
||||
|
||||
test('unsatisfiable range returns 416', async () => {
|
||||
const res = fakeRes();
|
||||
await handleStream(fakeEngine(100), { params: { infohash: 'a', fileIdx: '0' }, headers: { range: 'bytes=200-300' } }, res);
|
||||
assert.equal(res.statusCode, 416);
|
||||
});
|
||||
|
||||
test('unknown infohash returns 404', async () => {
|
||||
const res = fakeRes();
|
||||
await handleStream({ get: () => null }, { params: { infohash: 'z', fileIdx: '0' }, headers: {} }, res);
|
||||
assert.equal(res.statusCode, 404);
|
||||
});
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run to verify it fails** — `node --test stream.test.js` → FAIL.
|
||||
|
||||
- [ ] **Step 3: Write stream.js**
|
||||
|
||||
```javascript
|
||||
const MIME = { mp4: 'video/mp4', webm: 'video/webm', mkv: 'video/x-matroska',
|
||||
m4v: 'video/mp4', mp3: 'audio/mpeg', m4a: 'audio/mp4', ogg: 'audio/ogg',
|
||||
flac: 'audio/flac', wav: 'audio/wav' };
|
||||
|
||||
export async function handleStream(engine, req, res) {
|
||||
const t = engine.get(req.params.infohash);
|
||||
const idx = Number(req.params.fileIdx);
|
||||
const file = t && t.files && t.files[idx];
|
||||
if (!file) { res.writeHead(404); res.end('not found'); return; }
|
||||
const type = mimeOf(file.name);
|
||||
const total = file.length;
|
||||
const range = req.headers.range;
|
||||
if (!range) {
|
||||
res.writeHead(200, { 'Content-Length': total, 'Content-Type': type, 'Accept-Ranges': 'bytes' });
|
||||
file.stream({ start: 0, end: total - 1 }).pipe(res);
|
||||
return;
|
||||
}
|
||||
const m = /bytes=(\d+)-(\d*)/.exec(range);
|
||||
const start = m ? Number(m[1]) : 0;
|
||||
const end = m && m[2] ? Number(m[2]) : total - 1;
|
||||
if (start >= total || end >= total || start > end) {
|
||||
res.writeHead(416, { 'Content-Range': `bytes */${total}` }); res.end(); return;
|
||||
}
|
||||
res.writeHead(206, { 'Content-Range': `bytes ${start}-${end}/${total}`,
|
||||
'Content-Length': end - start + 1, 'Content-Type': type, 'Accept-Ranges': 'bytes' });
|
||||
file.stream({ start, end }).pipe(res);
|
||||
}
|
||||
function mimeOf(name) { const e = (/\.([A-Za-z0-9]+)$/.exec(name) || [])[1]; return MIME[(e || '').toLowerCase()] || 'application/octet-stream'; }
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run to verify it passes** — PASS.
|
||||
|
||||
- [ ] **Step 5: Commit** — `git commit -m "feat(torrent): stream.js HTTP Range handler (ref #917)"`
|
||||
|
||||
---
|
||||
|
||||
### Task 4: library.js — SQLite library + purge
|
||||
|
||||
**Files:**
|
||||
- Create: `packages/secubox-torrent/lxc/app/library.js`
|
||||
- Test: `packages/secubox-torrent/lxc/app/library.test.js`
|
||||
|
||||
**Interfaces:**
|
||||
- Produces:
|
||||
- `class Library { constructor(dbPath) }`
|
||||
- `lib.add({ infohash, name, magnet, path }) → void` (inserts kept=0)
|
||||
- `lib.list() → [{ infohash, name, magnet, added_at, last_played_at, kept, path }]`
|
||||
- `lib.touch(infohash) → void` (sets last_played_at = now)
|
||||
- `lib.keep(infohash, newPath) → void`; `lib.get(infohash) → row | null`; `lib.remove(infohash) → void`
|
||||
- `lib.expiredEphemeral(ttlSeconds, now) → [infohash]` (kept=0 AND last_played_at < now-ttl)
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
```javascript
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { Library } from './library.js';
|
||||
|
||||
const mk = () => new Library(':memory:');
|
||||
|
||||
test('add then list yields one ephemeral row', () => {
|
||||
const lib = mk();
|
||||
lib.add({ infohash: 'a', name: 'A', magnet: 'm', path: '/tmp/a' });
|
||||
const rows = lib.list();
|
||||
assert.equal(rows.length, 1);
|
||||
assert.equal(rows[0].kept, 0);
|
||||
});
|
||||
|
||||
test('keep flips kept and updates path', () => {
|
||||
const lib = mk();
|
||||
lib.add({ infohash: 'a', name: 'A', magnet: 'm', path: '/tmp/a' });
|
||||
lib.keep('a', '/data/torrent/library/a');
|
||||
assert.equal(lib.get('a').kept, 1);
|
||||
assert.equal(lib.get('a').path, '/data/torrent/library/a');
|
||||
});
|
||||
|
||||
test('expiredEphemeral returns only stale unkept torrents', () => {
|
||||
const lib = mk();
|
||||
lib.add({ infohash: 'old', name: 'O', magnet: 'm', path: '/tmp/o' });
|
||||
lib.add({ infohash: 'fresh', name: 'F', magnet: 'm', path: '/tmp/f' });
|
||||
lib.add({ infohash: 'kept', name: 'K', magnet: 'm', path: '/tmp/k' });
|
||||
lib.touchAt('old', 1000); lib.touchAt('fresh', 9000); lib.touchAt('kept', 1000);
|
||||
lib.keep('kept', '/data/k');
|
||||
const exp = lib.expiredEphemeral(3600, 10000); // now=10000, ttl=3600 → cutoff 6400
|
||||
assert.deepEqual(exp, ['old']);
|
||||
});
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run to verify it fails** — FAIL.
|
||||
|
||||
- [ ] **Step 3: Write library.js**
|
||||
|
||||
```javascript
|
||||
import Database from 'better-sqlite3';
|
||||
|
||||
export class Library {
|
||||
constructor(dbPath) {
|
||||
this.db = new Database(dbPath);
|
||||
this.db.exec(`CREATE TABLE IF NOT EXISTS torrents (
|
||||
infohash TEXT PRIMARY KEY, name TEXT, magnet TEXT, path TEXT,
|
||||
added_at INTEGER, last_played_at INTEGER, kept INTEGER DEFAULT 0)`);
|
||||
}
|
||||
add({ infohash, name, magnet, path }) {
|
||||
const now = Math.floor(Date.now() / 1000);
|
||||
this.db.prepare(`INSERT OR REPLACE INTO torrents
|
||||
(infohash,name,magnet,path,added_at,last_played_at,kept)
|
||||
VALUES (?,?,?,?,?,?,0)`).run(infohash, name, magnet, path, now, now);
|
||||
}
|
||||
list() { return this.db.prepare('SELECT * FROM torrents ORDER BY added_at DESC').all(); }
|
||||
get(infohash) { return this.db.prepare('SELECT * FROM torrents WHERE infohash=?').get(infohash) || null; }
|
||||
touch(infohash) { this.touchAt(infohash, Math.floor(Date.now() / 1000)); }
|
||||
touchAt(infohash, ts) { this.db.prepare('UPDATE torrents SET last_played_at=? WHERE infohash=?').run(ts, infohash); }
|
||||
keep(infohash, newPath) { this.db.prepare('UPDATE torrents SET kept=1, path=? WHERE infohash=?').run(newPath, infohash); }
|
||||
remove(infohash) { this.db.prepare('DELETE FROM torrents WHERE infohash=?').run(infohash); }
|
||||
expiredEphemeral(ttlSeconds, now = Math.floor(Date.now() / 1000)) {
|
||||
const cutoff = now - ttlSeconds;
|
||||
return this.db.prepare('SELECT infohash FROM torrents WHERE kept=0 AND last_played_at < ?')
|
||||
.all(cutoff).map(r => r.infohash);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run to verify it passes** — PASS.
|
||||
|
||||
- [ ] **Step 5: Commit** — `git commit -m "feat(torrent): library.js SQLite + purge selection (ref #917)"`
|
||||
|
||||
---
|
||||
|
||||
### Task 5: api.js — Fastify routes
|
||||
|
||||
**Files:**
|
||||
- Create: `packages/secubox-torrent/lxc/app/api.js`
|
||||
- Test: `packages/secubox-torrent/lxc/app/api.test.js`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `Engine` (Task 2), `Library` (Task 4), `handleStream` (Task 3).
|
||||
- Produces: `buildApi({ engine, library, diskFreeBytes }) → fastify instance` with routes:
|
||||
`GET /api/v1/torrent/status`, `POST /api/v1/torrent/add`, `GET /api/v1/torrent/list`,
|
||||
`GET /api/v1/torrent/files/:infohash`, `POST /api/v1/torrent/keep/:infohash`,
|
||||
`POST /api/v1/torrent/remove/:infohash`, `GET /api/v1/torrent/health`,
|
||||
`GET /stream/:infohash/:fileIdx`. `diskFreeBytes` is an injected `() → number`.
|
||||
|
||||
- [ ] **Step 1: Write the failing test** (uses fastify `.inject`, no network)
|
||||
|
||||
```javascript
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { buildApi } from './api.js';
|
||||
import { Library } from './library.js';
|
||||
import { Engine } from './engine.js';
|
||||
import { FakeWebTorrent } from './fakes.js';
|
||||
|
||||
function build() {
|
||||
const engine = new Engine({ WebTorrentCtor: FakeWebTorrent, downloadDir: '/tmp', maxActive: 5, webrtc: true });
|
||||
const library = new Library(':memory:');
|
||||
return buildApi({ engine, library, diskFreeBytes: () => 10 * 1e9 });
|
||||
}
|
||||
|
||||
test('health returns ok', async () => {
|
||||
const app = build();
|
||||
const r = await app.inject({ method: 'GET', url: '/api/v1/torrent/health' });
|
||||
assert.equal(r.statusCode, 200);
|
||||
assert.equal(r.json().status, 'ok');
|
||||
});
|
||||
|
||||
test('add then list returns the torrent', async () => {
|
||||
const app = build();
|
||||
await app.inject({ method: 'POST', url: '/api/v1/torrent/add',
|
||||
payload: { magnet: 'magnet:?xt=urn:btih:' + 'a'.repeat(40) } });
|
||||
const r = await app.inject({ method: 'GET', url: '/api/v1/torrent/list' });
|
||||
assert.equal(r.json().length, 1);
|
||||
assert.equal(r.json()[0].kept, 0);
|
||||
});
|
||||
|
||||
test('keep flips kept=1', async () => {
|
||||
const app = build();
|
||||
await app.inject({ method: 'POST', url: '/api/v1/torrent/add',
|
||||
payload: { magnet: 'magnet:?xt=urn:btih:' + 'a'.repeat(40) } });
|
||||
const r = await app.inject({ method: 'POST', url: '/api/v1/torrent/keep/' + 'a'.repeat(40) });
|
||||
assert.equal(r.statusCode, 200);
|
||||
const list = await app.inject({ method: 'GET', url: '/api/v1/torrent/list' });
|
||||
assert.equal(list.json()[0].kept, 1);
|
||||
});
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run to verify it fails** — FAIL.
|
||||
|
||||
- [ ] **Step 3: Write api.js**
|
||||
|
||||
```javascript
|
||||
import Fastify from 'fastify';
|
||||
import path from 'node:path';
|
||||
import { handleStream } from './stream.js';
|
||||
|
||||
export function buildApi({ engine, library, diskFreeBytes }) {
|
||||
const app = Fastify({ logger: false });
|
||||
|
||||
app.get('/api/v1/torrent/health', async () => ({ status: 'ok' }));
|
||||
app.get('/api/v1/torrent/status', async () => ({
|
||||
active: engine.client.torrents.length, disk_free: diskFreeBytes(), webrtc: engine.webrtc !== false }));
|
||||
|
||||
app.post('/api/v1/torrent/add', async (req, reply) => {
|
||||
const magnet = (req.body || {}).magnet;
|
||||
if (!magnet) return reply.code(400).send({ error: 'magnet required' });
|
||||
let meta;
|
||||
try { meta = await engine.add(magnet); }
|
||||
catch (e) { return reply.code(504).send({ error: e.message }); }
|
||||
library.add({ infohash: meta.infohash, name: meta.name, magnet,
|
||||
path: path.join(engine.downloadDir, 'tmp', meta.infohash) });
|
||||
return meta;
|
||||
});
|
||||
|
||||
app.get('/api/v1/torrent/list', async () =>
|
||||
library.list().map(r => ({ ...r, stats: engine.stats(r.infohash) })));
|
||||
|
||||
app.get('/api/v1/torrent/files/:infohash', async (req, reply) => {
|
||||
const t = engine.get(req.params.infohash);
|
||||
if (!t) return reply.code(404).send({ error: 'not found' });
|
||||
return t.files.map((f, i) => ({ idx: i, name: f.name, length: f.length }));
|
||||
});
|
||||
|
||||
app.post('/api/v1/torrent/keep/:infohash', async (req, reply) => {
|
||||
const ih = req.params.infohash;
|
||||
if (!library.get(ih)) return reply.code(404).send({ error: 'not found' });
|
||||
library.keep(ih, path.join(engine.downloadDir, 'library', ih));
|
||||
return { status: 'kept' };
|
||||
});
|
||||
|
||||
app.post('/api/v1/torrent/remove/:infohash', async (req) => {
|
||||
engine.remove(req.params.infohash, { deleteData: true });
|
||||
library.remove(req.params.infohash);
|
||||
return { status: 'removed' };
|
||||
});
|
||||
|
||||
app.get('/stream/:infohash/:fileIdx', (req, reply) => {
|
||||
library.touch(req.params.infohash);
|
||||
return handleStream(engine, req, reply.raw)
|
||||
.then(() => { reply.hijack(); });
|
||||
});
|
||||
|
||||
return app;
|
||||
}
|
||||
```
|
||||
|
||||
Note: `engine.webrtc` — add `this.webrtc = webrtc;` to the Engine constructor (Task 2) so `/status` reports it; if you reach here and it is missing, add that one line and re-run Task 2 tests.
|
||||
|
||||
- [ ] **Step 4: Run to verify it passes** — PASS. Also re-run `node --test` (whole dir) to confirm nothing regressed.
|
||||
|
||||
- [ ] **Step 5: Commit** — `git commit -m "feat(torrent): api.js Fastify routes + stream mount (ref #917)"`
|
||||
|
||||
---
|
||||
|
||||
### Task 6: server.js + purge sweep + player webui
|
||||
|
||||
**Files:**
|
||||
- Create: `packages/secubox-torrent/lxc/app/server.js`
|
||||
- Create: `packages/secubox-torrent/www/torrent/index.html` (replaces the old panel)
|
||||
- Test: `packages/secubox-torrent/lxc/app/server.test.js` (purge sweep only)
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: everything above.
|
||||
- Produces: `runPurge(engine, library, { ttlSeconds, diskFloorBytes, diskFreeBytes }) → [infohash]` (the swept list); `server.js` wires env → Engine/Library/buildApi and starts listening + schedules `runPurge` on an interval.
|
||||
|
||||
- [ ] **Step 1: Write the failing purge test**
|
||||
|
||||
```javascript
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { runPurge } from './server.js';
|
||||
import { Library } from './library.js';
|
||||
|
||||
test('purge removes expired ephemeral and calls engine.remove', () => {
|
||||
const lib = new Library(':memory:');
|
||||
lib.add({ infohash: 'old', name: 'O', magnet: 'm', path: '/tmp/o' });
|
||||
lib.touchAt('old', 0);
|
||||
const removed = [];
|
||||
const engine = { remove: (ih) => removed.push(ih) };
|
||||
const swept = runPurge(engine, lib, { ttlSeconds: 10, diskFloorBytes: 0, diskFreeBytes: () => 1e12 });
|
||||
assert.deepEqual(swept, ['old']);
|
||||
assert.deepEqual(removed, ['old']);
|
||||
assert.equal(lib.get('old'), null);
|
||||
});
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run to verify it fails** — FAIL.
|
||||
|
||||
- [ ] **Step 3: Write server.js** (purge + wiring)
|
||||
|
||||
```javascript
|
||||
import os from 'node:os';
|
||||
import WebTorrent from 'webtorrent';
|
||||
import { Engine } from './engine.js';
|
||||
import { Library } from './library.js';
|
||||
import { buildApi } from './api.js';
|
||||
|
||||
export function runPurge(engine, library, { ttlSeconds, diskFloorBytes, diskFreeBytes }) {
|
||||
const now = Math.floor(Date.now() / 1000);
|
||||
let victims = library.expiredEphemeral(ttlSeconds, now);
|
||||
if (diskFreeBytes() < diskFloorBytes) {
|
||||
const extra = library.list().filter(r => r.kept === 0).map(r => r.infohash);
|
||||
victims = [...new Set([...victims, ...extra])];
|
||||
}
|
||||
for (const ih of victims) { engine.remove(ih, { deleteData: true }); library.remove(ih); }
|
||||
return victims;
|
||||
}
|
||||
|
||||
export function start() {
|
||||
const cfg = {
|
||||
downloadDir: process.env.TORRENT_DOWNLOAD_DIR || '/data/torrent',
|
||||
maxActive: Number(process.env.TORRENT_MAX_ACTIVE || 5),
|
||||
webrtc: process.env.TORRENT_WEBRTC !== 'false',
|
||||
port: Number(process.env.TORRENT_PORT || 8090),
|
||||
ttl: Number(process.env.TORRENT_EPHEMERAL_TTL_HOURS || 6) * 3600,
|
||||
floor: Number(process.env.TORRENT_DISK_FLOOR_GB || 5) * 1e9,
|
||||
};
|
||||
const engine = new Engine({ WebTorrentCtor: WebTorrent, ...cfg });
|
||||
const library = new Library(`${cfg.downloadDir}/library.db`);
|
||||
const diskFreeBytes = () => { try { return os.freemem(); } catch { return Infinity; } }; // replaced by statvfs helper at impl
|
||||
const app = buildApi({ engine, library, diskFreeBytes });
|
||||
app.register(import('@fastify/static'), { root: '/opt/secubox-torrent/www' });
|
||||
setInterval(() => runPurge(engine, library, { ttlSeconds: cfg.ttl, diskFloorBytes: cfg.floor, diskFreeBytes }), 300000);
|
||||
app.listen({ host: '0.0.0.0', port: cfg.port });
|
||||
}
|
||||
if (process.argv[1] && process.argv[1].endsWith('server.js')) start();
|
||||
```
|
||||
|
||||
Note: `diskFreeBytes` must report `/data` free space, not RAM — implement with a small `statvfs` call (e.g. `child_process.execSync('df -B1 --output=avail /data')` parsed, or the `node:fs` statfs API `fs.statfsSync('/data')` on Node ≥ 18.15 → `bavail * bsize`). Use `fs.statfsSync` if available; the test injects `diskFreeBytes` so this detail is not under test. Add `@fastify/static` to package.json deps.
|
||||
|
||||
- [ ] **Step 4: Run to verify it passes** — PASS.
|
||||
|
||||
- [ ] **Step 5: Write www/torrent/index.html** (player webui — replaces old panel)
|
||||
|
||||
Full single-page player following `.claude/WEBUI-PANEL-GUIDELINES.md`: reads `localStorage.sbx_token`, sends it as `Authorization: Bearer` on every fetch; a magnet `<input>` + Add button → `POST /api/v1/torrent/add`; a list from `GET /api/v1/torrent/list` showing name + progress + peer/wire counts + **📌 Garder** / **🗑** buttons; clicking a playable file (type in mp4/webm/m4v/mp3/m4a/ogg) sets `<video controls src="/stream/{infohash}/{idx}">`, others render a Download link; a status strip from `/status` showing a **"BitTorrent-only"** badge when `webrtc:false`. Delete the old Transmission panel content entirely.
|
||||
|
||||
- [ ] **Step 6: Commit** — `git add packages/secubox-torrent/lxc/app/server.js packages/secubox-torrent/www/torrent/index.html && git commit -m "feat(torrent): server wiring + purge sweep + player webui (ref #917)"`
|
||||
|
||||
---
|
||||
|
||||
### Task 7: Host integration — install-lxc.sh (full) + vhost + WAF + nft + menu + config
|
||||
|
||||
**Files:**
|
||||
- Modify: `packages/secubox-torrent/lxc/install-lxc.sh` (extend Task 1 skeleton: copy app, npm ci, install the in-LXC service, start it)
|
||||
- Create: `packages/secubox-torrent/lxc/secubox-torrent.service` (systemd unit INSIDE the LXC)
|
||||
- Create: `packages/secubox-torrent/nginx/torrent.conf` (host vhost → LXC)
|
||||
- Create: `packages/secubox-torrent/nft/torrent-egress.nft`
|
||||
- Modify: `packages/secubox-torrent/menu.d/*.json` (keep entry, ensure host = torrent.gk2.secubox.in)
|
||||
- Create: `packages/secubox-torrent/conf/torrent.toml`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: the `app/` service (Tasks 2-6), env vars from `torrent.toml`.
|
||||
- Produces: an idempotent provisioner; a host vhost proxying `/`, `/api/`, `/stream/` to `http://10.100.0.130:8090` with `auth_request` JWT; an nft drop-in scoping the LXC veth egress.
|
||||
|
||||
- [ ] **Step 1: Write the in-LXC systemd unit** `lxc/secubox-torrent.service`
|
||||
|
||||
```ini
|
||||
[Unit]
|
||||
Description=SecuBox Torrent (WebTorrent streaming)
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
[Service]
|
||||
WorkingDirectory=/opt/secubox-torrent/app
|
||||
EnvironmentFile=/opt/secubox-torrent/torrent.env
|
||||
ExecStart=/usr/bin/node server.js
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Extend install-lxc.sh** (append after the skeleton's Node install; idempotent copy + build + service)
|
||||
|
||||
```bash
|
||||
# --- app deploy (runs every postinst; idempotent) ---
|
||||
APP_SRC="${SECUBOX_APP_SRC:-/usr/lib/secubox/torrent/app}"
|
||||
la mkdir -p /opt/secubox-torrent/app /opt/secubox-torrent/www
|
||||
tar -C "$APP_SRC" -cf - . | la tar -C /opt/secubox-torrent/app -xf -
|
||||
tar -C /usr/share/secubox/www/torrent -cf - . | la tar -C /opt/secubox-torrent/www -xf -
|
||||
la bash -lc 'cd /opt/secubox-torrent/app && npm ci --omit=dev || npm install --omit=dev'
|
||||
# env file from host TOML values (exported by postinst into these vars)
|
||||
la bash -c "cat > /opt/secubox-torrent/torrent.env <<EOF
|
||||
TORRENT_DOWNLOAD_DIR=/data/torrent
|
||||
TORRENT_MAX_ACTIVE=${TORRENT_MAX_ACTIVE:-5}
|
||||
TORRENT_WEBRTC=${TORRENT_WEBRTC:-true}
|
||||
TORRENT_PORT=8090
|
||||
TORRENT_EPHEMERAL_TTL_HOURS=${TORRENT_TTL:-6}
|
||||
TORRENT_DISK_FLOOR_GB=${TORRENT_FLOOR:-5}
|
||||
EOF"
|
||||
install -m 644 /dev/stdin_UNUSED 2>/dev/null || true
|
||||
tar -C "$(dirname "$0")" -cf - secubox-torrent.service | la tar -C /etc/systemd/system -xf -
|
||||
la systemctl daemon-reload
|
||||
la systemctl enable --now secubox-torrent.service
|
||||
log "torrent LXC app deployed + started on $LXC_IP:8090"
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Write host vhost** `nginx/torrent.conf` (proxies to LXC, JWT via auth_request — mirror peertube/podcaster vhost)
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 0.0.0.0:9080;
|
||||
server_name torrent.gk2.secubox.in;
|
||||
location = /__sbx_auth_verify { internal; proxy_pass http://unix:/run/secubox/aggregator.sock:/api/v1/auth/verify; proxy_pass_request_body off; proxy_set_header Content-Length ""; }
|
||||
location / {
|
||||
auth_request /__sbx_auth_verify;
|
||||
proxy_pass http://10.100.0.130:8090;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Host $host;
|
||||
proxy_buffering off; # streaming: don't buffer video
|
||||
proxy_request_buffering off;
|
||||
proxy_read_timeout 3600s;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Write nft egress drop-in** `nft/torrent-egress.nft` (host, scopes the LXC veth — drop-based, logged; sorts AFTER the table creator via zz- naming at install)
|
||||
|
||||
```
|
||||
# Scope the torrent LXC's egress. OUTPUT is policy accept repo-wide, so this
|
||||
# is an explicit LOG+limit on the LXC veth, not an allow. Applied on the host.
|
||||
table inet secubox_torrent {
|
||||
chain forward_torrent {
|
||||
type filter hook forward priority 0;
|
||||
iifname "veth-torrent*" ct state new log prefix "[SECUBOX-TORRENT] " level info
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Write conf/torrent.toml**
|
||||
|
||||
```toml
|
||||
[engine]
|
||||
max_active = 5
|
||||
webrtc = true # set false to force BitTorrent-only
|
||||
[retention]
|
||||
ephemeral_ttl_hours = 6
|
||||
disk_floor_gb = 5
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Verify menu.d entry** — ensure `menu.d/*.json` keeps the 🌊 Torrent entry pointing at `torrent.gk2.secubox.in` (Media category). Edit only if the host/URL is wrong.
|
||||
|
||||
- [ ] **Step 7: Commit** — `git add packages/secubox-torrent/{lxc,nginx,nft,conf,menu.d} && git commit -m "feat(torrent): host LXC provisioner + vhost + nft egress + config (ref #917)"`
|
||||
|
||||
---
|
||||
|
||||
### Task 8: Debian packaging v2.0.0 (remove Transmission)
|
||||
|
||||
**Files:**
|
||||
- Delete: `packages/secubox-torrent/api/main.py`, `packages/secubox-torrent/systemd/secubox-torrent.service` (old host FastAPI)
|
||||
- Modify: `packages/secubox-torrent/debian/control`, `debian/rules`, `debian/postinst`, `debian/postrm`, `debian/changelog`
|
||||
- Test: `packages/secubox-torrent/tests/test_packaging.py` (structural, pytest — reuse SecuBox packaging-test convention)
|
||||
|
||||
**Interfaces:**
|
||||
- Produces: a `secubox-torrent_2.0.0-1~bookworm1_all.deb` that installs the app source to `/usr/lib/secubox/torrent/app`, the webui to `/usr/share/secubox/www/torrent`, the vhost/nft/menu/conf, and runs `install-lxc.sh` at postinst.
|
||||
|
||||
- [ ] **Step 1: Write the failing packaging test**
|
||||
|
||||
```python
|
||||
from pathlib import Path
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
|
||||
def _read(p): return (ROOT / p).read_text()
|
||||
|
||||
def test_control_is_v2_no_transmission():
|
||||
c = _read("debian/control")
|
||||
assert "python3-uvicorn" not in c # old FastAPI gone
|
||||
assert "transmission" not in c.lower()
|
||||
assert "lxc" in c.lower()
|
||||
|
||||
def test_changelog_is_2_0_0():
|
||||
assert _read("debian/changelog").startswith("secubox-torrent (2.0.0-1~bookworm1)")
|
||||
|
||||
def test_postinst_runs_install_lxc():
|
||||
assert "install-lxc.sh" in _read("debian/postinst")
|
||||
|
||||
def test_no_old_fastapi_main():
|
||||
assert not (ROOT / "api" / "main.py").exists()
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run to verify it fails** — `cd packages/secubox-torrent && python3 -m pytest tests/test_packaging.py` → FAIL.
|
||||
|
||||
- [ ] **Step 3: Rewrite debian/control**
|
||||
|
||||
```
|
||||
Source: secubox-torrent
|
||||
Section: net
|
||||
Priority: optional
|
||||
Maintainer: Gerald KERMA <devel@cybermind.fr>
|
||||
Build-Depends: debhelper-compat (= 13)
|
||||
Standards-Version: 4.6.2
|
||||
|
||||
Package: secubox-torrent
|
||||
Architecture: all
|
||||
Depends: ${misc:Depends}, secubox-core (>= 1.0), lxc, debootstrap, nftables
|
||||
Description: WebTorrent streaming for SecuBox (LXC-native)
|
||||
Paste a magnet and stream it in the browser (HTTP Range) while it downloads.
|
||||
Ephemeral by default with an optional Keep to a persistent library on /data.
|
||||
The WebTorrent engine (hybrid WebRTC+BitTorrent) runs isolated in a dedicated
|
||||
LXC; the host only proxies an authenticated vhost.
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Rewrite debian/rules** (install app source, webui, vhost, nft, conf, menu)
|
||||
|
||||
```make
|
||||
#!/usr/bin/make -f
|
||||
%:
|
||||
dh $@
|
||||
override_dh_auto_install:
|
||||
install -d $(CURDIR)/debian/secubox-torrent/usr/lib/secubox/torrent
|
||||
cp -r $(CURDIR)/lxc/app $(CURDIR)/debian/secubox-torrent/usr/lib/secubox/torrent/
|
||||
install -m 755 $(CURDIR)/lxc/install-lxc.sh $(CURDIR)/debian/secubox-torrent/usr/lib/secubox/torrent/
|
||||
install -m 644 $(CURDIR)/lxc/secubox-torrent.service $(CURDIR)/debian/secubox-torrent/usr/lib/secubox/torrent/
|
||||
install -d $(CURDIR)/debian/secubox-torrent/usr/share/secubox/www/torrent
|
||||
cp -r $(CURDIR)/www/torrent/* $(CURDIR)/debian/secubox-torrent/usr/share/secubox/www/torrent/
|
||||
install -d $(CURDIR)/debian/secubox-torrent/etc/nginx/secubox-vhost.d
|
||||
install -m 644 $(CURDIR)/nginx/torrent.conf $(CURDIR)/debian/secubox-torrent/etc/nginx/secubox-vhost.d/
|
||||
install -d $(CURDIR)/debian/secubox-torrent/etc/secubox/nft.d
|
||||
install -m 644 $(CURDIR)/nft/torrent-egress.nft $(CURDIR)/debian/secubox-torrent/etc/secubox/nft.d/zz-torrent-egress.nft
|
||||
install -d $(CURDIR)/debian/secubox-torrent/etc/secubox
|
||||
install -m 644 $(CURDIR)/conf/torrent.toml $(CURDIR)/debian/secubox-torrent/etc/secubox/torrent.toml
|
||||
install -d $(CURDIR)/debian/secubox-torrent/usr/share/secubox/menu.d
|
||||
install -m 644 $(CURDIR)/menu.d/*.json $(CURDIR)/debian/secubox-torrent/usr/share/secubox/menu.d/
|
||||
```
|
||||
|
||||
Note: confirm the host vhost include dir the board actually uses for full `server{}` vhosts (peertube/podcaster ship theirs there — check `packages/secubox-peertube/debian/rules` for the exact path and match it; the plan uses `etc/nginx/secubox-vhost.d` as the placeholder, correct it to peertube's path in this step).
|
||||
|
||||
- [ ] **Step 5: Rewrite debian/postinst** (run provisioner + reload nginx/nft)
|
||||
|
||||
```bash
|
||||
#!/bin/sh
|
||||
set -e
|
||||
case "$1" in configure)
|
||||
# export config → env for install-lxc.sh (parse the TOML minimally)
|
||||
export TORRENT_MAX_ACTIVE=$(grep -E '^max_active' /etc/secubox/torrent.toml | grep -oE '[0-9]+' | head -1)
|
||||
export TORRENT_WEBRTC=$(grep -E '^webrtc' /etc/secubox/torrent.toml | grep -oE 'true|false' | head -1)
|
||||
export TORRENT_TTL=$(grep -E '^ephemeral_ttl_hours' /etc/secubox/torrent.toml | grep -oE '[0-9]+' | head -1)
|
||||
export TORRENT_FLOOR=$(grep -E '^disk_floor_gb' /etc/secubox/torrent.toml | grep -oE '[0-9]+' | head -1)
|
||||
sh /usr/lib/secubox/torrent/install-lxc.sh || echo "torrent: LXC provisioning deferred (run install-lxc.sh manually)"
|
||||
nft -f /etc/secubox/nft.d/zz-torrent-egress.nft 2>/dev/null || true
|
||||
if command -v nginx >/dev/null 2>&1 && nginx -t >/dev/null 2>&1; then systemctl reload nginx 2>/dev/null || true; fi
|
||||
;;
|
||||
esac
|
||||
#DEBHELPER#
|
||||
exit 0
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Rewrite debian/postrm** (stop + optionally destroy LXC on purge)
|
||||
|
||||
```bash
|
||||
#!/bin/sh
|
||||
set -e
|
||||
case "$1" in
|
||||
purge)
|
||||
lxc-stop -n torrent -P /data/lxc 2>/dev/null || true
|
||||
lxc-destroy -n torrent -P /data/lxc 2>/dev/null || true
|
||||
rm -f /var/lib/secubox/torrent/.lxc-provisioned 2>/dev/null || true
|
||||
;;
|
||||
esac
|
||||
#DEBHELPER#
|
||||
exit 0
|
||||
```
|
||||
|
||||
- [ ] **Step 7: Add debian/changelog entry**
|
||||
|
||||
```
|
||||
secubox-torrent (2.0.0-1~bookworm1) bookworm; urgency=medium
|
||||
|
||||
* Pivot from Transmission to WebTorrent streaming (LXC-native, #917).
|
||||
* Browser player (HTTP Range), ephemeral-by-default + Keep library on /data.
|
||||
* Hybrid WebRTC+BitTorrent engine isolated in a dedicated LXC.
|
||||
|
||||
-- Gerald KERMA <devel@cybermind.fr> Tue, 28 Jul 2026 14:00:00 +0200
|
||||
```
|
||||
|
||||
- [ ] **Step 8: Run packaging test to verify it passes** — PASS.
|
||||
|
||||
- [ ] **Step 9: Build the .deb** — `cd packages/secubox-torrent && dpkg-buildpackage -a arm64 --host-arch arm64 -us -uc -b` → produces `secubox-torrent_2.0.0-1~bookworm1_all.deb`.
|
||||
|
||||
- [ ] **Step 10: Commit** — `git add -A packages/secubox-torrent && git commit -m "feat(torrent): debian packaging v2.0.0, remove Transmission (ref #917)"`
|
||||
|
||||
---
|
||||
|
||||
## Self-Review
|
||||
|
||||
**Spec coverage:** ✅ LXC-native isolated (T1/T7), engine hybrid (T2), Range streaming (T3), library ephemeral/kept+purge (T4/T6), API (T5), player webui (T6), host vhost+WAF+JWT (T7), nft egress (T7), packaging v2.0.0 removing Transmission (T8), wrtc-arm64 spike + fallback (T1 → threads through engine `webrtc` + `/status` badge). Retention TTL + disk-floor: T4 selection + T6 sweep. Config TOML: T7/T8.
|
||||
|
||||
**Placeholder scan:** the two intentional impl-detail notes (statvfs `diskFreeBytes`, exact nginx vhost include path) are flagged inline with the concrete resolution to apply, not left vague. No TBD/TODO.
|
||||
|
||||
**Type consistency:** `add()→{infohash,name,files:[{idx,name,length,type}]}` consistent T2→T5→T6; `Library` method names (add/list/get/touch/touchAt/keep/remove/expiredEphemeral) consistent T4→T5→T6; `handleStream(engine,req,res)` consistent T3→T5; `buildApi({engine,library,diskFreeBytes})` consistent T5→T6; `runPurge(engine,library,{ttlSeconds,diskFloorBytes,diskFreeBytes})` consistent T6. `engine.webrtc` field flagged in T5 note to add in T2.
|
||||
|
||||
**Known cross-task fixups the implementer must honor:** (a) add `this.webrtc = webrtc;` to Engine (T2) — used by `/status` (T5); (b) resolve the nginx vhost include dir against peertube (T8 Step 4); (c) `diskFreeBytes` real impl via `fs.statfsSync('/data')` (T6).
|
||||
|
|
@ -0,0 +1,204 @@
|
|||
# secubox-torrent v2.0 — WebTorrent Streaming Pivot — Design Spec
|
||||
|
||||
**Issue:** #917
|
||||
**Date:** 2026-07-28
|
||||
**Status:** Design approved, pending spec review → implementation plan
|
||||
|
||||
## Goal
|
||||
|
||||
Pivot the dormant `secubox-torrent` module from a Transmission download-manager
|
||||
into a **WebTorrent streaming** module: paste a magnet → watch it in the browser
|
||||
(HTML5 `<video>`, HTTP Range) while it downloads. Ephemeral by default, with an
|
||||
optional **Keep** action that retains a title in a persistent library on the
|
||||
SSD `/data`. True WebRTC-hybrid: the browser participates in the swarm.
|
||||
|
||||
## Non-Goals (YAGNI)
|
||||
|
||||
- No RSS / scheduled downloads (that was Transmission's job; dropped).
|
||||
- No multi-user quotas / per-user libraries (single admin-scoped tool; master
|
||||
users gk2 + admin only).
|
||||
- No transcoding (serve the file as-is; the browser plays what it can — MP4/WebM
|
||||
natively; MKV/AVI may not play — documented, not solved in v1).
|
||||
- No built-in search/indexer (paste magnets/torrent files only).
|
||||
- No antivirus scan hook in v1 (noted as a future SecuBox-native extension).
|
||||
|
||||
## Context & Constraints
|
||||
|
||||
- **Board:** MOCHAbin gk2, Debian bookworm **arm64**. Storage: eMMC (small, fills
|
||||
up) + USB SSD mounted at `/data` (media MUST live here — see podcaster).
|
||||
- **Existing module:** `secubox-torrent` 1.0.0 (host FastAPI + Transmission via
|
||||
Docker) — installed on gk2 but **dormant/inactive**. This pivot **replaces**
|
||||
it; the package name, vhost, and menu entry are preserved (version → 2.0.0).
|
||||
- **SecuBox module pattern:** LXC-native app modules (peertube/podcaster/lyrion),
|
||||
fronted by a host vhost → HAProxy (TLS 1.3) → sbxwaf (:8085) → nginx (:9080) →
|
||||
LXC. Auth = JWT `sbx_token` (localStorage), master users gk2 + admin.
|
||||
- **Security posture:** torrent connects to **untrusted peers** — on a security
|
||||
appliance the engine MUST be isolated in a dedicated LXC, never on the host.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
Browser (player webui, <video>)
|
||||
│ https://torrent.gk2.secubox.in/ + /api/* + /stream/*
|
||||
▼
|
||||
HAProxy (TLS 1.3) ──► sbxwaf (:8085) ──► nginx (:9080)
|
||||
│ JWT gate (auth_request → secubox-auth) on /api + /stream + panel
|
||||
▼
|
||||
LXC "torrent" (isolated, 10.100.0.x, /data-backed)
|
||||
└─ Node.js service (systemd inside LXC):
|
||||
├─ engine (webtorrent-hybrid: BitTorrent + WebRTC via @roamhq/wrtc)
|
||||
├─ stream server (Fastify: GET /stream/:infohash/:file → Range → res)
|
||||
├─ library (better-sqlite3: ephemeral/kept state + purge)
|
||||
└─ api + static (Fastify: /api/* JSON, serves the player webui)
|
||||
```
|
||||
|
||||
**Isolation & egress:** the LXC's outbound torrent traffic is scoped by an nft
|
||||
rule set on the host (the LXC's veth), logged and rate-limitable. The webui is
|
||||
reachable ONLY through the WAF vhost; the LXC's Node port is not exposed on the
|
||||
LAN directly.
|
||||
|
||||
**wrtc-arm64 risk (Task 1 spike):** `@roamhq/wrtc` ships arm64 prebuilds; if the
|
||||
prebuild fails to load in the bookworm-arm64 LXC, fall back to **BitTorrent-only**
|
||||
(`webtorrent` without the WebRTC transport). The player + streaming are identical;
|
||||
only browser-peer swarm participation is lost. The spike gates which npm dep the
|
||||
package pins.
|
||||
|
||||
## Components
|
||||
|
||||
Each is a focused unit with one responsibility and a mockable interface.
|
||||
|
||||
### `engine.js` (torrent engine wrap)
|
||||
- Wraps a single `WebTorrent` client instance.
|
||||
- `add(magnetOrTorrent) → {infohash, name, files:[{idx,name,length,type}]}`
|
||||
(resolves once metadata is available; timeout → error).
|
||||
- `get(infohash) → torrent | null`; `remove(infohash, {deleteData})`.
|
||||
- `stats(infohash) → {progress, downloadSpeed, uploadSpeed, numPeers, wires:[{type:'webrtc'|'tcp'|'utp', addr}]}`.
|
||||
- Enforces a max-concurrent-torrents cap (config) to bound memory.
|
||||
- Interface is injectable so tests use a `FakeWebTorrent` (no network).
|
||||
|
||||
### `stream.js` (HTTP Range streaming)
|
||||
- `GET /stream/:infohash/:fileIdx` → validates infohash+idx, sets
|
||||
`Content-Type` from file ext, honours the `Range` header, pipes
|
||||
`file.stream({start,end})` to the response. Supports seeking (multiple Range
|
||||
requests) while the torrent is still downloading (WebTorrent handles
|
||||
out-of-order piece prioritisation on read).
|
||||
- 404 for unknown infohash/idx; 416 for unsatisfiable range.
|
||||
|
||||
### `library.js` (persistence + retention)
|
||||
- SQLite (`better-sqlite3`) at `/data/torrent/library.db`.
|
||||
- Row per torrent: `infohash, name, magnet, added_at, last_played_at, kept(0/1),
|
||||
path`. Ephemeral rows (kept=0) live under `/data/torrent/tmp/<infohash>/`;
|
||||
kept rows under `/data/torrent/library/<infohash>/`.
|
||||
- `keep(infohash)` → move files tmp→library, set kept=1 (continues seeding).
|
||||
- `unkeep(infohash)`, `remove(infohash)`.
|
||||
- **Purge policy:** a periodic sweep removes ephemeral (kept=0) torrents whose
|
||||
`last_played_at` is older than `ephemeral_ttl` (config, default 6 h) OR when
|
||||
`/data` free space drops below a floor (config) — oldest-ephemeral-first.
|
||||
Kept torrents are never auto-purged. All purges logged.
|
||||
|
||||
### `api.js` (control API, Fastify)
|
||||
- `GET /api/v1/torrent/status` → engine up, counts, disk free.
|
||||
- `POST /api/v1/torrent/add {magnet}` → engine.add + library insert (kept=0).
|
||||
- `GET /api/v1/torrent/list` → library rows + live stats.
|
||||
- `GET /api/v1/torrent/files/:infohash` → file list.
|
||||
- `POST /api/v1/torrent/keep/:infohash`, `POST /api/v1/torrent/remove/:infohash`.
|
||||
- `GET /api/v1/torrent/health`.
|
||||
- All JSON; auth enforced at the host WAF layer (see below), the LXC service
|
||||
trusts the proxied request (LXC is not LAN-exposed).
|
||||
|
||||
### `www/` player webui
|
||||
- Single-page (SecuBox webui look per WEBUI-PANEL-GUIDELINES, reads `sbx_token`):
|
||||
magnet input → on add, show file list → click a playable file → `<video
|
||||
src="/stream/:infohash/:idx" controls>`; **📌 Keep** / **🗑 Remove** buttons;
|
||||
a live stats strip (progress, peers, WebRTC vs TCP/uTP wire count).
|
||||
- Non-playable files (unknown container) show a **Download** link instead of the
|
||||
player.
|
||||
|
||||
### Host side (thin)
|
||||
- `debian/` package installs: the LXC (`install-lxc.sh`, idempotent, same
|
||||
pattern as peertube/podcaster), the host **vhost** (`nginx/torrent.conf` →
|
||||
proxies to the LXC), the **WAF route** (haproxy-routes.json), the **menu.d**
|
||||
entry, `/etc/secubox/torrent.toml`, and an **nft egress drop-in** for the LXC
|
||||
veth. No business logic on the host — it only proxies + authenticates.
|
||||
- Auth: the vhost uses the standard SecuBox `auth_request` → secubox-auth to
|
||||
gate `/`, `/api/`, and `/stream/` with the `sbx_token` cookie/bearer, exactly
|
||||
like other module vhosts.
|
||||
|
||||
## Data Flow
|
||||
|
||||
1. User pastes a magnet in the webui → `POST /api/v1/torrent/add`.
|
||||
2. `engine.add` fetches metadata → returns file list; `library` inserts kept=0
|
||||
under `/data/torrent/tmp/<infohash>/`.
|
||||
3. User clicks a playable file → browser opens `<video src=/stream/:infohash/:idx>`.
|
||||
4. `stream.js` serves Range requests; WebTorrent prioritises the needed pieces →
|
||||
playback starts before full download; seeking issues new Range requests.
|
||||
5. In hybrid mode, the browser (loading webtorrent in-page, optional v1.1) and the
|
||||
LXC engine both peer over WebRTC + BitTorrent.
|
||||
6. User clicks **Keep** → files moved tmp→library, kept=1, seeding continues.
|
||||
Otherwise the purge sweep removes the ephemeral torrent after TTL / on low disk.
|
||||
|
||||
## Config (`/etc/secubox/torrent.toml`)
|
||||
|
||||
```toml
|
||||
[engine]
|
||||
max_active = 5 # concurrent torrents cap
|
||||
download_dir = "/data/torrent"
|
||||
webrtc = true # false forces BitTorrent-only (fallback)
|
||||
|
||||
[retention]
|
||||
ephemeral_ttl_hours = 6 # purge ephemeral not played within this window
|
||||
disk_floor_gb = 5 # purge oldest ephemeral when /data free < this
|
||||
|
||||
[net]
|
||||
# torrent listen port inside the LXC; host nft scopes egress on the veth
|
||||
listen_port = 6881
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
- `engine.add` metadata timeout → 504 + user-facing "magnet unreachable / no
|
||||
peers".
|
||||
- wrtc load failure at boot → log once, run BitTorrent-only, expose
|
||||
`webrtc:false` in `/status` so the webui shows a "BitTorrent-only" badge.
|
||||
- Disk full on `/data` → `add` refused with a clear error; purge sweep triggers.
|
||||
- LXC down → host vhost returns 502; a host watchdog (existing secubox-watchdog
|
||||
pattern) may restart the LXC.
|
||||
- Stream of a still-metadata-less torrent → 409 "not ready".
|
||||
|
||||
## Security & Confinement
|
||||
|
||||
- **LXC isolation:** untrusted peer traffic never touches the host.
|
||||
- **nft egress scope:** host drop-in on the LXC veth — allow the torrent listen
|
||||
port + DHT/WebRTC, log the rest; rate-limitable. (OUTPUT is `policy accept`
|
||||
repo-wide, so containment is a **drop-based** rule on the veth, not an allow —
|
||||
same lesson as the meshtastic on-grid note.)
|
||||
- **WAF + JWT:** webui/API/stream only reachable through sbxwaf + `auth_request`;
|
||||
master users gk2 + admin.
|
||||
- **No host business logic:** compromise of the Node engine is contained to the LXC.
|
||||
- Future (not v1): scan **kept** files via antirootkit/ClamAV before they land in
|
||||
the library.
|
||||
|
||||
## Testing
|
||||
|
||||
- `engine` against a `FakeWebTorrent` (deterministic metadata/files/stats) — no
|
||||
network.
|
||||
- `stream` Range handling: full, partial, open-ended, unsatisfiable (416),
|
||||
seeking mid-download (mock file stream).
|
||||
- `library`: insert/keep/unkeep/remove + purge policy (TTL + disk-floor) on a
|
||||
tmp SQLite + tmp dirs.
|
||||
- `api`: each route, JSON shape, error codes (mock engine/library).
|
||||
- `packaging`: install-lxc.sh idempotency assertions, vhost + WAF route present,
|
||||
nft drop-in sorts after the table-creator, config installed.
|
||||
- No real BitTorrent/WebRTC in tests. The **wrtc-arm64 spike (Task 1)** is a
|
||||
manual/CI feasibility check, not a unit test.
|
||||
|
||||
## Open Questions (resolved defaults)
|
||||
|
||||
- **Name/version:** keep `secubox-torrent`, bump to **2.0.0-1~bookworm1**; the
|
||||
1.0.0 Transmission code is removed. Vhost host = `torrent.gk2.secubox.in`
|
||||
(existing) preserved.
|
||||
- **LXC provisioning:** reuse the existing `install-lxc.sh` LXC pattern
|
||||
(peertube/podcaster), Debian bookworm arm64 base + Node LTS.
|
||||
- **Browser-side WebRTC peer (in-page webtorrent):** deferred to v1.1 — v1 ships
|
||||
the server-side hybrid engine + player; the in-browser swarm participant is an
|
||||
additive enhancement once the server path is proven.
|
||||
|
|
@ -1,6 +1,8 @@
|
|||
# 🌊 Torrent
|
||||
|
||||
BitTorrent client
|
||||
WebTorrent streaming, LXC-native (v2.0.0). Paste a magnet and stream it in
|
||||
the browser over HTTP Range while it downloads. Ephemeral by default, with
|
||||
an optional Keep into a persistent library on `/data`.
|
||||
|
||||
**Category:** Media
|
||||
|
||||
|
|
@ -8,12 +10,19 @@ BitTorrent client
|
|||
|
||||
|
||||
|
||||
## Features
|
||||
## Architecture
|
||||
|
||||
- Downloads
|
||||
- RSS
|
||||
- Remote control
|
||||
- Bandwidth limits
|
||||
The hybrid WebRTC+BitTorrent engine (`webtorrent` + `fastify` +
|
||||
`better-sqlite3`) runs isolated inside a dedicated LXC (`torrent`,
|
||||
`10.100.0.160`). The host only ships:
|
||||
|
||||
- the public vhost `torrent.gk2.secubox.in` (nginx, proxies to the LXC's
|
||||
`:8090`, streamed via HTTP Range with no buffering),
|
||||
- an nft egress visibility scope for the LXC's veth,
|
||||
- `install-lxc.sh`, which provisions the container and deploys the app.
|
||||
|
||||
v1 (Transmission via Docker/Podman, host FastAPI on
|
||||
`/api/v1/torrent/*`) was removed in 2.0.0 — see `debian/changelog`.
|
||||
|
||||
## Installation
|
||||
|
||||
|
|
@ -27,12 +36,10 @@ sudo apt install secubox-torrent
|
|||
|
||||
## Configuration
|
||||
|
||||
Configuration file: `/etc/secubox/torrent.toml`
|
||||
|
||||
## API Endpoints
|
||||
|
||||
- `GET /api/v1/torrent/status` - Module status
|
||||
- `GET /api/v1/torrent/health` - Health check
|
||||
Configuration file: `/etc/secubox/torrent.toml` (LXC network, engine
|
||||
`max_active`/`webrtc`, ephemeral retention `ephemeral_ttl_hours`/
|
||||
`disk_floor_gb`). Edit then re-run
|
||||
`/usr/lib/secubox/torrent/install-lxc.sh` to apply.
|
||||
|
||||
## License
|
||||
|
||||
|
|
|
|||
|
|
@ -1,903 +0,0 @@
|
|||
"""secubox-torrent — FastAPI application for BitTorrent client management.
|
||||
|
||||
Provides Transmission/qBittorrent Docker container management with
|
||||
torrent lifecycle, RSS feeds, and speed limiting.
|
||||
|
||||
SecuBox-Deb :: secubox-torrent
|
||||
CyberMind — https://cybermind.fr
|
||||
Author: Gerald Kerma <devel@cybermind.fr>
|
||||
License: Proprietary / ANSSI CSPN candidate
|
||||
"""
|
||||
import asyncio
|
||||
import json
|
||||
import shutil
|
||||
import subprocess
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import List, Optional, Dict, Any
|
||||
|
||||
from fastapi import FastAPI, APIRouter, Depends, HTTPException, UploadFile, File, Form
|
||||
from pydantic import BaseModel
|
||||
|
||||
from secubox_core.auth import router as auth_router, require_jwt
|
||||
from secubox_core.logger import get_logger
|
||||
|
||||
app = FastAPI(title="secubox-torrent", version="1.0.0", root_path="/api/v1/torrent")
|
||||
|
||||
# ══════════════════════════════════════════════════════════════════
|
||||
# Health Check Endpoint (public, no auth)
|
||||
# ══════════════════════════════════════════════════════════════════
|
||||
|
||||
@app.get("/health")
|
||||
async def health_check():
|
||||
"""Public health check endpoint for sidebar status."""
|
||||
return {"status": "ok", "module": "deb"}
|
||||
|
||||
app.include_router(auth_router, prefix="/auth")
|
||||
router = APIRouter()
|
||||
log = get_logger("torrent")
|
||||
|
||||
# Configuration
|
||||
CONFIG_FILE = Path("/etc/secubox/torrent.toml")
|
||||
CONTAINER_NAME = "secbx-torrent"
|
||||
CACHE_DIR = Path("/var/cache/secubox/torrent")
|
||||
RSS_FILE = Path("/etc/secubox/torrent-rss.json")
|
||||
CATEGORIES_FILE = Path("/etc/secubox/torrent-categories.json")
|
||||
|
||||
DEFAULT_CONFIG = {
|
||||
"enabled": False,
|
||||
"client": "transmission", # transmission or qbittorrent
|
||||
"image": "linuxserver/transmission:latest",
|
||||
"port": 9091,
|
||||
"peer_port": 51413,
|
||||
"data_path": "/srv/torrent",
|
||||
"download_dir": "/srv/torrent/downloads",
|
||||
"watch_dir": "/srv/torrent/watch",
|
||||
"timezone": "Europe/Paris",
|
||||
"domain": "torrent.secubox.local",
|
||||
"haproxy": False,
|
||||
"auth_enabled": True,
|
||||
"username": "admin",
|
||||
"password_hash": "",
|
||||
# Speed limits (KB/s, 0 = unlimited)
|
||||
"download_limit": 0,
|
||||
"upload_limit": 0,
|
||||
"alt_download_limit": 1000,
|
||||
"alt_upload_limit": 500,
|
||||
"alt_speed_enabled": False,
|
||||
# Schedule (24h format)
|
||||
"schedule_enabled": False,
|
||||
"schedule_start": "08:00",
|
||||
"schedule_end": "23:00",
|
||||
# Seeding
|
||||
"seed_ratio_limit": 2.0,
|
||||
"seed_ratio_enabled": True,
|
||||
}
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# Models
|
||||
# ============================================================================
|
||||
|
||||
class TorrentConfig(BaseModel):
|
||||
enabled: bool = False
|
||||
client: str = "transmission"
|
||||
image: str = "linuxserver/transmission:latest"
|
||||
port: int = 9091
|
||||
peer_port: int = 51413
|
||||
data_path: str = "/srv/torrent"
|
||||
download_dir: str = "/srv/torrent/downloads"
|
||||
watch_dir: str = "/srv/torrent/watch"
|
||||
timezone: str = "Europe/Paris"
|
||||
domain: str = "torrent.secubox.local"
|
||||
haproxy: bool = False
|
||||
download_limit: int = 0
|
||||
upload_limit: int = 0
|
||||
alt_download_limit: int = 1000
|
||||
alt_upload_limit: int = 500
|
||||
alt_speed_enabled: bool = False
|
||||
schedule_enabled: bool = False
|
||||
schedule_start: str = "08:00"
|
||||
schedule_end: str = "23:00"
|
||||
seed_ratio_limit: float = 2.0
|
||||
seed_ratio_enabled: bool = True
|
||||
|
||||
|
||||
class AddTorrentRequest(BaseModel):
|
||||
magnet: Optional[str] = None
|
||||
url: Optional[str] = None
|
||||
paused: bool = False
|
||||
download_dir: Optional[str] = None
|
||||
category: Optional[str] = None
|
||||
|
||||
|
||||
class RSSFeed(BaseModel):
|
||||
name: str
|
||||
url: str
|
||||
category: Optional[str] = None
|
||||
auto_download: bool = False
|
||||
filter_pattern: Optional[str] = None
|
||||
|
||||
|
||||
class Category(BaseModel):
|
||||
name: str
|
||||
save_path: Optional[str] = None
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# Helpers
|
||||
# ============================================================================
|
||||
|
||||
def get_config() -> dict:
|
||||
"""Load torrent configuration."""
|
||||
if CONFIG_FILE.exists():
|
||||
try:
|
||||
import tomllib
|
||||
return tomllib.loads(CONFIG_FILE.read_text())
|
||||
except Exception:
|
||||
pass
|
||||
return DEFAULT_CONFIG.copy()
|
||||
|
||||
|
||||
def save_config(config: dict):
|
||||
"""Save torrent configuration."""
|
||||
CONFIG_FILE.parent.mkdir(parents=True, exist_ok=True)
|
||||
lines = ["# Torrent client configuration"]
|
||||
for k, v in config.items():
|
||||
if isinstance(v, bool):
|
||||
lines.append(f"{k} = {str(v).lower()}")
|
||||
elif isinstance(v, (int, float)):
|
||||
lines.append(f"{k} = {v}")
|
||||
elif isinstance(v, list):
|
||||
lines.append(f'{k} = {v}')
|
||||
else:
|
||||
lines.append(f'{k} = "{v}"')
|
||||
CONFIG_FILE.write_text("\n".join(lines) + "\n")
|
||||
|
||||
|
||||
def detect_runtime() -> Optional[str]:
|
||||
"""Detect container runtime."""
|
||||
if shutil.which("podman"):
|
||||
return "podman"
|
||||
if shutil.which("docker"):
|
||||
return "docker"
|
||||
return None
|
||||
|
||||
|
||||
def get_container_status() -> dict:
|
||||
"""Get torrent container status."""
|
||||
rt = detect_runtime()
|
||||
if not rt:
|
||||
return {"status": "no_runtime", "uptime": ""}
|
||||
|
||||
try:
|
||||
result = subprocess.run(
|
||||
[rt, "ps", "--filter", f"name={CONTAINER_NAME}", "--format", "{{.Status}}"],
|
||||
capture_output=True, text=True, timeout=5
|
||||
)
|
||||
if result.stdout.strip():
|
||||
return {"status": "running", "uptime": result.stdout.strip()}
|
||||
|
||||
result = subprocess.run(
|
||||
[rt, "ps", "-a", "--filter", f"name={CONTAINER_NAME}", "--format", "{{.Status}}"],
|
||||
capture_output=True, text=True, timeout=5
|
||||
)
|
||||
if result.stdout.strip():
|
||||
return {"status": "stopped", "uptime": ""}
|
||||
|
||||
return {"status": "not_installed", "uptime": ""}
|
||||
except Exception:
|
||||
return {"status": "error", "uptime": ""}
|
||||
|
||||
|
||||
def is_running() -> bool:
|
||||
"""Check if container is running."""
|
||||
return get_container_status()["status"] == "running"
|
||||
|
||||
|
||||
def get_rss_feeds() -> List[dict]:
|
||||
"""Get RSS feed subscriptions."""
|
||||
if RSS_FILE.exists():
|
||||
try:
|
||||
return json.loads(RSS_FILE.read_text())
|
||||
except Exception:
|
||||
pass
|
||||
return []
|
||||
|
||||
|
||||
def save_rss_feeds(feeds: List[dict]):
|
||||
"""Save RSS feeds."""
|
||||
RSS_FILE.parent.mkdir(parents=True, exist_ok=True)
|
||||
RSS_FILE.write_text(json.dumps(feeds, indent=2))
|
||||
|
||||
|
||||
def get_categories() -> List[dict]:
|
||||
"""Get torrent categories."""
|
||||
if CATEGORIES_FILE.exists():
|
||||
try:
|
||||
return json.loads(CATEGORIES_FILE.read_text())
|
||||
except Exception:
|
||||
pass
|
||||
# Default categories
|
||||
return [
|
||||
{"id": "movies", "name": "Movies", "save_path": "/downloads/movies"},
|
||||
{"id": "tv", "name": "TV Shows", "save_path": "/downloads/tv"},
|
||||
{"id": "music", "name": "Music", "save_path": "/downloads/music"},
|
||||
{"id": "software", "name": "Software", "save_path": "/downloads/software"},
|
||||
]
|
||||
|
||||
|
||||
def save_categories(categories: List[dict]):
|
||||
"""Save categories."""
|
||||
CATEGORIES_FILE.parent.mkdir(parents=True, exist_ok=True)
|
||||
CATEGORIES_FILE.write_text(json.dumps(categories, indent=2))
|
||||
|
||||
|
||||
def get_transmission_rpc_url() -> str:
|
||||
"""Get Transmission RPC URL."""
|
||||
cfg = get_config()
|
||||
return f"http://127.0.0.1:{cfg.get('port', 9091)}/transmission/rpc"
|
||||
|
||||
|
||||
def transmission_rpc(method: str, arguments: dict = None) -> dict:
|
||||
"""Call Transmission RPC API."""
|
||||
import urllib.request
|
||||
import urllib.error
|
||||
|
||||
cfg = get_config()
|
||||
url = get_transmission_rpc_url()
|
||||
session_id = ""
|
||||
|
||||
payload = {"method": method}
|
||||
if arguments:
|
||||
payload["arguments"] = arguments
|
||||
|
||||
headers = {
|
||||
"Content-Type": "application/json",
|
||||
"X-Transmission-Session-Id": session_id,
|
||||
}
|
||||
|
||||
try:
|
||||
# First request to get session ID
|
||||
req = urllib.request.Request(
|
||||
url,
|
||||
data=json.dumps(payload).encode(),
|
||||
headers=headers,
|
||||
method="POST"
|
||||
)
|
||||
urllib.request.urlopen(req, timeout=5)
|
||||
except urllib.error.HTTPError as e:
|
||||
if e.code == 409:
|
||||
# Get session ID from response
|
||||
session_id = e.headers.get("X-Transmission-Session-Id", "")
|
||||
headers["X-Transmission-Session-Id"] = session_id
|
||||
else:
|
||||
raise
|
||||
|
||||
# Retry with session ID
|
||||
req = urllib.request.Request(
|
||||
url,
|
||||
data=json.dumps(payload).encode(),
|
||||
headers=headers,
|
||||
method="POST"
|
||||
)
|
||||
response = urllib.request.urlopen(req, timeout=10)
|
||||
return json.loads(response.read().decode())
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# Cache for stats
|
||||
# ============================================================================
|
||||
|
||||
_stats_cache: dict = {}
|
||||
|
||||
|
||||
async def refresh_stats_cache():
|
||||
"""Background task to refresh statistics cache."""
|
||||
global _stats_cache
|
||||
while True:
|
||||
try:
|
||||
if is_running():
|
||||
# Get session stats from Transmission
|
||||
try:
|
||||
result = transmission_rpc("session-stats")
|
||||
if result.get("result") == "success":
|
||||
args = result.get("arguments", {})
|
||||
_stats_cache = {
|
||||
"download_speed": args.get("downloadSpeed", 0),
|
||||
"upload_speed": args.get("uploadSpeed", 0),
|
||||
"active_torrents": args.get("activeTorrentCount", 0),
|
||||
"paused_torrents": args.get("pausedTorrentCount", 0),
|
||||
"total_downloaded": args.get("cumulative-stats", {}).get("downloadedBytes", 0),
|
||||
"total_uploaded": args.get("cumulative-stats", {}).get("uploadedBytes", 0),
|
||||
"updated": datetime.now().isoformat(),
|
||||
}
|
||||
except Exception as e:
|
||||
log.debug(f"Stats cache refresh failed: {e}")
|
||||
else:
|
||||
_stats_cache = {}
|
||||
|
||||
# Cache to file
|
||||
CACHE_DIR.mkdir(parents=True, exist_ok=True)
|
||||
(CACHE_DIR / "stats.json").write_text(json.dumps(_stats_cache))
|
||||
except Exception as e:
|
||||
log.error(f"Stats cache error: {e}")
|
||||
|
||||
await asyncio.sleep(10)
|
||||
|
||||
|
||||
@app.on_event("startup")
|
||||
async def startup():
|
||||
asyncio.create_task(refresh_stats_cache())
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# Public Endpoints
|
||||
# ============================================================================
|
||||
|
||||
@router.get("/health")
|
||||
async def health():
|
||||
"""Health check."""
|
||||
return {"status": "ok", "module": "torrent"}
|
||||
|
||||
|
||||
@router.get("/status")
|
||||
def status():
|
||||
"""Get torrent service status."""
|
||||
cfg = get_config()
|
||||
rt = detect_runtime()
|
||||
container = get_container_status()
|
||||
rss_feeds = get_rss_feeds()
|
||||
categories = get_categories()
|
||||
|
||||
# Disk usage
|
||||
disk_usage = ""
|
||||
data_path = Path(cfg.get("data_path", "/srv/torrent"))
|
||||
if data_path.exists():
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["du", "-sh", str(data_path)],
|
||||
capture_output=True, text=True, timeout=10
|
||||
)
|
||||
disk_usage = result.stdout.split()[0] if result.stdout else ""
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
return {
|
||||
"enabled": cfg.get("enabled", False),
|
||||
"client": cfg.get("client", "transmission"),
|
||||
"image": cfg.get("image", "linuxserver/transmission:latest"),
|
||||
"port": cfg.get("port", 9091),
|
||||
"peer_port": cfg.get("peer_port", 51413),
|
||||
"data_path": cfg.get("data_path", "/srv/torrent"),
|
||||
"download_dir": cfg.get("download_dir", "/srv/torrent/downloads"),
|
||||
"timezone": cfg.get("timezone", "Europe/Paris"),
|
||||
"domain": cfg.get("domain", "torrent.secubox.local"),
|
||||
"haproxy": cfg.get("haproxy", False),
|
||||
"docker_available": rt is not None,
|
||||
"runtime": rt or "none",
|
||||
"container_status": container["status"],
|
||||
"container_uptime": container["uptime"],
|
||||
"disk_usage": disk_usage,
|
||||
"rss_count": len(rss_feeds),
|
||||
"category_count": len(categories),
|
||||
"download_limit": cfg.get("download_limit", 0),
|
||||
"upload_limit": cfg.get("upload_limit", 0),
|
||||
"alt_speed_enabled": cfg.get("alt_speed_enabled", False),
|
||||
}
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# Protected Endpoints — Configuration
|
||||
# ============================================================================
|
||||
|
||||
@router.get("/config")
|
||||
async def get_torrent_config(user=Depends(require_jwt)):
|
||||
"""Get torrent configuration."""
|
||||
return get_config()
|
||||
|
||||
|
||||
@router.post("/config")
|
||||
async def set_torrent_config(config: TorrentConfig, user=Depends(require_jwt)):
|
||||
"""Update torrent configuration."""
|
||||
cfg = get_config()
|
||||
cfg.update(config.dict())
|
||||
save_config(cfg)
|
||||
log.info(f"Config updated by {user.get('sub', 'unknown')}")
|
||||
return {"success": True}
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# Protected Endpoints — Torrents
|
||||
# ============================================================================
|
||||
|
||||
@router.get("/torrents")
|
||||
async def list_torrents(user=Depends(require_jwt)):
|
||||
"""List all torrents."""
|
||||
if not is_running():
|
||||
return {"torrents": [], "error": "Service not running"}
|
||||
|
||||
try:
|
||||
result = transmission_rpc("torrent-get", {
|
||||
"fields": [
|
||||
"id", "name", "status", "percentDone", "totalSize",
|
||||
"downloadedEver", "uploadedEver", "rateDownload", "rateUpload",
|
||||
"eta", "addedDate", "doneDate", "error", "errorString",
|
||||
"trackerStats", "labels", "downloadDir", "isFinished",
|
||||
"peersConnected", "seedRatioLimit", "uploadRatio"
|
||||
]
|
||||
})
|
||||
|
||||
if result.get("result") != "success":
|
||||
return {"torrents": [], "error": result.get("result")}
|
||||
|
||||
torrents = []
|
||||
for t in result.get("arguments", {}).get("torrents", []):
|
||||
status_map = {
|
||||
0: "stopped",
|
||||
1: "queued_verify",
|
||||
2: "verifying",
|
||||
3: "queued_download",
|
||||
4: "downloading",
|
||||
5: "queued_seed",
|
||||
6: "seeding",
|
||||
}
|
||||
torrents.append({
|
||||
"id": t["id"],
|
||||
"name": t["name"],
|
||||
"status": status_map.get(t["status"], "unknown"),
|
||||
"progress": round(t["percentDone"] * 100, 1),
|
||||
"size": t["totalSize"],
|
||||
"downloaded": t["downloadedEver"],
|
||||
"uploaded": t["uploadedEver"],
|
||||
"download_speed": t["rateDownload"],
|
||||
"upload_speed": t["rateUpload"],
|
||||
"eta": t["eta"] if t["eta"] >= 0 else None,
|
||||
"added": datetime.fromtimestamp(t["addedDate"]).isoformat() if t["addedDate"] else None,
|
||||
"completed": datetime.fromtimestamp(t["doneDate"]).isoformat() if t["doneDate"] else None,
|
||||
"error": t["errorString"] if t["error"] else None,
|
||||
"peers": t["peersConnected"],
|
||||
"ratio": round(t["uploadRatio"], 2),
|
||||
"category": t.get("labels", [""])[0] if t.get("labels") else "",
|
||||
"download_dir": t["downloadDir"],
|
||||
})
|
||||
|
||||
return {"torrents": torrents}
|
||||
|
||||
except Exception as e:
|
||||
log.error(f"Failed to list torrents: {e}")
|
||||
return {"torrents": [], "error": str(e)}
|
||||
|
||||
|
||||
@router.post("/torrent/add")
|
||||
async def add_torrent(
|
||||
request: AddTorrentRequest = None,
|
||||
torrent_file: UploadFile = File(None),
|
||||
magnet: str = Form(None),
|
||||
user=Depends(require_jwt)
|
||||
):
|
||||
"""Add a torrent from magnet link, URL, or file."""
|
||||
if not is_running():
|
||||
raise HTTPException(503, "Service not running")
|
||||
|
||||
try:
|
||||
args = {"paused": False}
|
||||
|
||||
# Handle different input types
|
||||
if torrent_file:
|
||||
# Upload torrent file
|
||||
import base64
|
||||
content = await torrent_file.read()
|
||||
args["metainfo"] = base64.b64encode(content).decode()
|
||||
elif magnet:
|
||||
args["filename"] = magnet
|
||||
elif request:
|
||||
if request.magnet:
|
||||
args["filename"] = request.magnet
|
||||
elif request.url:
|
||||
args["filename"] = request.url
|
||||
args["paused"] = request.paused
|
||||
if request.download_dir:
|
||||
args["download-dir"] = request.download_dir
|
||||
else:
|
||||
raise HTTPException(400, "No torrent source provided")
|
||||
|
||||
result = transmission_rpc("torrent-add", args)
|
||||
|
||||
if result.get("result") == "success":
|
||||
torrent_info = result.get("arguments", {})
|
||||
added = torrent_info.get("torrent-added") or torrent_info.get("torrent-duplicate")
|
||||
log.info(f"Torrent added by {user.get('sub', 'unknown')}: {added.get('name', 'unknown')}")
|
||||
return {
|
||||
"success": True,
|
||||
"torrent": {
|
||||
"id": added.get("id"),
|
||||
"name": added.get("name"),
|
||||
"hash": added.get("hashString"),
|
||||
}
|
||||
}
|
||||
else:
|
||||
return {"success": False, "error": result.get("result")}
|
||||
|
||||
except HTTPException:
|
||||
raise
|
||||
except Exception as e:
|
||||
log.error(f"Add torrent failed: {e}")
|
||||
return {"success": False, "error": str(e)}
|
||||
|
||||
|
||||
@router.delete("/torrent/{torrent_id}")
|
||||
async def remove_torrent(torrent_id: int, delete_data: bool = False, user=Depends(require_jwt)):
|
||||
"""Remove a torrent."""
|
||||
if not is_running():
|
||||
raise HTTPException(503, "Service not running")
|
||||
|
||||
try:
|
||||
result = transmission_rpc("torrent-remove", {
|
||||
"ids": [torrent_id],
|
||||
"delete-local-data": delete_data,
|
||||
})
|
||||
|
||||
if result.get("result") == "success":
|
||||
log.info(f"Torrent {torrent_id} removed by {user.get('sub', 'unknown')} (delete_data={delete_data})")
|
||||
return {"success": True}
|
||||
else:
|
||||
return {"success": False, "error": result.get("result")}
|
||||
|
||||
except Exception as e:
|
||||
log.error(f"Remove torrent failed: {e}")
|
||||
return {"success": False, "error": str(e)}
|
||||
|
||||
|
||||
@router.post("/torrent/{torrent_id}/pause")
|
||||
async def pause_torrent(torrent_id: int, user=Depends(require_jwt)):
|
||||
"""Pause a torrent."""
|
||||
if not is_running():
|
||||
raise HTTPException(503, "Service not running")
|
||||
|
||||
try:
|
||||
result = transmission_rpc("torrent-stop", {"ids": [torrent_id]})
|
||||
if result.get("result") == "success":
|
||||
log.info(f"Torrent {torrent_id} paused by {user.get('sub', 'unknown')}")
|
||||
return {"success": True}
|
||||
return {"success": False, "error": result.get("result")}
|
||||
except Exception as e:
|
||||
return {"success": False, "error": str(e)}
|
||||
|
||||
|
||||
@router.post("/torrent/{torrent_id}/resume")
|
||||
async def resume_torrent(torrent_id: int, user=Depends(require_jwt)):
|
||||
"""Resume a torrent."""
|
||||
if not is_running():
|
||||
raise HTTPException(503, "Service not running")
|
||||
|
||||
try:
|
||||
result = transmission_rpc("torrent-start", {"ids": [torrent_id]})
|
||||
if result.get("result") == "success":
|
||||
log.info(f"Torrent {torrent_id} resumed by {user.get('sub', 'unknown')}")
|
||||
return {"success": True}
|
||||
return {"success": False, "error": result.get("result")}
|
||||
except Exception as e:
|
||||
return {"success": False, "error": str(e)}
|
||||
|
||||
|
||||
@router.get("/torrent/{torrent_id}/files")
|
||||
async def get_torrent_files(torrent_id: int, user=Depends(require_jwt)):
|
||||
"""Get files in a torrent."""
|
||||
if not is_running():
|
||||
raise HTTPException(503, "Service not running")
|
||||
|
||||
try:
|
||||
result = transmission_rpc("torrent-get", {
|
||||
"ids": [torrent_id],
|
||||
"fields": ["files", "fileStats"]
|
||||
})
|
||||
|
||||
if result.get("result") != "success":
|
||||
return {"files": [], "error": result.get("result")}
|
||||
|
||||
torrents = result.get("arguments", {}).get("torrents", [])
|
||||
if not torrents:
|
||||
return {"files": [], "error": "Torrent not found"}
|
||||
|
||||
t = torrents[0]
|
||||
files = []
|
||||
for i, f in enumerate(t.get("files", [])):
|
||||
stats = t.get("fileStats", [])[i] if i < len(t.get("fileStats", [])) else {}
|
||||
files.append({
|
||||
"name": f["name"],
|
||||
"size": f["length"],
|
||||
"downloaded": f["bytesCompleted"],
|
||||
"progress": round(f["bytesCompleted"] / f["length"] * 100, 1) if f["length"] > 0 else 0,
|
||||
"wanted": stats.get("wanted", True),
|
||||
"priority": stats.get("priority", 0),
|
||||
})
|
||||
|
||||
return {"files": files}
|
||||
|
||||
except Exception as e:
|
||||
return {"files": [], "error": str(e)}
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# Statistics
|
||||
# ============================================================================
|
||||
|
||||
@router.get("/stats")
|
||||
async def get_stats(user=Depends(require_jwt)):
|
||||
"""Get download/upload statistics."""
|
||||
if _stats_cache:
|
||||
return _stats_cache
|
||||
|
||||
# Try loading from cache file
|
||||
cache_file = CACHE_DIR / "stats.json"
|
||||
if cache_file.exists():
|
||||
try:
|
||||
return json.loads(cache_file.read_text())
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
return {
|
||||
"download_speed": 0,
|
||||
"upload_speed": 0,
|
||||
"active_torrents": 0,
|
||||
"paused_torrents": 0,
|
||||
"total_downloaded": 0,
|
||||
"total_uploaded": 0,
|
||||
}
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# RSS Feeds
|
||||
# ============================================================================
|
||||
|
||||
@router.get("/rss/feeds")
|
||||
async def list_rss_feeds(user=Depends(require_jwt)):
|
||||
"""List RSS feed subscriptions."""
|
||||
return {"feeds": get_rss_feeds()}
|
||||
|
||||
|
||||
@router.post("/rss/add")
|
||||
async def add_rss_feed(feed: RSSFeed, user=Depends(require_jwt)):
|
||||
"""Add an RSS feed subscription."""
|
||||
feeds = get_rss_feeds()
|
||||
|
||||
# Check for duplicate
|
||||
for f in feeds:
|
||||
if f.get("url") == feed.url:
|
||||
return {"success": False, "error": "Feed URL already exists"}
|
||||
|
||||
feeds.append({
|
||||
"id": f"rss_{len(feeds)+1}_{int(datetime.now().timestamp())}",
|
||||
"name": feed.name,
|
||||
"url": feed.url,
|
||||
"category": feed.category,
|
||||
"auto_download": feed.auto_download,
|
||||
"filter_pattern": feed.filter_pattern,
|
||||
"added": datetime.now().isoformat(),
|
||||
})
|
||||
save_rss_feeds(feeds)
|
||||
|
||||
log.info(f"RSS feed added: {feed.name} by {user.get('sub', 'unknown')}")
|
||||
return {"success": True}
|
||||
|
||||
|
||||
@router.delete("/rss/{feed_id}")
|
||||
async def remove_rss_feed(feed_id: str, user=Depends(require_jwt)):
|
||||
"""Remove an RSS feed."""
|
||||
feeds = get_rss_feeds()
|
||||
new_feeds = [f for f in feeds if f.get("id") != feed_id]
|
||||
|
||||
if len(new_feeds) == len(feeds):
|
||||
return {"success": False, "error": "Feed not found"}
|
||||
|
||||
save_rss_feeds(new_feeds)
|
||||
log.info(f"RSS feed removed: {feed_id} by {user.get('sub', 'unknown')}")
|
||||
return {"success": True}
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# Categories
|
||||
# ============================================================================
|
||||
|
||||
@router.get("/categories")
|
||||
async def list_categories(user=Depends(require_jwt)):
|
||||
"""List torrent categories."""
|
||||
return {"categories": get_categories()}
|
||||
|
||||
|
||||
@router.post("/categories")
|
||||
async def add_category(category: Category, user=Depends(require_jwt)):
|
||||
"""Add a category."""
|
||||
categories = get_categories()
|
||||
|
||||
for c in categories:
|
||||
if c.get("name").lower() == category.name.lower():
|
||||
return {"success": False, "error": "Category already exists"}
|
||||
|
||||
categories.append({
|
||||
"id": category.name.lower().replace(" ", "_"),
|
||||
"name": category.name,
|
||||
"save_path": category.save_path or f"/downloads/{category.name.lower()}",
|
||||
})
|
||||
save_categories(categories)
|
||||
|
||||
log.info(f"Category added: {category.name} by {user.get('sub', 'unknown')}")
|
||||
return {"success": True}
|
||||
|
||||
|
||||
@router.delete("/categories/{category_id}")
|
||||
async def remove_category(category_id: str, user=Depends(require_jwt)):
|
||||
"""Remove a category."""
|
||||
categories = get_categories()
|
||||
new_cats = [c for c in categories if c.get("id") != category_id]
|
||||
|
||||
if len(new_cats) == len(categories):
|
||||
return {"success": False, "error": "Category not found"}
|
||||
|
||||
save_categories(new_cats)
|
||||
log.info(f"Category removed: {category_id} by {user.get('sub', 'unknown')}")
|
||||
return {"success": True}
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# Container Management
|
||||
# ============================================================================
|
||||
|
||||
@router.get("/container/status")
|
||||
async def container_status(user=Depends(require_jwt)):
|
||||
"""Get container status."""
|
||||
return get_container_status()
|
||||
|
||||
|
||||
@router.post("/container/install")
|
||||
def install_container(user=Depends(require_jwt)):
|
||||
"""Install torrent container."""
|
||||
rt = detect_runtime()
|
||||
if not rt:
|
||||
return {"success": False, "error": "No container runtime (docker/podman) found"}
|
||||
|
||||
cfg = get_config()
|
||||
data_path = Path(cfg.get("data_path", "/srv/torrent"))
|
||||
|
||||
# Create directories
|
||||
(data_path / "config").mkdir(parents=True, exist_ok=True)
|
||||
(data_path / "downloads").mkdir(parents=True, exist_ok=True)
|
||||
(data_path / "watch").mkdir(parents=True, exist_ok=True)
|
||||
|
||||
image = cfg.get("image", "linuxserver/transmission:latest")
|
||||
log.info(f"Installing torrent client ({image}) by {user.get('sub', 'unknown')}")
|
||||
|
||||
try:
|
||||
result = subprocess.run(
|
||||
[rt, "pull", image],
|
||||
capture_output=True, text=True, timeout=300
|
||||
)
|
||||
if result.returncode != 0:
|
||||
return {"success": False, "error": result.stderr.strip(), "output": result.stdout}
|
||||
|
||||
return {"success": True, "output": "Image pulled successfully"}
|
||||
except subprocess.TimeoutExpired:
|
||||
return {"success": False, "error": "Pull timeout"}
|
||||
except Exception as e:
|
||||
return {"success": False, "error": str(e)}
|
||||
|
||||
|
||||
@router.post("/container/start")
|
||||
async def start_container(user=Depends(require_jwt)):
|
||||
"""Start torrent container."""
|
||||
if is_running():
|
||||
return {"success": False, "error": "Already running"}
|
||||
|
||||
rt = detect_runtime()
|
||||
if not rt:
|
||||
return {"success": False, "error": "No container runtime"}
|
||||
|
||||
cfg = get_config()
|
||||
data_path = Path(cfg.get("data_path", "/srv/torrent"))
|
||||
port = cfg.get("port", 9091)
|
||||
peer_port = cfg.get("peer_port", 51413)
|
||||
image = cfg.get("image", "linuxserver/transmission:latest")
|
||||
tz = cfg.get("timezone", "Europe/Paris")
|
||||
|
||||
# Ensure directories exist
|
||||
(data_path / "config").mkdir(parents=True, exist_ok=True)
|
||||
(data_path / "downloads").mkdir(parents=True, exist_ok=True)
|
||||
(data_path / "watch").mkdir(parents=True, exist_ok=True)
|
||||
|
||||
cmd = [
|
||||
rt, "run", "-d",
|
||||
"--name", CONTAINER_NAME,
|
||||
"-v", f"{data_path}/config:/config",
|
||||
"-v", f"{data_path}/downloads:/downloads",
|
||||
"-v", f"{data_path}/watch:/watch",
|
||||
"-e", f"TZ={tz}",
|
||||
"-e", "PUID=1000",
|
||||
"-e", "PGID=1000",
|
||||
"-p", f"127.0.0.1:{port}:9091",
|
||||
"-p", f"{peer_port}:51413",
|
||||
"-p", f"{peer_port}:51413/udp",
|
||||
"--restart", "unless-stopped",
|
||||
]
|
||||
|
||||
cmd.append(image)
|
||||
|
||||
log.info(f"Starting torrent client by {user.get('sub', 'unknown')}")
|
||||
|
||||
try:
|
||||
subprocess.run([rt, "rm", "-f", CONTAINER_NAME], capture_output=True, timeout=10)
|
||||
result = subprocess.run(cmd, capture_output=True, text=True, timeout=60)
|
||||
await asyncio.sleep(3)
|
||||
|
||||
if is_running():
|
||||
return {"success": True}
|
||||
else:
|
||||
return {"success": False, "error": result.stderr.strip() or "Failed to start"}
|
||||
except Exception as e:
|
||||
return {"success": False, "error": str(e)}
|
||||
|
||||
|
||||
@router.post("/container/stop")
|
||||
async def stop_container(user=Depends(require_jwt)):
|
||||
"""Stop torrent container."""
|
||||
rt = detect_runtime()
|
||||
if not rt:
|
||||
return {"success": False, "error": "No container runtime"}
|
||||
|
||||
log.info(f"Stopping torrent client by {user.get('sub', 'unknown')}")
|
||||
|
||||
try:
|
||||
subprocess.run([rt, "stop", CONTAINER_NAME], capture_output=True, timeout=30)
|
||||
return {"success": True}
|
||||
except Exception as e:
|
||||
return {"success": False, "error": str(e)}
|
||||
|
||||
|
||||
@router.post("/container/restart")
|
||||
async def restart_container(user=Depends(require_jwt)):
|
||||
"""Restart torrent container."""
|
||||
await stop_container(user)
|
||||
await asyncio.sleep(2)
|
||||
return await start_container(user)
|
||||
|
||||
|
||||
@router.post("/container/uninstall")
|
||||
def uninstall_container(user=Depends(require_jwt)):
|
||||
"""Uninstall torrent container."""
|
||||
rt = detect_runtime()
|
||||
if not rt:
|
||||
return {"success": False, "error": "No container runtime"}
|
||||
|
||||
log.info(f"Uninstalling torrent client by {user.get('sub', 'unknown')}")
|
||||
|
||||
try:
|
||||
subprocess.run([rt, "stop", CONTAINER_NAME], capture_output=True, timeout=30)
|
||||
subprocess.run([rt, "rm", "-f", CONTAINER_NAME], capture_output=True, timeout=10)
|
||||
return {"success": True}
|
||||
except Exception as e:
|
||||
return {"success": False, "error": str(e)}
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# Logs
|
||||
# ============================================================================
|
||||
|
||||
@router.get("/logs")
|
||||
def get_logs(lines: int = 100, user=Depends(require_jwt)):
|
||||
"""Get container logs."""
|
||||
rt = detect_runtime()
|
||||
if not rt:
|
||||
return {"logs": "No container runtime"}
|
||||
|
||||
try:
|
||||
result = subprocess.run(
|
||||
[rt, "logs", "--tail", str(lines), CONTAINER_NAME],
|
||||
capture_output=True, text=True, timeout=10
|
||||
)
|
||||
logs = result.stdout + result.stderr
|
||||
return {"logs": logs}
|
||||
except Exception:
|
||||
return {"logs": "No logs available"}
|
||||
|
||||
|
||||
app.include_router(router)
|
||||
28
packages/secubox-torrent/conf/torrent.toml
Normal file
28
packages/secubox-torrent/conf/torrent.toml
Normal file
|
|
@ -0,0 +1,28 @@
|
|||
# /etc/secubox/torrent.toml — SecuBox Torrent module configuration
|
||||
#
|
||||
# Operator-facing override of the package defaults. Edit values then re-run
|
||||
# install-lxc.sh (Task 8's postinst exports these into TORRENT_* env vars
|
||||
# consumed by lxc/install-lxc.sh, which writes them into the in-LXC
|
||||
# /opt/secubox-torrent/torrent.env read by server.js).
|
||||
|
||||
# ── LXC ─────────────────────────────────────────────────────────────────
|
||||
[lxc]
|
||||
name = "torrent"
|
||||
ip = "10.100.0.160"
|
||||
gateway = "10.100.0.1"
|
||||
bridge = "br-lxc"
|
||||
path = "/data/lxc"
|
||||
|
||||
# ── WebTorrent engine (inside the LXC) ─────────────────────────────────
|
||||
[engine]
|
||||
max_active = 5
|
||||
webrtc = true # set false to force BitTorrent-only (no @roamhq/wrtc)
|
||||
|
||||
# ── Ephemeral retention (purge sweep, server.js runPurge) ──────────────
|
||||
[retention]
|
||||
ephemeral_ttl_hours = 6
|
||||
disk_floor_gb = 5
|
||||
|
||||
# ── Public exposure (nginx vhost :9080 → LXC :8090; HAProxy SNI ACL) ───
|
||||
[exposure]
|
||||
public_hostname = "torrent.gk2.secubox.in"
|
||||
|
|
@ -1,3 +1,11 @@
|
|||
secubox-torrent (2.0.0-1~bookworm1) bookworm; urgency=medium
|
||||
|
||||
* Pivot from Transmission to WebTorrent streaming (LXC-native, #917).
|
||||
* Browser player (HTTP Range), ephemeral-by-default + Keep library on /data.
|
||||
* Hybrid WebRTC+BitTorrent engine isolated in a dedicated LXC.
|
||||
|
||||
-- Gerald KERMA <devel@cybermind.fr> Tue, 28 Jul 2026 14:00:00 +0200
|
||||
|
||||
secubox-torrent (1.1.0-1~bookworm2) bookworm; urgency=medium
|
||||
|
||||
* Phase 2: Requires=secubox-core.service -> Wants= on this module's unit
|
||||
|
|
|
|||
|
|
@ -7,20 +7,9 @@ Standards-Version: 4.6.2
|
|||
|
||||
Package: secubox-torrent
|
||||
Architecture: all
|
||||
Depends: ${misc:Depends}, secubox-core (>= 1.0), python3-uvicorn | python3-pip
|
||||
Recommends: docker.io | podman
|
||||
Description: BitTorrent client management for SecuBox
|
||||
SecuBox module for managing BitTorrent downloads via Transmission.
|
||||
Provides Docker container management, torrent lifecycle (add/remove/
|
||||
pause/resume), RSS feed subscriptions, category management, and
|
||||
speed limiting.
|
||||
.
|
||||
Features:
|
||||
- Transmission daemon via Docker/Podman
|
||||
- Torrent add via magnet links, URLs, or .torrent files
|
||||
- Download/upload speed limiting with scheduling
|
||||
- RSS feed subscriptions for automatic downloads
|
||||
- Category/label management
|
||||
- P31 Phosphor light theme web UI
|
||||
.
|
||||
Provides FastAPI backend on /api/v1/torrent/ via Unix socket.
|
||||
Depends: ${misc:Depends}, secubox-core (>= 1.0), secubox-hub, lxc, debootstrap, nftables
|
||||
Description: WebTorrent streaming for SecuBox (LXC-native)
|
||||
Paste a magnet and stream it in the browser (HTTP Range) while it downloads.
|
||||
Ephemeral by default with an optional Keep to a persistent library on /data.
|
||||
The WebTorrent engine (hybrid WebRTC+BitTorrent) runs isolated in a dedicated
|
||||
LXC; the host only proxies an authenticated vhost.
|
||||
|
|
|
|||
|
|
@ -1,27 +1,53 @@
|
|||
#!/bin/bash
|
||||
#!/bin/sh
|
||||
# SPDX-License-Identifier: LicenseRef-CMSD-1.0
|
||||
# Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
|
||||
set -e
|
||||
case "$1" in
|
||||
configure)
|
||||
# Ensure secubox user exists
|
||||
# Ensure secubox user exists (shared across all secubox-* modules).
|
||||
id -u secubox >/dev/null 2>&1 || \
|
||||
adduser --system --group --no-create-home \
|
||||
--home /var/lib/secubox --shell /usr/sbin/nologin secubox
|
||||
# Create runtime directories
|
||||
|
||||
# Runtime + state directories (shared conventions; do not chown the
|
||||
# /etc/secubox parent here — secubox-core owns it).
|
||||
install -d -o root -g root -m 1777 /run/secubox
|
||||
install -d -o secubox -g secubox -m 755 /var/lib/secubox
|
||||
install -d -o secubox -g secubox -m 750 /var/cache/secubox/torrent
|
||||
# Create data directories
|
||||
install -d -m 755 /srv/torrent/config
|
||||
install -d -m 755 /srv/torrent/downloads
|
||||
install -d -m 755 /srv/torrent/watch
|
||||
# Ensure nginx secubox.d directory exists
|
||||
install -d -m 755 /etc/nginx/secubox.d
|
||||
# Enable and start service
|
||||
systemctl daemon-reload
|
||||
systemctl enable secubox-torrent.service
|
||||
systemctl start secubox-torrent.service || true
|
||||
# Reload nginx to pick up new location
|
||||
systemctl reload nginx 2>/dev/null || true
|
||||
[ -d /etc/secubox ] || install -d -o secubox -g secubox -m 755 /etc/secubox
|
||||
|
||||
# Parse the operator-facing TOML into the env vars install-lxc.sh (via
|
||||
# lxc/install-lxc.sh's torrent.env writer) expects. Minimal grep parse —
|
||||
# matches the flat `key = value` lines under [engine]/[retention].
|
||||
TORRENT_MAX_ACTIVE=$(grep -E '^max_active' /etc/secubox/torrent.toml 2>/dev/null | grep -oE '[0-9]+' | head -1)
|
||||
TORRENT_WEBRTC=$(grep -E '^webrtc' /etc/secubox/torrent.toml 2>/dev/null | grep -oE 'true|false' | head -1)
|
||||
TORRENT_TTL=$(grep -E '^ephemeral_ttl_hours' /etc/secubox/torrent.toml 2>/dev/null | grep -oE '[0-9]+' | head -1)
|
||||
TORRENT_FLOOR=$(grep -E '^disk_floor_gb' /etc/secubox/torrent.toml 2>/dev/null | grep -oE '[0-9]+' | head -1)
|
||||
export TORRENT_MAX_ACTIVE TORRENT_WEBRTC TORRENT_TTL TORRENT_FLOOR
|
||||
|
||||
# Provision/refresh the dedicated WebTorrent LXC (idempotent: creates
|
||||
# the container once, then redeploys the app + restarts the in-LXC
|
||||
# unit on every upgrade). Never fail the postinst if LXC tooling isn't
|
||||
# ready yet (e.g. first apt run before /data is mounted).
|
||||
bash /usr/lib/secubox/torrent/install-lxc.sh \
|
||||
|| echo "secubox-torrent: LXC provisioning deferred (run /usr/lib/secubox/torrent/install-lxc.sh manually)"
|
||||
|
||||
# Load the egress visibility scope for the torrent LXC's veth.
|
||||
if [ -f /etc/nftables.d/zz-torrent-egress.nft ]; then
|
||||
nft -f /etc/nftables.d/zz-torrent-egress.nft 2>/dev/null || true
|
||||
fi
|
||||
|
||||
# Symlink the public vhost into sites-enabled (idempotent). Proxies
|
||||
# torrent.gk2.secubox.in -> torrent LXC at 10.100.0.160:8090. Operator
|
||||
# wires the HAProxy SNI ACL + ACME cert + sbxwaf route separately.
|
||||
if [ -f /etc/nginx/sites-available/torrent.conf ] && [ -d /etc/nginx/sites-enabled ]; then
|
||||
ln -sf ../sites-available/torrent.conf /etc/nginx/sites-enabled/torrent.conf
|
||||
fi
|
||||
|
||||
if command -v nginx >/dev/null 2>&1 && nginx -t >/dev/null 2>&1; then
|
||||
systemctl reload nginx 2>/dev/null || true
|
||||
fi
|
||||
;;
|
||||
esac
|
||||
|
||||
#DEBHELPER#
|
||||
exit 0
|
||||
|
|
|
|||
17
packages/secubox-torrent/debian/postrm
Executable file
17
packages/secubox-torrent/debian/postrm
Executable file
|
|
@ -0,0 +1,17 @@
|
|||
#!/bin/sh
|
||||
# SPDX-License-Identifier: LicenseRef-CMSD-1.0
|
||||
# Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
|
||||
set -e
|
||||
case "$1" in
|
||||
purge)
|
||||
# Destroy the dedicated WebTorrent LXC and its provisioning sentinel.
|
||||
# Downloaded/kept media on /data/torrent is left in place (operator
|
||||
# data, not package state) — same convention as other LXC-native
|
||||
# modules (peertube, picobrew).
|
||||
lxc-stop -n torrent -P /data/lxc 2>/dev/null || true
|
||||
lxc-destroy -n torrent -P /data/lxc 2>/dev/null || true
|
||||
rm -f /var/lib/secubox/torrent/.lxc-provisioned 2>/dev/null || true
|
||||
;;
|
||||
esac
|
||||
#DEBHELPER#
|
||||
exit 0
|
||||
|
|
@ -1,14 +1,16 @@
|
|||
#!/bin/bash
|
||||
#!/bin/sh
|
||||
# SPDX-License-Identifier: LicenseRef-CMSD-1.0
|
||||
# Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
|
||||
set -e
|
||||
case "$1" in
|
||||
remove|upgrade)
|
||||
# Remove nginx location config
|
||||
rm -f /etc/nginx/secubox.d/torrent.conf
|
||||
# Stop and disable service
|
||||
systemctl stop secubox-torrent.service 2>/dev/null || true
|
||||
systemctl disable secubox-torrent.service 2>/dev/null || true
|
||||
# Reload nginx to remove location
|
||||
systemctl reload nginx 2>/dev/null || true
|
||||
remove|upgrade|deconfigure)
|
||||
# No host-side daemon to stop (the WebTorrent engine runs inside the
|
||||
# dedicated LXC, not as a host systemd unit). Just unwire the vhost.
|
||||
rm -f /etc/nginx/sites-enabled/torrent.conf
|
||||
if command -v nginx >/dev/null 2>&1 && nginx -t >/dev/null 2>&1; then
|
||||
systemctl reload nginx 2>/dev/null || true
|
||||
fi
|
||||
;;
|
||||
esac
|
||||
#DEBHELPER#
|
||||
exit 0
|
||||
|
|
|
|||
|
|
@ -3,18 +3,37 @@
|
|||
dh $@
|
||||
|
||||
override_dh_auto_install:
|
||||
# API files
|
||||
install -d debian/secubox-torrent/usr/lib/secubox/torrent/
|
||||
cp -r api debian/secubox-torrent/usr/lib/secubox/torrent/
|
||||
# Static www files
|
||||
install -d debian/secubox-torrent/usr/share/secubox/www
|
||||
[ -d www ] && cp -r www/. debian/secubox-torrent/usr/share/secubox/www/ || true
|
||||
# Menu definitions
|
||||
# WebTorrent LXC app source. Ships package-lock.json (needed by
|
||||
# `npm ci` at provision time inside the LXC) but NOT node_modules
|
||||
# (gitignored dev artifact; install-lxc.sh runs npm ci in the LXC).
|
||||
install -d debian/secubox-torrent/usr/lib/secubox/torrent/app
|
||||
cp -r lxc/app/. debian/secubox-torrent/usr/lib/secubox/torrent/app/
|
||||
rm -rf debian/secubox-torrent/usr/lib/secubox/torrent/app/node_modules
|
||||
# LXC bootstrap script + in-LXC systemd unit (installed side by side so
|
||||
# install-lxc.sh's $(dirname "$0") resolves the unit file on the board).
|
||||
install -m 755 lxc/install-lxc.sh debian/secubox-torrent/usr/lib/secubox/torrent/install-lxc.sh
|
||||
install -m 644 lxc/secubox-torrent.service debian/secubox-torrent/usr/lib/secubox/torrent/secubox-torrent.service
|
||||
# Webui (browser player)
|
||||
install -d debian/secubox-torrent/usr/share/secubox/www/torrent
|
||||
cp -r www/torrent/. debian/secubox-torrent/usr/share/secubox/www/torrent/
|
||||
# Menu definition
|
||||
install -d debian/secubox-torrent/usr/share/secubox/menu.d
|
||||
[ -d menu.d ] && cp -r menu.d/. debian/secubox-torrent/usr/share/secubox/menu.d/ || true
|
||||
# Modular nginx config
|
||||
install -d debian/secubox-torrent/etc/nginx/secubox.d
|
||||
[ -f nginx/torrent.conf ] && cp nginx/torrent.conf debian/secubox-torrent/etc/nginx/secubox.d/ || true
|
||||
# Systemd service
|
||||
install -d debian/secubox-torrent/lib/systemd/system
|
||||
[ -f systemd/secubox-torrent.service ] && cp systemd/secubox-torrent.service debian/secubox-torrent/lib/systemd/system/ || true
|
||||
install -m 644 menu.d/612-torrent.json debian/secubox-torrent/usr/share/secubox/menu.d/
|
||||
# Config
|
||||
install -d debian/secubox-torrent/etc/secubox
|
||||
install -m 644 conf/torrent.toml debian/secubox-torrent/etc/secubox/torrent.toml
|
||||
# Public vhost: full standalone server{} block (own server_name), so it
|
||||
# MUST land in sites-available/ (+ postinst symlink into sites-enabled),
|
||||
# NOT etc/nginx/secubox.d/ — that dir is location-only snippets merged
|
||||
# into the shared hub server block (see nginx/torrent.conf header and
|
||||
# packages/secubox-peertube/debian/rules for the verified pattern).
|
||||
install -d debian/secubox-torrent/etc/nginx/sites-available
|
||||
install -m 644 nginx/torrent.conf debian/secubox-torrent/etc/nginx/sites-available/torrent.conf
|
||||
# nft egress scope: ship straight to /etc/nftables.d/ with a zz- prefix
|
||||
# so it sorts after any table-creator base file (repo convention; see
|
||||
# secubox-p2p/secubox-toolbox/secubox-threatmesh debian/rules+postinst).
|
||||
# NOTE: /etc/secubox/nft.d/ is not an include path anywhere on the board
|
||||
# (/etc/nftables.conf only does `include "/etc/nftables.d/*.nft"`) — the
|
||||
# original task brief's path would ship a rule that never loads.
|
||||
install -d debian/secubox-torrent/etc/nftables.d
|
||||
install -m 644 nft/torrent-egress.nft debian/secubox-torrent/etc/nftables.d/zz-torrent-egress.nft
|
||||
|
|
|
|||
|
|
@ -4,14 +4,10 @@
|
|||
name: secubox-torrent
|
||||
category: misc
|
||||
tier: lite
|
||||
description: "BitTorrent client management for SecuBox"
|
||||
description: "WebTorrent streaming for SecuBox (LXC-native)"
|
||||
|
||||
depends:
|
||||
- secubox-core
|
||||
|
||||
api:
|
||||
socket: /run/secubox/torrent.sock
|
||||
health: /api/v1/torrent/health
|
||||
|
||||
ui:
|
||||
path: /srv/secubox/www/torrent
|
||||
|
|
|
|||
52
packages/secubox-torrent/lxc/SPIKE-RESULT.md
Normal file
52
packages/secubox-torrent/lxc/SPIKE-RESULT.md
Normal file
|
|
@ -0,0 +1,52 @@
|
|||
# secubox-torrent v2.0 — wrtc-arm64 spike result
|
||||
|
||||
**Date:** 2026-07-28
|
||||
**Ref:** #917
|
||||
|
||||
## Result
|
||||
|
||||
```
|
||||
WEBRTC_AVAILABLE=true
|
||||
```
|
||||
|
||||
`@roamhq/wrtc` loads a working `RTCPeerConnection` on arm64. The pinned
|
||||
dependency set in `app/package.json` (including `@roamhq/wrtc`) is kept
|
||||
as-is — no fallback removal needed.
|
||||
|
||||
## How it was validated
|
||||
|
||||
The dev workstation is x86-64 and cannot answer the arm64 question. Per
|
||||
the task-1 resolution, an EXISTING arm64 Node environment on gk2 was used
|
||||
instead of provisioning the `torrent` LXC on prod (that provisioning is
|
||||
Task 7's job):
|
||||
|
||||
- Host: gk2 (`ssh root@192.168.1.200`), Debian 12.14 aarch64.
|
||||
- Container: the `peertube` LXC (`lxc-attach -n peertube -P /data/lxc`),
|
||||
which already has Node installed.
|
||||
- Node version tested: **v22.22.2** (npm 10.9.7).
|
||||
- Scratch dir: `/tmp/wrtc-spike` inside the `peertube` LXC — `npm init -y`
|
||||
then `npm install @roamhq/wrtc`.
|
||||
- **npm install outcome:** clean, no errors. Resolved `@roamhq/wrtc@0.10.0`
|
||||
(brief pins `^0.8.0`; the installed prebuild satisfies the same major
|
||||
API and pulled in the `@roamhq/wrtc-linux-arm64` optional prebuild
|
||||
package — confirms a linux-arm64 native binding exists and installs
|
||||
without a source build). `npm warn deprecated domexception@4.0.0` only
|
||||
(transitive, non-blocking).
|
||||
- Ran the exact shipped `app/wrtc-probe.js` (ESM, matching this package's
|
||||
`"type": "module"`) against the scratch install: printed
|
||||
`WEBRTC_AVAILABLE=true`, exit 0, no crash.
|
||||
- Also ran an equivalent CommonJS one-liner
|
||||
(`require('@roamhq/wrtc'); new RTCPeerConnection(); pc.close()`)
|
||||
directly — same result, confirming the binding itself (not just the
|
||||
ESM interop wrapper) works.
|
||||
- Scratch dirs (`/tmp/wrtc-spike`, `/tmp/wrtc-spike2`) removed after the
|
||||
test; nothing persisted in the `peertube` LXC or on gk2.
|
||||
|
||||
## Consequence for later tasks
|
||||
|
||||
- `WEBRTC_AVAILABLE=true` is the default for `engine.js`'s `webrtc` mode
|
||||
in later tasks (webtorrent + WebRTC hybrid streaming, as designed).
|
||||
- `@roamhq/wrtc` stays in `app/package.json` — no removal required.
|
||||
- Final confirmation still happens when the dedicated `torrent` LXC is
|
||||
provisioned in Task 7 (this spike used a neighboring LXC's Node/OS
|
||||
environment as a stand-in, same host arch/Debian release/kernel).
|
||||
1
packages/secubox-torrent/lxc/app/.gitignore
vendored
Normal file
1
packages/secubox-torrent/lxc/app/.gitignore
vendored
Normal file
|
|
@ -0,0 +1 @@
|
|||
node_modules/
|
||||
59
packages/secubox-torrent/lxc/app/api.js
Normal file
59
packages/secubox-torrent/lxc/app/api.js
Normal file
|
|
@ -0,0 +1,59 @@
|
|||
// SPDX-License-Identifier: LicenseRef-CMSD-1.0
|
||||
// Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
|
||||
// SecuBox-Deb :: secubox-torrent :: api.js — CyberMind https://cybermind.fr
|
||||
//
|
||||
// Fastify control API: torrent lifecycle (add/list/keep/remove) + status/health,
|
||||
// plus the /stream mount that hands off to the Range streaming handler.
|
||||
|
||||
import Fastify from 'fastify';
|
||||
import path from 'node:path';
|
||||
import { handleStream } from './stream.js';
|
||||
|
||||
export function buildApi({ engine, library, diskFreeBytes }) {
|
||||
const app = Fastify({ logger: false });
|
||||
|
||||
app.get('/api/v1/torrent/health', async () => ({ status: 'ok' }));
|
||||
app.get('/api/v1/torrent/status', async () => ({
|
||||
active: engine.client.torrents.length, disk_free: diskFreeBytes(), webrtc: engine.webrtc !== false }));
|
||||
|
||||
app.post('/api/v1/torrent/add', async (req, reply) => {
|
||||
const magnet = (req.body || {}).magnet;
|
||||
if (!magnet) return reply.code(400).send({ error: 'magnet required' });
|
||||
let meta;
|
||||
try { meta = await engine.add(magnet); }
|
||||
catch (e) { return reply.code(504).send({ error: e.message }); }
|
||||
library.add({ infohash: meta.infohash, name: meta.name, magnet,
|
||||
path: path.join(engine.downloadDir, meta.infohash) });
|
||||
return meta;
|
||||
});
|
||||
|
||||
app.get('/api/v1/torrent/list', async () =>
|
||||
library.list().map(r => ({ ...r, stats: engine.stats(r.infohash) })));
|
||||
|
||||
app.get('/api/v1/torrent/files/:infohash', async (req, reply) => {
|
||||
const t = engine.get(req.params.infohash);
|
||||
if (!t) return reply.code(404).send({ error: 'not found' });
|
||||
return t.files.map((f, i) => ({ idx: i, name: f.name, length: f.length }));
|
||||
});
|
||||
|
||||
app.post('/api/v1/torrent/keep/:infohash', async (req, reply) => {
|
||||
const ih = req.params.infohash;
|
||||
if (!library.get(ih)) return reply.code(404).send({ error: 'not found' });
|
||||
library.keep(ih, path.join(engine.downloadDir, ih));
|
||||
return { status: 'kept' };
|
||||
});
|
||||
|
||||
app.post('/api/v1/torrent/remove/:infohash', async (req) => {
|
||||
engine.remove(req.params.infohash, { deleteData: true });
|
||||
library.remove(req.params.infohash);
|
||||
return { status: 'removed' };
|
||||
});
|
||||
|
||||
app.get('/stream/:infohash/:fileIdx', (req, reply) => {
|
||||
library.touch(req.params.infohash);
|
||||
return handleStream(engine, req, reply.raw)
|
||||
.then(() => { reply.hijack(); });
|
||||
});
|
||||
|
||||
return app;
|
||||
}
|
||||
45
packages/secubox-torrent/lxc/app/api.test.js
Normal file
45
packages/secubox-torrent/lxc/app/api.test.js
Normal file
|
|
@ -0,0 +1,45 @@
|
|||
// SPDX-License-Identifier: LicenseRef-CMSD-1.0
|
||||
// Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
|
||||
// SecuBox-Deb :: secubox-torrent :: api.test.js — CyberMind https://cybermind.fr
|
||||
//
|
||||
// Unit tests for the Fastify control API. Uses fastify .inject (no network),
|
||||
// the Engine with FakeWebTorrent, and an in-memory Library.
|
||||
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { buildApi } from './api.js';
|
||||
import { Library } from './library.js';
|
||||
import { Engine } from './engine.js';
|
||||
import { FakeWebTorrent } from './fakes.js';
|
||||
|
||||
function build() {
|
||||
const engine = new Engine({ WebTorrentCtor: FakeWebTorrent, downloadDir: '/tmp', maxActive: 5, webrtc: true });
|
||||
const library = new Library(':memory:');
|
||||
return buildApi({ engine, library, diskFreeBytes: () => 10 * 1e9 });
|
||||
}
|
||||
|
||||
test('health returns ok', async () => {
|
||||
const app = build();
|
||||
const r = await app.inject({ method: 'GET', url: '/api/v1/torrent/health' });
|
||||
assert.equal(r.statusCode, 200);
|
||||
assert.equal(r.json().status, 'ok');
|
||||
});
|
||||
|
||||
test('add then list returns the torrent', async () => {
|
||||
const app = build();
|
||||
await app.inject({ method: 'POST', url: '/api/v1/torrent/add',
|
||||
payload: { magnet: 'magnet:?xt=urn:btih:' + 'a'.repeat(40) } });
|
||||
const r = await app.inject({ method: 'GET', url: '/api/v1/torrent/list' });
|
||||
assert.equal(r.json().length, 1);
|
||||
assert.equal(r.json()[0].kept, 0);
|
||||
});
|
||||
|
||||
test('keep flips kept=1', async () => {
|
||||
const app = build();
|
||||
await app.inject({ method: 'POST', url: '/api/v1/torrent/add',
|
||||
payload: { magnet: 'magnet:?xt=urn:btih:' + 'a'.repeat(40) } });
|
||||
const r = await app.inject({ method: 'POST', url: '/api/v1/torrent/keep/' + 'a'.repeat(40) });
|
||||
assert.equal(r.statusCode, 200);
|
||||
const list = await app.inject({ method: 'GET', url: '/api/v1/torrent/list' });
|
||||
assert.equal(list.json()[0].kept, 1);
|
||||
});
|
||||
43
packages/secubox-torrent/lxc/app/engine.js
Normal file
43
packages/secubox-torrent/lxc/app/engine.js
Normal file
|
|
@ -0,0 +1,43 @@
|
|||
// SPDX-License-Identifier: LicenseRef-CMSD-1.0
|
||||
// Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
|
||||
// SecuBox-Deb :: secubox-torrent :: engine.js — CyberMind https://cybermind.fr
|
||||
//
|
||||
// WebTorrent engine wrapper: lifecycle, metadata, stats, peer/wire inspection.
|
||||
|
||||
export class Engine {
|
||||
constructor({ WebTorrentCtor, downloadDir, maxActive, webrtc }) {
|
||||
this.downloadDir = downloadDir; this.maxActive = maxActive;
|
||||
this.webrtc = webrtc;
|
||||
// webrtc=false (spike fallback) tells WebTorrent to skip the WebRTC transport.
|
||||
this.client = new WebTorrentCtor(webrtc ? {} : { tracker: { wrtc: false } });
|
||||
}
|
||||
add(magnet) {
|
||||
if (this.client.torrents.length >= this.maxActive) {
|
||||
return Promise.reject(new Error('max active torrents reached'));
|
||||
}
|
||||
return new Promise((resolve, reject) => {
|
||||
const to = setTimeout(() => reject(new Error('metadata timeout')), 60000);
|
||||
const t = this.client.add(magnet, { path: this.downloadDir }, () => {});
|
||||
const done = () => {
|
||||
clearTimeout(to);
|
||||
resolve({
|
||||
infohash: t.infoHash, name: t.name,
|
||||
files: t.files.map((f, i) => ({ idx: i, name: f.name, length: f.length, type: ext(f.name) })),
|
||||
});
|
||||
};
|
||||
t.on('metadata', done);
|
||||
if (t.files && t.files.length) done();
|
||||
});
|
||||
}
|
||||
get(infohash) { return this.client.get(infohash); }
|
||||
stats(infohash) {
|
||||
const t = this.get(infohash); if (!t) return null;
|
||||
return { progress: t.progress, downloadSpeed: t.downloadSpeed, uploadSpeed: t.uploadSpeed,
|
||||
numPeers: t.numPeers, wires: (t.wires || []).map(w => ({ type: w.type })) };
|
||||
}
|
||||
remove(infohash, { deleteData } = {}) {
|
||||
const t = this.get(infohash); if (t) t.destroy({ destroyStore: !!deleteData }, () => {});
|
||||
}
|
||||
}
|
||||
|
||||
function ext(name) { const m = /\.([A-Za-z0-9]+)$/.exec(name); return m ? m[1].toLowerCase() : ''; }
|
||||
44
packages/secubox-torrent/lxc/app/engine.test.js
Normal file
44
packages/secubox-torrent/lxc/app/engine.test.js
Normal file
|
|
@ -0,0 +1,44 @@
|
|||
// SPDX-License-Identifier: LicenseRef-CMSD-1.0
|
||||
// Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
|
||||
// SecuBox-Deb :: secubox-torrent :: engine.test.js — CyberMind https://cybermind.fr
|
||||
//
|
||||
// Unit tests for Engine (WebTorrent wrap). Uses fakes, no real network.
|
||||
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { Engine } from './engine.js';
|
||||
import { FakeWebTorrent, FakeTorrent } from './fakes.js';
|
||||
|
||||
test('add returns infohash, name and typed file list', async () => {
|
||||
const eng = new Engine({ WebTorrentCtor: FakeWebTorrent, downloadDir: '/tmp/x', maxActive: 5, webrtc: true });
|
||||
const r = await eng.add('magnet:?xt=urn:btih:' + 'a'.repeat(40));
|
||||
assert.equal(r.infohash, 'a'.repeat(40));
|
||||
assert.equal(r.files[0].type, 'mp4');
|
||||
assert.equal(r.files[0].idx, 0);
|
||||
});
|
||||
|
||||
test('stats returns wire types', () => {
|
||||
const eng = new Engine({ WebTorrentCtor: FakeWebTorrent, downloadDir: '/tmp/x', maxActive: 5, webrtc: true });
|
||||
return eng.add('magnet:?xt=urn:btih:' + 'a'.repeat(40)).then(r => {
|
||||
const s = eng.stats(r.infohash);
|
||||
assert.deepEqual(s.wires.map(w => w.type).sort(), ['tcp', 'webrtc']);
|
||||
assert.equal(s.numPeers, 3);
|
||||
});
|
||||
});
|
||||
|
||||
test('maxActive cap rejects beyond limit', async () => {
|
||||
const eng = new Engine({ WebTorrentCtor: FakeWebTorrent, downloadDir: '/tmp/x', maxActive: 1, webrtc: true });
|
||||
await eng.add('magnet:?xt=urn:btih:' + 'a'.repeat(40));
|
||||
eng.client.torrents.push({ infoHash: 'b'.repeat(40) }); // simulate a second active
|
||||
await assert.rejects(() => eng.add('magnet:?xt=urn:btih:' + 'c'.repeat(40)), /max active/);
|
||||
});
|
||||
|
||||
test('remove frees capacity for new add', async () => {
|
||||
const eng = new Engine({ WebTorrentCtor: FakeWebTorrent, downloadDir: '/tmp/x', maxActive: 1, webrtc: true });
|
||||
const r1 = await eng.add('magnet:?xt=urn:btih:' + 'a'.repeat(40));
|
||||
eng.remove(r1.infohash);
|
||||
// Set up FakeWebTorrent to return a different torrent for the second add
|
||||
eng.client._next = new FakeTorrent('b'.repeat(40), 'Fake2', [{ name: 'video.mp4', length: 200 }]);
|
||||
const r2 = await eng.add('magnet:?xt=urn:btih:' + 'b'.repeat(40));
|
||||
assert.equal(r2.infohash, 'b'.repeat(40));
|
||||
});
|
||||
37
packages/secubox-torrent/lxc/app/fakes.js
Normal file
37
packages/secubox-torrent/lxc/app/fakes.js
Normal file
|
|
@ -0,0 +1,37 @@
|
|||
// SPDX-License-Identifier: LicenseRef-CMSD-1.0
|
||||
// Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
|
||||
// SecuBox-Deb :: secubox-torrent :: fakes.js — CyberMind https://cybermind.fr
|
||||
//
|
||||
// Fake WebTorrent client + torrent objects for unit tests (no real network, no wrtc).
|
||||
|
||||
export class FakeFile {
|
||||
constructor(name, length) { this.name = name; this.length = length; }
|
||||
}
|
||||
|
||||
export class FakeTorrent {
|
||||
constructor(infoHash, name, files, { failMeta = false } = {}) {
|
||||
this.infoHash = infoHash; this.name = name;
|
||||
this.files = files.map((f, i) => Object.assign(new FakeFile(f.name, f.length), { idx: i }));
|
||||
this.progress = 0.1; this.downloadSpeed = 1000; this.uploadSpeed = 500;
|
||||
this.numPeers = 3; this.wires = [{ type: 'webrtc' }, { type: 'tcp' }];
|
||||
this._failMeta = failMeta; this._handlers = {}; this._client = null;
|
||||
}
|
||||
on(ev, cb) { this._handlers[ev] = cb; if (ev === 'metadata' && !this._failMeta) queueMicrotask(cb); }
|
||||
destroy(_opts, cb) {
|
||||
if (this._client) {
|
||||
const idx = this._client.torrents.indexOf(this);
|
||||
if (idx !== -1) this._client.torrents.splice(idx, 1);
|
||||
}
|
||||
if (cb) cb();
|
||||
}
|
||||
}
|
||||
|
||||
export class FakeWebTorrent {
|
||||
constructor() { this.torrents = []; }
|
||||
add(magnet, _opts, cb) {
|
||||
const t = this._next || new FakeTorrent('a'.repeat(40), 'Fake', [{ name: 'movie.mp4', length: 100 }]);
|
||||
t._client = this; this.torrents.push(t); if (cb) queueMicrotask(() => cb(t)); return t;
|
||||
}
|
||||
get(infohash) { return this.torrents.find(t => t.infoHash === infohash) || null; }
|
||||
destroy(cb) { if (cb) cb(); }
|
||||
}
|
||||
39
packages/secubox-torrent/lxc/app/library.js
Normal file
39
packages/secubox-torrent/lxc/app/library.js
Normal file
|
|
@ -0,0 +1,39 @@
|
|||
// SPDX-License-Identifier: LicenseRef-CMSD-1.0
|
||||
// Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
|
||||
// SecuBox-Deb :: secubox-torrent :: library.js — CyberMind https://cybermind.fr
|
||||
//
|
||||
// SQLite-backed torrent library: ephemeral (kept=0) vs. kept torrents, plus
|
||||
// the selection query used by the purge job to find stale ephemeral entries.
|
||||
|
||||
import Database from 'better-sqlite3';
|
||||
|
||||
export class Library {
|
||||
constructor(dbPath) {
|
||||
this.db = new Database(dbPath);
|
||||
this.db.exec(`CREATE TABLE IF NOT EXISTS torrents (
|
||||
infohash TEXT PRIMARY KEY, name TEXT, magnet TEXT, path TEXT,
|
||||
added_at INTEGER, last_played_at INTEGER, kept INTEGER DEFAULT 0)`);
|
||||
}
|
||||
add({ infohash, name, magnet, path }) {
|
||||
const now = Math.floor(Date.now() / 1000);
|
||||
// ON CONFLICT touches last_played_at only — re-adding an already-known
|
||||
// infohash (e.g. a kept magnet pasted again) must NOT clobber its
|
||||
// `kept` flag or `added_at`, or the purge sweep would later delete it.
|
||||
this.db.prepare(`INSERT INTO torrents
|
||||
(infohash,name,magnet,path,added_at,last_played_at,kept)
|
||||
VALUES (?,?,?,?,?,?,0)
|
||||
ON CONFLICT(infohash) DO UPDATE SET last_played_at=excluded.last_played_at`)
|
||||
.run(infohash, name, magnet, path, now, now);
|
||||
}
|
||||
list() { return this.db.prepare('SELECT * FROM torrents ORDER BY added_at DESC').all(); }
|
||||
get(infohash) { return this.db.prepare('SELECT * FROM torrents WHERE infohash=?').get(infohash) || null; }
|
||||
touch(infohash) { this.touchAt(infohash, Math.floor(Date.now() / 1000)); }
|
||||
touchAt(infohash, ts) { this.db.prepare('UPDATE torrents SET last_played_at=? WHERE infohash=?').run(ts, infohash); }
|
||||
keep(infohash, newPath) { this.db.prepare('UPDATE torrents SET kept=1, path=? WHERE infohash=?').run(newPath, infohash); }
|
||||
remove(infohash) { this.db.prepare('DELETE FROM torrents WHERE infohash=?').run(infohash); }
|
||||
expiredEphemeral(ttlSeconds, now = Math.floor(Date.now() / 1000)) {
|
||||
const cutoff = now - ttlSeconds;
|
||||
return this.db.prepare('SELECT infohash FROM torrents WHERE kept=0 AND last_played_at < ?')
|
||||
.all(cutoff).map(r => r.infohash);
|
||||
}
|
||||
}
|
||||
50
packages/secubox-torrent/lxc/app/library.test.js
Normal file
50
packages/secubox-torrent/lxc/app/library.test.js
Normal file
|
|
@ -0,0 +1,50 @@
|
|||
// SPDX-License-Identifier: LicenseRef-CMSD-1.0
|
||||
// Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
|
||||
// SecuBox-Deb :: secubox-torrent :: library.test.js — CyberMind https://cybermind.fr
|
||||
//
|
||||
// Tests for the SQLite-backed torrent library + ephemeral purge selection.
|
||||
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { Library } from './library.js';
|
||||
|
||||
const mk = () => new Library(':memory:');
|
||||
|
||||
test('add then list yields one ephemeral row', () => {
|
||||
const lib = mk();
|
||||
lib.add({ infohash: 'a', name: 'A', magnet: 'm', path: '/tmp/a' });
|
||||
const rows = lib.list();
|
||||
assert.equal(rows.length, 1);
|
||||
assert.equal(rows[0].kept, 0);
|
||||
});
|
||||
|
||||
test('keep flips kept and updates path', () => {
|
||||
const lib = mk();
|
||||
lib.add({ infohash: 'a', name: 'A', magnet: 'm', path: '/tmp/a' });
|
||||
lib.keep('a', '/data/torrent/library/a');
|
||||
assert.equal(lib.get('a').kept, 1);
|
||||
assert.equal(lib.get('a').path, '/data/torrent/library/a');
|
||||
});
|
||||
|
||||
test('re-adding an already-kept infohash does not un-keep it', () => {
|
||||
const lib = mk();
|
||||
lib.add({ infohash: 'a', name: 'A', magnet: 'm', path: '/tmp/a' });
|
||||
lib.touchAt('a', 1000);
|
||||
lib.keep('a', '/data/torrent/a');
|
||||
const addedAtBefore = lib.get('a').added_at;
|
||||
lib.add({ infohash: 'a', name: 'A', magnet: 'm', path: '/tmp/a' });
|
||||
const row = lib.get('a');
|
||||
assert.equal(row.kept, 1);
|
||||
assert.equal(row.added_at, addedAtBefore);
|
||||
});
|
||||
|
||||
test('expiredEphemeral returns only stale unkept torrents', () => {
|
||||
const lib = mk();
|
||||
lib.add({ infohash: 'old', name: 'O', magnet: 'm', path: '/tmp/o' });
|
||||
lib.add({ infohash: 'fresh', name: 'F', magnet: 'm', path: '/tmp/f' });
|
||||
lib.add({ infohash: 'kept', name: 'K', magnet: 'm', path: '/tmp/k' });
|
||||
lib.touchAt('old', 1000); lib.touchAt('fresh', 9000); lib.touchAt('kept', 1000);
|
||||
lib.keep('kept', '/data/k');
|
||||
const exp = lib.expiredEphemeral(3600, 10000); // now=10000, ttl=3600 → cutoff 6400
|
||||
assert.deepEqual(exp, ['old']);
|
||||
});
|
||||
3582
packages/secubox-torrent/lxc/app/package-lock.json
generated
Normal file
3582
packages/secubox-torrent/lxc/app/package-lock.json
generated
Normal file
File diff suppressed because it is too large
Load Diff
19
packages/secubox-torrent/lxc/app/package.json
Normal file
19
packages/secubox-torrent/lxc/app/package.json
Normal file
|
|
@ -0,0 +1,19 @@
|
|||
{
|
||||
"name": "secubox-torrent",
|
||||
"version": "2.0.0",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"engines": {
|
||||
"node": ">=18"
|
||||
},
|
||||
"scripts": {
|
||||
"test": "node --test"
|
||||
},
|
||||
"dependencies": {
|
||||
"@fastify/static": "^7.0.0",
|
||||
"@roamhq/wrtc": "^0.8.0",
|
||||
"better-sqlite3": "^11.3.0",
|
||||
"fastify": "^4.29.1",
|
||||
"webtorrent": "^2.5.1"
|
||||
}
|
||||
}
|
||||
116
packages/secubox-torrent/lxc/app/server.js
Normal file
116
packages/secubox-torrent/lxc/app/server.js
Normal file
|
|
@ -0,0 +1,116 @@
|
|||
// SPDX-License-Identifier: LicenseRef-CMSD-1.0
|
||||
// Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
|
||||
// SecuBox-Deb :: secubox-torrent :: server.js — CyberMind https://cybermind.fr
|
||||
//
|
||||
// Wires env → Engine/Library/buildApi, serves the player webui, and schedules
|
||||
// the ephemeral-torrent purge sweep. `webtorrent` and `@fastify/static` are
|
||||
// imported dynamically inside start() — this keeps runPurge (and this whole
|
||||
// module) importable in a plain `node --test` run with no native deps built
|
||||
// (webtorrent needs @roamhq/wrtc, which only builds on the target LXC).
|
||||
|
||||
import fs from 'node:fs';
|
||||
import { Engine } from './engine.js';
|
||||
import { Library } from './library.js';
|
||||
import { buildApi } from './api.js';
|
||||
|
||||
/**
|
||||
* Sweep ephemeral (kept=0) torrents that have outlived their TTL, plus —
|
||||
* if free disk has dropped below the configured floor — every remaining
|
||||
* ephemeral torrent regardless of age (kept torrents are NEVER swept).
|
||||
* Returns the list of infohashes removed.
|
||||
*/
|
||||
export function runPurge(engine, library, { ttlSeconds, diskFloorBytes, diskFreeBytes }) {
|
||||
const now = Math.floor(Date.now() / 1000);
|
||||
let victims = library.expiredEphemeral(ttlSeconds, now);
|
||||
if (diskFreeBytes() < diskFloorBytes) {
|
||||
const extra = library.list().filter(r => r.kept === 0).map(r => r.infohash);
|
||||
victims = [...new Set([...victims, ...extra])];
|
||||
}
|
||||
for (const ih of victims) {
|
||||
engine.remove(ih, { deleteData: true });
|
||||
library.remove(ih);
|
||||
}
|
||||
return victims;
|
||||
}
|
||||
|
||||
/** Free bytes available on /data — the real disk-floor signal (not RAM). */
|
||||
export function diskFreeBytesOnData() {
|
||||
try {
|
||||
const st = fs.statfsSync('/data');
|
||||
return st.bavail * st.bsize;
|
||||
} catch {
|
||||
return Infinity;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Re-sync the WebTorrent engine with the library DB at startup. Kept rows
|
||||
* survive a service/LXC restart only in the DB (the engine's in-memory
|
||||
* torrent list is empty on every process start) — re-add them so /stream
|
||||
* and purge see them again. Ephemeral (kept=0) rows can't be meaningfully
|
||||
* resumed (no seek/priority state was persisted) and would otherwise leak
|
||||
* their downloaded bytes forever once runPurge's engine.remove() becomes a
|
||||
* no-op for a torrent the engine never re-learned about — so they're
|
||||
* dropped from the library and their download dir is reclaimed here
|
||||
* instead of waiting on a purge sweep that can no longer see them.
|
||||
*/
|
||||
export function resumeLibrary(engine, library, { rmrf = defaultRmrf } = {}) {
|
||||
for (const row of library.list()) {
|
||||
if (row.kept) {
|
||||
try {
|
||||
// engine.add() is async in the real Engine (Promise) but the fake
|
||||
// used in tests may be sync — handle both without letting either
|
||||
// path abort the loop or leave an unhandled rejection.
|
||||
const result = engine.add(row.magnet);
|
||||
if (result && typeof result.catch === 'function') {
|
||||
result.catch((e) =>
|
||||
console.error(`secubox-torrent: resume failed for ${row.infohash}: ${e.message}`));
|
||||
}
|
||||
} catch (e) {
|
||||
console.error(`secubox-torrent: resume failed for ${row.infohash}: ${e.message}`);
|
||||
}
|
||||
} else {
|
||||
library.remove(row.infohash);
|
||||
try { rmrf(row.path); }
|
||||
catch (e) { console.error(`secubox-torrent: cleanup failed for ${row.path}: ${e.message}`); }
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function defaultRmrf(p) {
|
||||
if (p) fs.rmSync(p, { recursive: true, force: true });
|
||||
}
|
||||
|
||||
export async function start() {
|
||||
const cfg = {
|
||||
downloadDir: process.env.TORRENT_DOWNLOAD_DIR || '/data/torrent',
|
||||
maxActive: Number(process.env.TORRENT_MAX_ACTIVE || 5),
|
||||
webrtc: process.env.TORRENT_WEBRTC !== 'false',
|
||||
port: Number(process.env.TORRENT_PORT || 8090),
|
||||
ttl: Number(process.env.TORRENT_EPHEMERAL_TTL_HOURS || 6) * 3600,
|
||||
floor: Number(process.env.TORRENT_DISK_FLOOR_GB || 5) * 1e9,
|
||||
};
|
||||
|
||||
const { default: WebTorrent } = await import('webtorrent');
|
||||
const engine = new Engine({ WebTorrentCtor: WebTorrent, ...cfg });
|
||||
const library = new Library(`${cfg.downloadDir}/library.db`);
|
||||
const diskFreeBytes = diskFreeBytesOnData;
|
||||
|
||||
resumeLibrary(engine, library);
|
||||
|
||||
const app = buildApi({ engine, library, diskFreeBytes });
|
||||
|
||||
const fastifyStatic = await import('@fastify/static');
|
||||
await app.register(fastifyStatic.default, { root: '/opt/secubox-torrent/www' });
|
||||
|
||||
setInterval(() => {
|
||||
runPurge(engine, library, { ttlSeconds: cfg.ttl, diskFloorBytes: cfg.floor, diskFreeBytes });
|
||||
}, 300000);
|
||||
|
||||
await app.listen({ host: '0.0.0.0', port: cfg.port });
|
||||
return app;
|
||||
}
|
||||
|
||||
if (process.argv[1] && process.argv[1].endsWith('server.js')) {
|
||||
start().catch((err) => { console.error(err); process.exit(1); });
|
||||
}
|
||||
99
packages/secubox-torrent/lxc/app/server.test.js
Normal file
99
packages/secubox-torrent/lxc/app/server.test.js
Normal file
|
|
@ -0,0 +1,99 @@
|
|||
// SPDX-License-Identifier: LicenseRef-CMSD-1.0
|
||||
// Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
|
||||
// SecuBox-Deb :: secubox-torrent :: server.test.js — CyberMind https://cybermind.fr
|
||||
//
|
||||
// Unit tests for the purge sweep (runPurge) only. server.js's start() imports
|
||||
// webtorrent/@fastify/static dynamically inside the function body, so this
|
||||
// file never pulls those heavy/native deps in — no network, no real disk.
|
||||
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { runPurge, resumeLibrary } from './server.js';
|
||||
import { Library } from './library.js';
|
||||
|
||||
test('purge removes expired ephemeral and calls engine.remove', () => {
|
||||
const lib = new Library(':memory:');
|
||||
lib.add({ infohash: 'old', name: 'O', magnet: 'm', path: '/tmp/o' });
|
||||
lib.touchAt('old', 0);
|
||||
const removed = [];
|
||||
const engine = { remove: (ih) => removed.push(ih) };
|
||||
const swept = runPurge(engine, lib, { ttlSeconds: 10, diskFloorBytes: 0, diskFreeBytes: () => 1e12 });
|
||||
assert.deepEqual(swept, ['old']);
|
||||
assert.deepEqual(removed, ['old']);
|
||||
assert.equal(lib.get('old'), null);
|
||||
});
|
||||
|
||||
test('purge keeps fresh ephemeral torrents under the disk floor', () => {
|
||||
const lib = new Library(':memory:');
|
||||
lib.add({ infohash: 'fresh', name: 'F', magnet: 'm', path: '/tmp/f' });
|
||||
const removed = [];
|
||||
const engine = { remove: (ih) => removed.push(ih) };
|
||||
const swept = runPurge(engine, lib, { ttlSeconds: 3600, diskFloorBytes: 0, diskFreeBytes: () => 1e12 });
|
||||
assert.deepEqual(swept, []);
|
||||
assert.deepEqual(removed, []);
|
||||
assert.ok(lib.get('fresh'));
|
||||
});
|
||||
|
||||
test('purge never sweeps kept torrents even when stale', () => {
|
||||
const lib = new Library(':memory:');
|
||||
lib.add({ infohash: 'keeper', name: 'K', magnet: 'm', path: '/tmp/k' });
|
||||
lib.keep('keeper', '/tmp/k-kept');
|
||||
lib.touchAt('keeper', 0);
|
||||
const removed = [];
|
||||
const engine = { remove: (ih) => removed.push(ih) };
|
||||
const swept = runPurge(engine, lib, { ttlSeconds: 10, diskFloorBytes: 0, diskFreeBytes: () => 1e12 });
|
||||
assert.deepEqual(swept, []);
|
||||
assert.deepEqual(removed, []);
|
||||
assert.ok(lib.get('keeper'));
|
||||
});
|
||||
|
||||
test('purge below the disk floor also sweeps fresh ephemeral entries (but never kept ones)', () => {
|
||||
const lib = new Library(':memory:');
|
||||
lib.add({ infohash: 'fresh', name: 'F', magnet: 'm', path: '/tmp/f' });
|
||||
lib.add({ infohash: 'keeper', name: 'K', magnet: 'm', path: '/tmp/k' });
|
||||
lib.keep('keeper', '/tmp/k-kept');
|
||||
const removed = [];
|
||||
const engine = { remove: (ih) => removed.push(ih) };
|
||||
const swept = runPurge(engine, lib, { ttlSeconds: 3600, diskFloorBytes: 10e9, diskFreeBytes: () => 1e9 });
|
||||
assert.deepEqual(swept, ['fresh']);
|
||||
assert.deepEqual(removed, ['fresh']);
|
||||
assert.equal(lib.get('fresh'), null);
|
||||
assert.ok(lib.get('keeper'));
|
||||
});
|
||||
|
||||
test('resumeLibrary re-adds kept torrents to the engine and reaps ephemeral ones', () => {
|
||||
const lib = new Library(':memory:');
|
||||
lib.add({ infohash: 'keeper', name: 'K', magnet: 'magnet:keeper', path: '/data/torrent/keeper' });
|
||||
lib.keep('keeper', '/data/torrent/keeper');
|
||||
lib.add({ infohash: 'eph', name: 'E', magnet: 'magnet:eph', path: '/data/torrent/eph' });
|
||||
|
||||
const added = [];
|
||||
const engine = { add: (magnet) => { added.push(magnet); } };
|
||||
const rmrfCalls = [];
|
||||
const rmrf = (p) => rmrfCalls.push(p);
|
||||
|
||||
resumeLibrary(engine, lib, { rmrf });
|
||||
|
||||
assert.deepEqual(added, ['magnet:keeper']);
|
||||
assert.ok(lib.get('keeper'));
|
||||
assert.equal(lib.get('eph'), null);
|
||||
assert.deepEqual(rmrfCalls, ['/data/torrent/eph']);
|
||||
});
|
||||
|
||||
test('resumeLibrary does not abort the loop if engine.add throws for one kept row', () => {
|
||||
const lib = new Library(':memory:');
|
||||
lib.add({ infohash: 'bad', name: 'B', magnet: 'magnet:bad', path: '/data/torrent/bad' });
|
||||
lib.keep('bad', '/data/torrent/bad');
|
||||
lib.add({ infohash: 'good', name: 'G', magnet: 'magnet:good', path: '/data/torrent/good' });
|
||||
lib.keep('good', '/data/torrent/good');
|
||||
|
||||
const added = [];
|
||||
const engine = {
|
||||
add: (magnet) => {
|
||||
if (magnet === 'magnet:bad') throw new Error('boom');
|
||||
added.push(magnet);
|
||||
},
|
||||
};
|
||||
resumeLibrary(engine, lib, { rmrf: () => {} });
|
||||
assert.deepEqual(added, ['magnet:good']);
|
||||
});
|
||||
44
packages/secubox-torrent/lxc/app/stream.js
Normal file
44
packages/secubox-torrent/lxc/app/stream.js
Normal file
|
|
@ -0,0 +1,44 @@
|
|||
// SPDX-License-Identifier: LicenseRef-CMSD-1.0
|
||||
// Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
|
||||
// Source-Disclosed License — All rights reserved except as expressly granted.
|
||||
// See LICENCE-CMSD-1.0.md for terms.
|
||||
|
||||
/**
|
||||
* SecuBox-Deb :: secubox-torrent HTTP Range streaming handler
|
||||
* CyberMind — https://cybermind.fr
|
||||
*/
|
||||
|
||||
const MIME = { mp4: 'video/mp4', webm: 'video/webm', mkv: 'video/x-matroska',
|
||||
m4v: 'video/mp4', mp3: 'audio/mpeg', m4a: 'audio/mp4', ogg: 'audio/ogg',
|
||||
flac: 'audio/flac', wav: 'audio/wav' };
|
||||
|
||||
export async function handleStream(engine, req, res) {
|
||||
const t = engine.get(req.params.infohash);
|
||||
const idx = Number(req.params.fileIdx);
|
||||
const file = t && t.files && t.files[idx];
|
||||
if (!file) { res.writeHead(404); res.end('not found'); return; }
|
||||
const type = mimeOf(file.name);
|
||||
const total = file.length;
|
||||
const range = req.headers.range;
|
||||
if (!range) {
|
||||
res.writeHead(200, { 'Content-Length': total, 'Content-Type': type, 'Accept-Ranges': 'bytes' });
|
||||
const stream = file.createReadStream({ start: 0, end: total - 1 });
|
||||
stream.on('error', () => { try { res.destroy(); } catch {} });
|
||||
stream.pipe(res);
|
||||
return;
|
||||
}
|
||||
const m = /bytes=(\d+)-(\d*)/.exec(range);
|
||||
let start = m ? Number(m[1]) : 0;
|
||||
let end = m && m[2] ? Number(m[2]) : total - 1;
|
||||
if (end >= total) end = total - 1;
|
||||
if (start >= total || start > end) {
|
||||
res.writeHead(416, { 'Content-Range': `bytes */${total}` }); res.end(); return;
|
||||
}
|
||||
res.writeHead(206, { 'Content-Range': `bytes ${start}-${end}/${total}`,
|
||||
'Content-Length': end - start + 1, 'Content-Type': type, 'Accept-Ranges': 'bytes' });
|
||||
const stream = file.createReadStream({ start, end });
|
||||
stream.on('error', () => { try { res.destroy(); } catch {} });
|
||||
stream.pipe(res);
|
||||
}
|
||||
|
||||
function mimeOf(name) { const e = (/\.([A-Za-z0-9]+)$/.exec(name) || [])[1]; return MIME[(e || '').toLowerCase()] || 'application/octet-stream'; }
|
||||
74
packages/secubox-torrent/lxc/app/stream.test.js
Normal file
74
packages/secubox-torrent/lxc/app/stream.test.js
Normal file
|
|
@ -0,0 +1,74 @@
|
|||
// SPDX-License-Identifier: LicenseRef-CMSD-1.0
|
||||
// Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
|
||||
// Source-Disclosed License — All rights reserved except as expressly granted.
|
||||
// See LICENCE-CMSD-1.0.md for terms.
|
||||
|
||||
/**
|
||||
* SecuBox-Deb :: secubox-torrent stream handler tests
|
||||
* CyberMind — https://cybermind.fr
|
||||
*/
|
||||
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { Readable } from 'node:stream';
|
||||
import { handleStream } from './stream.js';
|
||||
|
||||
function fakeRes() {
|
||||
const listeners = {};
|
||||
return { statusCode: 200, headers: {}, ended: false, body: '',
|
||||
setHeader(k, v) { this.headers[k.toLowerCase()] = v; },
|
||||
writeHead(c, h) { this.statusCode = c; Object.assign(this.headers, lower(h || {})); },
|
||||
end(s) { if (s) this.body += s; this.ended = true; },
|
||||
on(event, listener) { if (!listeners[event]) listeners[event] = []; listeners[event].push(listener); return this; },
|
||||
once(event, listener) { const wrapped = (...args) => { listener(...args); const idx = listeners[event].indexOf(wrapped); if (idx >= 0) listeners[event].splice(idx, 1); }; if (!listeners[event]) listeners[event] = []; listeners[event].push(wrapped); return this; },
|
||||
write(chunk, encoding, cb) { this.body += chunk; if (cb) cb(); return true; },
|
||||
emit(event, data) { if (listeners[event]) listeners[event].forEach(l => l(data)); } };
|
||||
}
|
||||
const lower = o => Object.fromEntries(Object.entries(o).map(([k, v]) => [k.toLowerCase(), v]));
|
||||
function fakeEngine(len = 100) {
|
||||
const file = { name: 'movie.mp4', length: len,
|
||||
createReadStream: ({ start, end }) => Readable.from([`bytes[${start}-${end}]`]) };
|
||||
return { get: () => ({ files: [file] }) };
|
||||
}
|
||||
|
||||
test('full request returns 200 with content-length and type', async () => {
|
||||
const res = fakeRes();
|
||||
await handleStream(fakeEngine(100), { params: { infohash: 'a', fileIdx: '0' }, headers: {} }, res);
|
||||
assert.equal(res.statusCode, 200);
|
||||
assert.equal(res.headers['content-length'], 100);
|
||||
assert.match(res.headers['content-type'], /mp4/);
|
||||
});
|
||||
|
||||
test('range request returns 206 with content-range', async () => {
|
||||
const res = fakeRes();
|
||||
await handleStream(fakeEngine(100), { params: { infohash: 'a', fileIdx: '0' }, headers: { range: 'bytes=10-19' } }, res);
|
||||
assert.equal(res.statusCode, 206);
|
||||
assert.equal(res.headers['content-range'], 'bytes 10-19/100');
|
||||
assert.equal(res.headers['content-length'], 10);
|
||||
});
|
||||
|
||||
test('unsatisfiable range returns 416', async () => {
|
||||
const res = fakeRes();
|
||||
await handleStream(fakeEngine(100), { params: { infohash: 'a', fileIdx: '0' }, headers: { range: 'bytes=200-300' } }, res);
|
||||
assert.equal(res.statusCode, 416);
|
||||
});
|
||||
|
||||
test('unknown infohash returns 404', async () => {
|
||||
const res = fakeRes();
|
||||
await handleStream({ get: () => null }, { params: { infohash: 'z', fileIdx: '0' }, headers: {} }, res);
|
||||
assert.equal(res.statusCode, 404);
|
||||
});
|
||||
|
||||
test('range clamping: bytes=50-999999 on 100-byte file returns 206 with clamped end', async () => {
|
||||
const res = fakeRes();
|
||||
await handleStream(fakeEngine(100), { params: { infohash: 'a', fileIdx: '0' }, headers: { range: 'bytes=50-999999' } }, res);
|
||||
assert.equal(res.statusCode, 206);
|
||||
assert.equal(res.headers['content-range'], 'bytes 50-99/100');
|
||||
assert.equal(res.headers['content-length'], 50);
|
||||
});
|
||||
|
||||
test('out-of-range fileIdx returns 404', async () => {
|
||||
const res = fakeRes();
|
||||
await handleStream(fakeEngine(100), { params: { infohash: 'a', fileIdx: '5' }, headers: {} }, res);
|
||||
assert.equal(res.statusCode, 404);
|
||||
});
|
||||
15
packages/secubox-torrent/lxc/app/wrtc-probe.js
Normal file
15
packages/secubox-torrent/lxc/app/wrtc-probe.js
Normal file
|
|
@ -0,0 +1,15 @@
|
|||
// SPDX-License-Identifier: LicenseRef-CMSD-1.0
|
||||
// Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
|
||||
// SecuBox-Deb :: secubox-torrent :: wrtc-probe.js — CyberMind https://cybermind.fr
|
||||
//
|
||||
// Exits 0 and prints WEBRTC_AVAILABLE=true if @roamhq/wrtc loads a
|
||||
// RTCPeerConnection on this arch; exits 0 with =false otherwise (fallback).
|
||||
try {
|
||||
const wrtc = await import('@roamhq/wrtc');
|
||||
const pc = new wrtc.default.RTCPeerConnection();
|
||||
pc.close();
|
||||
console.log('WEBRTC_AVAILABLE=true');
|
||||
} catch (e) {
|
||||
console.log('WEBRTC_AVAILABLE=false');
|
||||
console.error('wrtc load failed:', e.message);
|
||||
}
|
||||
93
packages/secubox-torrent/lxc/install-lxc.sh
Executable file
93
packages/secubox-torrent/lxc/install-lxc.sh
Executable file
|
|
@ -0,0 +1,93 @@
|
|||
#!/usr/bin/env bash
|
||||
# SPDX-License-Identifier: LicenseRef-CMSD-1.0
|
||||
# SecuBox-Deb :: secubox-torrent :: install-lxc.sh
|
||||
# CyberMind — https://cybermind.fr
|
||||
#
|
||||
# Idempotent LXC bootstrap + app deploy for the secubox-torrent module
|
||||
# (v2.0, WebTorrent streaming pivot). Creates the dedicated Debian LXC,
|
||||
# installs Node, then deploys the WebTorrent engine (webtorrent + fastify
|
||||
# + better-sqlite3, optionally @roamhq/wrtc — see SPIKE-RESULT.md) and its
|
||||
# in-LXC systemd unit. The host nginx vhost (../nginx/torrent.conf) proxies
|
||||
# torrent.gk2.secubox.in to this LXC at $LXC_IP:8090.
|
||||
set -euo pipefail
|
||||
readonly LXC_NAME="${SECUBOX_LXC_NAME:-torrent}"
|
||||
readonly LXC_IP="${SECUBOX_LXC_IP:-10.100.0.160}"
|
||||
readonly LXC_PATH="${SECUBOX_LXC_PATH:-/data/lxc}"
|
||||
readonly LXC_BRIDGE="${SECUBOX_LXC_BRIDGE:-br-lxc}"
|
||||
readonly LXC_GW="${SECUBOX_LXC_GW:-10.100.0.1}"
|
||||
# Predictable veth pair name (default LXC naming is a random vethXXXXXX on
|
||||
# every start) — the host nft egress drop-in (../nft/torrent-egress.nft)
|
||||
# matches on this exact name.
|
||||
readonly LXC_VETH="${SECUBOX_LXC_VETH:-veth-torrent0}"
|
||||
readonly DATA_DIR="${SECUBOX_DATA_DIR:-/data/torrent}"
|
||||
readonly STATE_DIR="${SECUBOX_STATE_DIR:-/var/lib/secubox/torrent}"
|
||||
readonly SENTINEL="$STATE_DIR/.lxc-provisioned"
|
||||
# Unprivileged LXC: container-root maps to this host UID (idmap below). The
|
||||
# bind-mounted /data/torrent must be owned by it so the container can write.
|
||||
readonly LXC_ROOT_UID="${SECUBOX_LXC_ROOT_UID:-100000}"
|
||||
log() { printf '[torrent-install] %s\n' "$*"; }
|
||||
la() { lxc-attach -n "$LXC_NAME" -P "$LXC_PATH" -- "$@"; }
|
||||
|
||||
mkdir -p "$STATE_DIR" "$DATA_DIR"
|
||||
chown -R "$LXC_ROOT_UID:$LXC_ROOT_UID" "$DATA_DIR"
|
||||
if [ -f "$SENTINEL" ]; then log "already provisioned; skipping create"; else
|
||||
# Use the DOWNLOAD template (not -t debian): gk2 runs UNPRIVILEGED containers
|
||||
# and the debian template refuses ("can't be used for unprivileged
|
||||
# containers"). Download template + idmap is the working pattern (matches
|
||||
# secubox-peertube).
|
||||
lxc-create -n "$LXC_NAME" -t download -P "$LXC_PATH" -- \
|
||||
--dist debian --release bookworm --arch "$(dpkg --print-architecture)"
|
||||
# idmap: the download template copies /etc/lxc/default.conf, which on gk2
|
||||
# already provides the unprivileged u/g 0->100000 65536 mapping. Only add it
|
||||
# if absent — a DUPLICATE mapping makes newuidmap fail ("write to uid_map
|
||||
# failed: Invalid argument") and the container ABORTS at start.
|
||||
if ! grep -q '^lxc.idmap' "$LXC_PATH/$LXC_NAME/config"; then
|
||||
printf 'lxc.idmap = u 0 %s 65536\nlxc.idmap = g 0 %s 65536\n' \
|
||||
"$LXC_ROOT_UID" "$LXC_ROOT_UID" >> "$LXC_PATH/$LXC_NAME/config"
|
||||
fi
|
||||
# static IP on br-lxc (config appended)
|
||||
cat >> "$LXC_PATH/$LXC_NAME/config" <<EOF
|
||||
lxc.net.0.type = veth
|
||||
lxc.net.0.link = $LXC_BRIDGE
|
||||
lxc.net.0.flags = up
|
||||
lxc.net.0.veth.pair = $LXC_VETH
|
||||
lxc.net.0.ipv4.address = $LXC_IP/24
|
||||
lxc.net.0.ipv4.gateway = $LXC_GW
|
||||
|
||||
# Persist downloaded torrent data + the library DB on host storage (survives
|
||||
# LXC re-provisioning). server.js also statfs()'s /data for the disk-floor
|
||||
# purge signal, so this must be a real mount, not container-overlay space.
|
||||
lxc.mount.entry = $DATA_DIR data/torrent none bind,create=dir 0 0
|
||||
EOF
|
||||
lxc-start -n "$LXC_NAME" -P "$LXC_PATH"
|
||||
sleep 5
|
||||
# Seed DNS: the download-template rootfs ships no resolver, so apt/npm can't
|
||||
# resolve deb.debian.org / the npm registry. Matches secubox-peertube.
|
||||
la sh -c 'rm -f /etc/resolv.conf; printf "nameserver 1.1.1.1\nnameserver 9.9.9.9\n" > /etc/resolv.conf'
|
||||
la apt-get update
|
||||
la apt-get install -y --no-install-recommends nodejs npm ca-certificates python3 build-essential
|
||||
touch "$SENTINEL"
|
||||
fi
|
||||
log "node version: $(la node --version 2>/dev/null || echo MISSING)"
|
||||
|
||||
# --- app deploy (runs every postinst; idempotent) ---
|
||||
APP_SRC="${SECUBOX_APP_SRC:-/usr/lib/secubox/torrent/app}"
|
||||
la mkdir -p /opt/secubox-torrent/app /opt/secubox-torrent/www
|
||||
tar -C "$APP_SRC" -cf - . | la tar -C /opt/secubox-torrent/app -xf -
|
||||
tar -C /usr/share/secubox/www/torrent -cf - . | la tar -C /opt/secubox-torrent/www -xf -
|
||||
la bash -lc 'cd /opt/secubox-torrent/app && npm ci --omit=dev || npm install --omit=dev'
|
||||
# env file from host TOML values (exported by postinst into these vars)
|
||||
la bash -c "cat > /opt/secubox-torrent/torrent.env <<EOF
|
||||
TORRENT_DOWNLOAD_DIR=/data/torrent
|
||||
TORRENT_MAX_ACTIVE=${TORRENT_MAX_ACTIVE:-5}
|
||||
TORRENT_WEBRTC=${TORRENT_WEBRTC:-true}
|
||||
TORRENT_PORT=8090
|
||||
TORRENT_EPHEMERAL_TTL_HOURS=${TORRENT_TTL:-6}
|
||||
TORRENT_DISK_FLOOR_GB=${TORRENT_FLOOR:-5}
|
||||
EOF"
|
||||
# the unit ships alongside this script (Task 8 installs both under the same
|
||||
# /usr/lib/secubox/torrent/ dir so $(dirname "$0") resolves on the board)
|
||||
tar -C "$(dirname "$0")" -cf - secubox-torrent.service | la tar -C /etc/systemd/system -xf -
|
||||
la systemctl daemon-reload
|
||||
la systemctl enable --now secubox-torrent.service
|
||||
log "torrent LXC app deployed + started on $LXC_IP:8090"
|
||||
14
packages/secubox-torrent/lxc/secubox-torrent.service
Normal file
14
packages/secubox-torrent/lxc/secubox-torrent.service
Normal file
|
|
@ -0,0 +1,14 @@
|
|||
[Unit]
|
||||
Description=SecuBox Torrent (WebTorrent streaming)
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
WorkingDirectory=/opt/secubox-torrent/app
|
||||
EnvironmentFile=/opt/secubox-torrent/torrent.env
|
||||
ExecStart=/usr/bin/node server.js
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
|
|
@ -1,9 +1,9 @@
|
|||
{
|
||||
"name": "Torrent",
|
||||
"path": "/torrent/",
|
||||
"icon": "📥",
|
||||
"category": "mesh",
|
||||
"icon": "🌊",
|
||||
"category": "media",
|
||||
"order": 612,
|
||||
"description": "BitTorrent client management",
|
||||
"description": "WebTorrent streaming (torrent.gk2.secubox.in)",
|
||||
"id": "torrent"
|
||||
}
|
||||
|
|
|
|||
27
packages/secubox-torrent/nft/torrent-egress.nft
Normal file
27
packages/secubox-torrent/nft/torrent-egress.nft
Normal file
|
|
@ -0,0 +1,27 @@
|
|||
# SecuBox-Deb :: secubox-torrent :: host nft egress scope
|
||||
# CyberMind — https://cybermind.fr
|
||||
#
|
||||
# Scopes the torrent LXC's egress. FORWARD/OUTPUT are policy-accept
|
||||
# repo-wide (see CLAUDE.md nftables rules) — no blanket DROP is introduced
|
||||
# here. This is an explicit visibility tap (LOG on new connections) on the
|
||||
# LXC's veth, not an allow/deny gate.
|
||||
#
|
||||
# Own, fully self-contained table + base chain (mirrors
|
||||
# packages/secubox-p2p/nft/secubox-p2p-ephemeral.nft): loads independently
|
||||
# of any other table already present on the board, and never issues a
|
||||
# global ruleset reset.
|
||||
#
|
||||
# Matches the exact veth pair name pinned by install-lxc.sh
|
||||
# (`lxc.net.0.veth.pair = veth-torrent0`) — LXC otherwise assigns a random
|
||||
# vethXXXXXX name on every start, which a static nft rule can't target.
|
||||
#
|
||||
# Loaded by /etc/nftables.conf via `include "/etc/nftables.d/*.nft"`. Ship
|
||||
# this file as zz-torrent-egress.nft at install time (Task 8) so it sorts
|
||||
# after any table-creator base file, per repo convention (see
|
||||
# feedback_nft_layered_dropins_persistence).
|
||||
table inet secubox_torrent {
|
||||
chain forward_torrent {
|
||||
type filter hook forward priority 0; policy accept;
|
||||
iifname "veth-torrent0" ct state new log prefix "[SECUBOX-TORRENT] " level info
|
||||
}
|
||||
}
|
||||
|
|
@ -1,6 +1,98 @@
|
|||
# /etc/nginx/secubox.d/torrent.conf
|
||||
# Installed by secubox-torrent — auto-registered on install, removed on purge
|
||||
location /api/v1/torrent/ {
|
||||
proxy_pass http://unix:/run/secubox/torrent.sock:/;
|
||||
include /etc/nginx/snippets/secubox-proxy.conf;
|
||||
# /etc/nginx/sites-available/torrent.conf — SecuBox Torrent public vhost
|
||||
# Installed by secubox-torrent; operator enables with:
|
||||
# ln -s ../sites-available/torrent.conf /etc/nginx/sites-enabled/
|
||||
# systemctl reload nginx
|
||||
#
|
||||
# IMPORTANT for packaging (Task 8): this is a full, standalone server{}
|
||||
# block with its own server_name (torrent.gk2.secubox.in) — it must go to
|
||||
# /etc/nginx/sites-available/ (+ sites-enabled symlink), NOT
|
||||
# /etc/nginx/secubox.d/. secubox.d/*.conf files are location-only snippets
|
||||
# that get merged into the shared admin/hub server block via
|
||||
# `include /etc/nginx/secubox.d/*.conf;` (see common/nginx/secubox.conf) —
|
||||
# nginx does not allow a nested `server {}` inside another `server {}`, so
|
||||
# dropping this file into secubox.d/ would fail nginx -t. Verified against
|
||||
# the real peertube pattern: its full vhost is
|
||||
# packages/secubox-peertube/conf/peertube.nginx.conf, installed to
|
||||
# etc/nginx/sites-available/peertube.conf by debian/rules — the file
|
||||
# peertube ships under secubox.d/ is only the legacy /api/v1/peertube/
|
||||
# location snippet, not a full vhost.
|
||||
#
|
||||
# Public chain: Browser → HAProxy:443 (TLS) → sbxwaf (WAF) → nginx:9080
|
||||
# (this vhost) → torrent LXC at 10.100.0.160:8090.
|
||||
# Requires: sbxwaf route (haproxy-routes.json "torrent.gk2.secubox.in" →
|
||||
# [<host-ip>, 9080]) + HAProxy ACL (use_backend mitmproxy_inspector).
|
||||
#
|
||||
# Auth: the shared SecuBox LAN-only gate stub (`/__sbx_auth_verify` +
|
||||
# `@sbx_auth_login`; see packages/secubox-hub/nginx/zz-sbx-authgate.conf).
|
||||
# secubox-authelia (SSO) was decommissioned (v2.14.0) — the stub is now a
|
||||
# simple default-deny check against $lan_client (defined globally by
|
||||
# packages/secubox-hub/nginx/secubox-lan-geo.conf, available in every
|
||||
# server block since conf.d/*.conf loads at the http{} level). Because this
|
||||
# is a standalone server{} (own file, not merged into the hub's block), the
|
||||
# two named locations are re-declared here — same pattern as
|
||||
# packages/secubox-grafana/nginx/grafana-vhost.conf and
|
||||
# packages/secubox-spiderfoot/nginx/spiderfoot.gk2.conf.
|
||||
server {
|
||||
listen 0.0.0.0:9080;
|
||||
server_name torrent.gk2.secubox.in;
|
||||
|
||||
# ACME HTTP-01 challenge — outside the auth gate so renewals work.
|
||||
location ^~ /.well-known/acme-challenge/ {
|
||||
root /usr/share/secubox/www;
|
||||
try_files $uri =404;
|
||||
}
|
||||
|
||||
sub_filter_types text/html;
|
||||
sub_filter_once on;
|
||||
sub_filter "</body>" '<script src="/shared/health-banner.js"></script></body>';
|
||||
|
||||
location /shared/ {
|
||||
allow 127.0.0.1;
|
||||
allow 10.0.0.0/8;
|
||||
allow 172.16.0.0/12;
|
||||
allow 192.168.0.0/16;
|
||||
deny all;
|
||||
add_header Access-Control-Allow-Origin "*" always;
|
||||
alias /usr/share/secubox/www/shared/;
|
||||
}
|
||||
|
||||
location / {
|
||||
auth_request /__sbx_auth_verify;
|
||||
error_page 401 = @sbx_auth_login;
|
||||
|
||||
proxy_pass http://10.100.0.160:8090;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto https;
|
||||
proxy_set_header X-Forwarded-Host $host;
|
||||
|
||||
# Streaming: don't buffer video/torrent data either direction, and
|
||||
# keep long-lived connections (large seeds, live range requests).
|
||||
proxy_buffering off;
|
||||
proxy_request_buffering off;
|
||||
proxy_read_timeout 3600s;
|
||||
|
||||
# WebRTC signalling (webtorrent trackers) rides over the same
|
||||
# WebSocket upgrade path.
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection "upgrade";
|
||||
}
|
||||
|
||||
# Re-declared per-server because auth_request lives at server scope.
|
||||
# LAN-only default-deny — matches zz-sbx-authgate.conf's current
|
||||
# (post-Authelia) semantics, NOT a JWT/aggregator check (no such
|
||||
# endpoint exists yet — see task-7-report.md for the rationale).
|
||||
location = /__sbx_auth_verify {
|
||||
internal;
|
||||
if ($lan_client) { return 204; }
|
||||
return 403;
|
||||
}
|
||||
location @sbx_auth_login {
|
||||
return 403;
|
||||
}
|
||||
|
||||
access_log /var/log/nginx/torrent_access.log;
|
||||
error_log /var/log/nginx/torrent_error.log;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,28 +0,0 @@
|
|||
[Unit]
|
||||
# Only start if explicitly enabled (backend installed)
|
||||
ConditionPathExists=/etc/secubox/torrent/enabled
|
||||
Description=SecuBox Torrent — BitTorrent Client Management API
|
||||
After=network.target secubox-core.service
|
||||
Wants=secubox-core.service
|
||||
|
||||
[Service]
|
||||
UMask=0000
|
||||
Type=simple
|
||||
User=secubox
|
||||
Group=secubox
|
||||
WorkingDirectory=/usr/lib/secubox/torrent
|
||||
ExecStart=/usr/bin/python3 -m uvicorn api.main:app \
|
||||
--uds /run/secubox/torrent.sock \
|
||||
--log-level warning
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
PrivateTmp=true
|
||||
NoNewPrivileges=true
|
||||
RuntimeDirectory=secubox
|
||||
RuntimeDirectoryPreserve=yes
|
||||
RuntimeDirectoryMode=0775
|
||||
ProtectSystem=full
|
||||
ReadWritePaths=/run/secubox /var/lib/secubox /var/cache/secubox /etc/secubox /srv/torrent
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
52
packages/secubox-torrent/tests/test_packaging.py
Normal file
52
packages/secubox-torrent/tests/test_packaging.py
Normal file
|
|
@ -0,0 +1,52 @@
|
|||
# SPDX-License-Identifier: LicenseRef-CMSD-1.0
|
||||
# Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
|
||||
# Source-Disclosed License — All rights reserved except as expressly granted.
|
||||
# See LICENCE-CMSD-1.0.md for terms.
|
||||
|
||||
"""
|
||||
SecuBox-Deb :: secubox-torrent :: packaging structural tests (v2.0.0 pivot)
|
||||
|
||||
Guards the Transmission -> WebTorrent/LXC packaging pivot (#917, Task 8):
|
||||
old FastAPI/Transmission control-plane gone, LXC deps present, install-lxc.sh
|
||||
wired into postinst, and the public vhost lands in sites-available (not the
|
||||
location-snippet secubox.d/ dir).
|
||||
"""
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
|
||||
|
||||
def _read(p):
|
||||
return (ROOT / p).read_text()
|
||||
|
||||
|
||||
def test_control_is_v2_no_transmission():
|
||||
c = _read("debian/control")
|
||||
assert "python3-uvicorn" not in c # old FastAPI gone
|
||||
assert "transmission" not in c.lower()
|
||||
assert "lxc" in c.lower()
|
||||
|
||||
|
||||
def test_changelog_is_2_0_0():
|
||||
assert _read("debian/changelog").startswith("secubox-torrent (2.0.0-1~bookworm1)")
|
||||
|
||||
|
||||
def test_postinst_runs_install_lxc():
|
||||
assert "install-lxc.sh" in _read("debian/postinst")
|
||||
|
||||
|
||||
def test_no_old_fastapi_main():
|
||||
assert not (ROOT / "api" / "main.py").exists()
|
||||
|
||||
|
||||
def test_no_old_host_systemd_unit():
|
||||
assert not (ROOT / "systemd" / "secubox-torrent.service").exists()
|
||||
|
||||
|
||||
def test_vhost_installed_to_sites_available_not_secubox_d():
|
||||
rules = _read("debian/rules")
|
||||
assert "sites-available" in rules
|
||||
# Guard against reintroducing the location-snippet dir for this full vhost.
|
||||
assert "secubox-torrent/etc/nginx/secubox.d/torrent.conf" not in rules
|
||||
assert "secubox-torrent/etc/nginx/secubox-vhost.d" not in rules
|
||||
File diff suppressed because it is too large
Load Diff
Loading…
Reference in New Issue
Block a user