secubox/secubox-deb
1
0
Fork 0
mirror of https://github.com/CyberMind-FR/secubox-deb.git synced 2026-07-28 21:17:36 +00:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
5e4c0d2dac
secubox-deb/.claude/MIGRATION-MAP.md
CyberMind 2744758b9e
Health banner: live panel (visitor-origin + live-hosts + cert-status) (#98) 
* docs(spec): Health banner live panel design (ref #92)

Three public banner sections sharing one polling/CORS pipeline:
- VisitorOrigin: nft set seen_src + GeoLite2-ASN.mmdb, threshold-gated
  rollup, raw IPs discarded before persistence
- LiveHosts: HAProxy admin socket, 60 x 1-min ring buffer over req_tot
  deltas, hostname-heuristic frontend filter
- CertStatus: scan /etc/letsencrypt/live + cryptography parse, classify
  valid / expiring_soon / expiring_critical / expired

Each section fails independently; section hidden on enabled=false,
empty entries, or fetch error. All three endpoints are unauthenticated,
CORS-open, Cache-Control max-age=300.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(plan): Health banner live panel implementation plan (ref #92)

14-task TDD plan: tests scaffold -> config helpers -> three aggregators
(visitor-origin / live-hosts / cert-status) -> FastAPI lifespan wiring ->
nftables ruleset -> geoipupdate timer -> debian packaging -> banner v1.3.0
-> README + tracking docs -> full-suite verification + PR.

Also reconciles spec with codebase conventions: service user is 'secubox'
(not 'secubox-metrics'); config lives in /etc/secubox/secubox.conf, not a
separate metrics.toml.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(metrics): Scaffold pytest layout for new aggregators (ref #92)

Adds tests/__init__.py and conftest.py that wire packages/secubox-metrics/api
and the repo-wide common/ onto sys.path so individual aggregator modules can
be imported in isolation.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(core): Add visitor_origin / live_hosts / cert_status config helpers (ref #92)

Three new section helpers in secubox_core.config that merge defaults with
operator-supplied TOML overrides. Each section defaults to enabled=false so
the live-panel aggregators stay quiet on systems that haven't opted in.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(metrics): Use tmp_path fixture + drop unused import (ref #92)

Switches the config-helper tests from a hard-coded /tmp path to pytest's
tmp_path fixture, matching the pattern in packages/secubox-haproxy/tests/.
Removes the now-unused 'from unittest.mock import patch' line.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(metrics): VisitorOrigin aggregator (ref #92)

Pure-Python aggregator that polls the nft seen_src set, resolves ASNs via
GeoLite2 mmdb, and emits a threshold-gated top-N rollup. Private/loopback IPs
are skipped at lookup time; raw IPs never leave the function scope; the
threshold gate runs before persistence so the cache file never contains
attributable counts.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(metrics): VisitorOrigin code-review followups (ref #92)

- mmdb auto-reopen on mtime change (Important): close + reopen when stat
  reports a different mtime; previously _mmdb_mtime was dead state.
- _read_nft_set defensive parse (Important): guard against {elem: str} shapes
  to prevent a latent TypeError.
- current() returns a defensive copy (Minor): no more by-reference leak of
  internal state.
- Drop unused ip_address import and unused monkeypatch parameter (Minor).
- Tighten tiebreak test to assert ASN order, not just count order (Minor).
- Add test for mtime-based reopen.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(metrics): Pin VisitorOrigin error-path behaviour (ref #92)

Regression tests for refresh_once: disabled config, missing mmdb, and nft
subprocess failure must all yield a non-throwing degraded payload.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(metrics): LiveHosts aggregator with 60x1-min ring buffer (ref #92)

Reads HAProxy admin socket via raw AF_UNIX, parses 'show stat' CSV, filters
internal frontends (leading underscore or no dot), ring-buffers per-frontend
deltas over 60 minutes, and emits a top-N hostname rollup. Counter-reset
detection (cur < prev) yields a fresh-baseline bucket instead of a negative
delta. current() returns a defensive copy mirroring the VisitorOrigin fix.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(metrics): Pin LiveHosts CSV parser + missing-socket paths (ref #92)

Tests the show-stat CSV parser against the real HAProxy column order and
asserts that an absent admin socket returns a degraded payload rather than
raising.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(metrics): CertStatus aggregator (ref #92)

Scans /etc/letsencrypt/live for cert.pem files, classifies each by days
remaining (valid / expiring_soon / expiring_critical / expired) using the
operator's warn_days/critical_days thresholds, and emits a summary + soonest
next-renewal host. A single corrupt PEM never kills the scan. Days remaining
computed with math.ceil so a cert expiring in 2.99d reports 3d, consistent
with certbot/renewal tooling expectations.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(metrics): correct expired boundary when using math.ceil (ref #92)

math.ceil maps any cert that expired within the last 24 h to days=0,
which the previous `days < 0` guard treated as expiring_critical instead
of expired. Changing the guard to `days <= 0` closes the gap: with ceil,
days=0 means actual remaining time is in (-86400, 0] — i.e. already
past or exactly at expiry — so classifying it as expired is correct.
All four existing tests continue to pass.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(metrics): Wire three live-panel aggregators into FastAPI lifespan (ref #92)

Adds the three asyncio background tasks under a single lifespan and exposes
their current() payloads on /api/v1/metrics/{visitor-origin,live-hosts,cert-status}
with a 5-min Cache-Control. Endpoints stay unauthenticated by design — the
aggregators only emit threshold-gated, hostname-only data.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(metrics): Await cancelled lifespan tasks + tidy import order (ref #92)

- Important: lifespan finally now awaits gather(*tasks, return_exceptions=True)
  after cancel(), so blocking subprocess/socket I/O in aggregator refreshes
  doesn't race uvicorn's shutdown timeout.
- Minor: move 'from contextlib import asynccontextmanager' into the stdlib
  import group at the top of the file.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(metrics): Ship nftables ingress tap for visitor-origin (ref #92)

Private inet secubox_metrics table with a timeout'd src-IP set, hooked from
prerouting at priority -300 so additions happen before secubox-firewall's
filter chain decides whether to drop the packet.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(metrics): Weekly GeoLite2 ASN refresh timer (ref #92)

Conditional on /etc/secubox/secrets/maxmind.conf existing, so the unit is a
silent no-op on installs that haven't supplied a license key. RandomizedDelay
spreads load when many boxes deploy together.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* build(metrics): Package live-panel deps, nft ruleset, and geoipupdate timer (ref #92)

- control: add python3-maxminddb, python3-cryptography, geoipupdate, nftables
- rules: install nftables/ and systemd/ assets
- postinst: secubox -> haproxy group, cache + secrets + GeoIP dirs,
            nftables reload, timer enable
- service: ReadWritePaths gains /var/cache/secubox

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(metrics): Run geoipupdate as secubox + restart svc on upgrade (ref #92)

- secubox-geoipupdate.service: User/Group=secubox so .mmdb files inherit the
  ownership the metrics service expects when reading them. Previously the
  unit ran as root and created root-owned files that secubox-metrics could
  not open.
- postinst: switch 'systemctl start' to 'systemctl restart' so the secubox
  user's new haproxy-group membership is picked up by an already-running
  service after an upgrade.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(banner): v1.3.0 live panel — visitor origin, live hosts, cert status (ref #92)

Three independent fetch loops on a shared 30s cadence, three DOM sections,
per-section hide on enabled=false / empty / fetch error. Uses existing
design tokens (gold/cyan/matrix-green) so no new CSS variables are added.
A failing section never affects the others.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(banner): Correct banner element id lookup in live-panel (ref #92)

The new live-panel sectionContainer() helper looked up
getElementById('sbx-health-banner'), but the actual banner element is
created with id='health-banner'. The mismatch made banner null, so the
three live-panel sections were silently never appended to the DOM.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: Session 160 — Health Banner Live Panel (ref #92)

README documents the three new endpoints + config blocks. HISTORY / WIP /
MIGRATION-MAP entries describe the feature, the spec/plan paths, and the
session's outcome.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(metrics): Address whole-branch review findings (ref #92)

- Wrap _read_nft_set / _read_haproxy_stats in asyncio.to_thread so blocking
  I/O (subprocess up to 5s, AF_UNIX recv up to 2s) no longer stalls the
  event loop on every refresh tick.
- Replace falsy current() guard with explicit _refreshed flag. Previously,
  a successful refresh that produced entries=[] would fall through to the
  on-disk cache, serving stale non-empty data during low-traffic periods.
- Move geoipupdate from Depends to Recommends. It lives in bookworm/contrib,
  so a hard dependency breaks 'apt install secubox-metrics' on systems
  without contrib enabled. The aggregator already degrades gracefully when
  the mmdb is absent, making Recommends the correct strength. README
  documents the contrib note.
- prerm stops + disables secubox-geoipupdate.timer/service so 'apt remove'
  doesn't leave an orphan timer firing weekly with a missing unit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 16:27:56 +02:00

12 KiB
Raw Blame History

MIGRATION MAP — SecuBox OpenWrt → Debian

Mis à jour : 2026-04-05

Légende : ✅ Terminé · 🔄 En cours · ⬜ À faire · ⏸ Bloqué

Voir aussi: OPENWRT-DEBIAN-COMPARISON.md pour la comparaison complète des 103 modules OpenWRT vs 52 paquets Debian.

Infrastructure

Composant Statut Notes
Repo structure ✅ Structure créée
CLAUDE.md ✅ Instructions Claude Code
secubox_core lib ✅ auth, config, logger, system
nginx template ✅ Reverse proxy complet
rewrite-xhr.py ✅ Gère accolades imbriquées
CI build-image ✅ Scaffold créé
CI build-packages ✅ Scaffold créé
build-image.sh ✅ arm64 + amd64 (VirtualBox)
create-vbox-vm.sh ✅ Création VM automatique
firstboot.sh ✅ JWT + SSH + hostname + nftables
APT repo ✅ apt.secubox.in (reprepro + GPG + CI)
Local cache ✅ apt-cacher-ng + repo local

Boards supportés

Board SoC Arch Profil Statut
mochabin Armada 7040 arm64 secubox-full ✅
espressobin-v7 Armada 3720 arm64 secubox-lite ✅
espressobin-ultra Armada 3720 arm64 secubox-lite ✅
vm-x64 x86_64-generic amd64 secubox-full ✅

Paquets Debian — 124 modules (80 UI + 3 backend + 30 tools + 11 infrastructure)

Module www/ API deb/ Endpoints Statut
secubox-core — ✅ ✅ kiosk.py (board detect, kiosk mgmt) v1.1.0 ✅
secubox-hub ✅ (71) ✅ ✅ 50+ endpoints (net mode select) v1.1.0 ✅
secubox-portal ✅ ✅ ✅ login, auth, theme, branding v2.1.0 ✅
secubox-crowdsec ✅ (54) ✅ ✅ 54 endpoints ✅
secubox-netdata ✅ (16) ✅ ✅ 16 endpoints ✅
secubox-wireguard ✅ (28) ✅ ✅ 28+ endpoints ✅
secubox-vhost ✅ ✅ ✅ vhosts, ssl, certs ✅
secubox-mediaflow ✅ (20) ✅ ✅ streams, alerts... ✅
secubox-dpi ✅ ✅ ✅ 40+ endpoints netifyd ✅
secubox-qos ✅ (80) ✅ ✅ 80+ endpoints HTB + VLAN v1.1.0 ✅
secubox-auth ✅ (11) ✅ ✅ 20+ endpoints ✅
secubox-cdn ✅ (36) ✅ ✅ 25+ endpoints ✅
secubox-system ✅ (42) ✅ ✅ 45+ endpoints (board detect, kiosk) v1.2.0 ✅
secubox-netmodes ✅ (34) ✅ ✅ 25+ endpoints + templates ✅
secubox-nac ✅ (32) ✅ ✅ 25+ endpoints ✅
secubox-haproxy ✅ ✅ ✅ stats, backends, acls ✅
secubox-droplet ✅ ✅ ✅ upload, publish ✅
secubox-streamlit ✅ ✅ ✅ apps, deploy ✅
secubox-streamforge ✅ ✅ ✅ apps, templates ✅
secubox-metablogizer ✅ ✅ ✅ sites, tor, publish ✅
secubox-dns ✅ ✅ ✅ zones, records, BIND ✅
secubox-mail ✅ ✅ ✅ Postfix/Dovecot + webmail ✅
secubox-users ✅ ✅ ✅ unified identity v1.1.0 ✅
secubox-webmail ✅ ✅ ✅ Roundcube/SOGo ✅
secubox-mail-lxc — ✅ ✅ LXC backend (no UI) ✅
secubox-webmail-lxc — ✅ ✅ LXC backend (no UI) ✅
secubox-publish ✅ ✅ ✅ Unified publishing ✅
secubox-waf ✅ ✅ ✅ 300+ rules, CrowdSec ✅
secubox-gitea ✅ ✅ ✅ Git server LXC ✅
secubox-nextcloud ✅ ✅ ✅ File sync LXC ✅
secubox-c3box ✅ ✅ ✅ Services portal ✅
secubox-backup ✅ ✅ ✅ config, container backup ✅
secubox-watchdog ✅ ✅ ✅ containers, services, endpoints ✅
secubox-tor ✅ ✅ ✅ circuits, hidden services ✅
secubox-exposure ✅ ✅ ✅ Tor, SSL, DNS, Mesh ✅
secubox-mitmproxy ✅ ✅ ✅ WAF, alerts, bans ✅
secubox-traffic ✅ ✅ ✅ TC/CAKE QoS ✅
secubox-device-intel ✅ ✅ ✅ asset discovery, fingerprinting ✅
secubox-vortex-dns ✅ ✅ ✅ DNS firewall, RPZ, threat feeds ✅
secubox-vortex-firewall ✅ ✅ ✅ nftables threat enforcement ✅
secubox-meshname ✅ ✅ ✅ mesh DNS, mDNS, Avahi ✅
secubox-soc ✅ ✅ ✅ SOC dashboard, clock, map, tickets ✅
secubox-roadmap ✅ ✅ ✅ migration roadmap tracker ✅
secubox-metrics ✅ ✅ ✅ real-time metrics dashboard ✅
secubox-mesh ✅ ✅ ✅ Yggdrasil mesh network ✅
secubox-p2p ✅ ✅ ✅ P2P networking ✅
secubox-zkp ✅ ✅ ✅ ZKP Hamiltonian proofs ✅
secubox-hardening ✅ ✅ ✅ sysctl + module blacklist ✅
secubox-repo ✅ ✅ ✅ APT repository management ✅
secubox-daemon — Go ✅ Mesh daemon (secuboxd, secuboxctl) ✅
secubox-c3box ✅ Go ✅ C3BOX situational awareness dashboard ✅

| secubox-ollama | ✅ | ✅ | ✅ | models, chat, generate, system | ✅ | | secubox-jellyfin | ✅ | ✅ | ✅ | media, config, backup, logs | ✅ | | secubox-lyrion | ✅ | ✅ | ✅ | players, library, backup, LMS JSON-RPC | ✅ | | secubox-console | — | ✅ | ✅ | Textual TUI dashboard (no www) v1.1.0 | ✅ | | secubox-soc-agent | — | ✅ | ✅ | Edge node metrics agent v1.0.0 | ✅ | | secubox-soc-gateway | — | ✅ | ✅ | SOC aggregation gateway v1.0.0 | ✅ | | secubox-soc-web | ✅ | — | ✅ | React SOC dashboard v1.0.0 | ✅ |

| secubox-hexo | ✅ | ✅ | ✅ | blogs, posts, themes, deploy | ✅ | | secubox-webradio | ✅ | ✅ | ✅ | stations, streaming, recording | ✅ | | secubox-torrent | ✅ | ✅ | ✅ | torrents, RSS, categories | ✅ | | secubox-newsbin | ✅ | ✅ | ✅ | NZB queue, history, servers | ✅ | | secubox-domoticz | ✅ | ✅ | ✅ | devices, rooms, scenes, automation | ✅ | | secubox-gotosocial | ✅ | ✅ | ✅ | accounts, federation, moderation | ✅ | | secubox-simplex | ✅ | ✅ | ✅ | SMP relay, queues, TLS | ✅ | | secubox-photoprism | ✅ | ✅ | ✅ | library, albums, faces, storage | ✅ | | secubox-homeassistant | ✅ | ✅ | ✅ | entities, automations, scenes, addons | ✅ | | secubox-matrix | ✅ | ✅ | ✅ | users, rooms, federation, media | ✅ | | secubox-jitsi | ✅ | ✅ | ✅ | rooms, recordings, auth, prosody | ✅ | | secubox-peertube | ✅ | ✅ | ✅ | videos, channels, federation, transcoding | ✅ | | secubox-voip | ✅ | ✅ | ✅ | extensions, trunks, routes, IVR, CDR | ✅ |

| secubox-wazuh | — | ✅ | ✅ | SIEM, agent enrollment | ✅ | | secubox-ossec | — | ✅ | ✅ | Host IDS | ✅ | | secubox-ai-insights | ✅ | ✅ | ✅ | ML threat detection, anomalies | ✅ | | secubox-ipblock | ✅ | ✅ | ✅ | IP blocklist, nftables sets | ✅ | | secubox-interceptor | ✅ | ✅ | ✅ | traffic interception, SSL inspect | ✅ | | secubox-cookies | ✅ | ✅ | ✅ | cookie tracking, GDPR | ✅ | | secubox-mac-guard | ✅ | ✅ | ✅ | MAC whitelist/blacklist | ✅ | | secubox-dns-provider | ✅ | ✅ | ✅ | OVH, Gandi, Cloudflare API | ✅ | | secubox-threats | ✅ | ✅ | ✅ | unified threat dashboard, IOCs | ✅ | | secubox-openclaw | ✅ | ✅ | ✅ | OSINT reconnaissance | ✅ | | secubox-modem | ✅ | ✅ | ✅ | LTE/5G modem, SMS, AT terminal | ✅ |

Phase 9+ — System & Infrastructure Tools (11 new)

Module www/ API deb/ Endpoints Statut
secubox-nettweak ✅ ✅ ✅ sysctl, profiles, TCP/IP tuning ✅
secubox-ksm ✅ ✅ ✅ KSM memory optimization ✅
secubox-avatar ✅ ✅ ✅ identity, avatar upload, service sync ✅
secubox-admin ✅ ✅ ✅ services, logs, disk, processes, reboot ✅
secubox-metabolizer ✅ ✅ ✅ log processing, pattern detection ✅
secubox-metacatalog ✅ ✅ ✅ service registry, health, dependencies ✅
secubox-cyberfeed ✅ ✅ ✅ threat feeds, nftables/hosts export ✅
secubox-mirror ✅ ✅ ✅ APT/NPM/PyPI/Docker cache ✅
secubox-saas-relay ✅ ✅ ✅ API proxy, Fernet, rate limiting ✅
secubox-rezapp ✅ ✅ ✅ Docker/LXC deployment, templates ✅
secubox-picobrew ✅ ✅ ✅ homebrew sensors, fermentation profiles ✅

Total : 125 modules | ~2000+ endpoints API + Go mesh daemon + TUI console + SOC

Note: mail-lxc and webmail-lxc are backend components integrated into secubox-mail

Addenda

2026-05-12 — Issue #92 extension: secubox-metrics gains three public live-panel endpoints (visitor-origin, live-hosts, cert-status) consumed by the health banner.

Phases du projet

Phase 1 — Hardware ✅

  • build-image.sh (debootstrap arm64 + amd64)
  • Board configs (mochabin, espressobin-v7, espressobin-ultra, vm-x64)
  • create-vbox-vm.sh (VirtualBox)
  • firstboot.sh (détection board améliorée)
  • Templates netplan par board
  • Kernel 6.6 LTS cross-compile (optionnel — peut utiliser stock Debian)

Phase 2 — Infrastructure ✅

  • secubox_core Python lib
  • nginx reverse proxy template
  • rewrite-xhr.py script
  • CI scaffolds

Phase 3 — Modules ✅

  • Tous les frontends portés (13/13)
  • Tous les APIs implémentés (14/14)
  • Packaging debian complet (14/14)
  • Templates netplan pour netmodes
  • Frontend secubox-dpi créé

Phase 4 — APT Repo ✅

  • apt.secubox.in (reprepro config)
  • GPG signing (generate-gpg-key.sh)
  • CI publish workflow (publish-packages.yml)
  • repo-manage.sh (add/remove/list/sync)
  • setup-repo-server.sh (nginx + Let's Encrypt)
  • Metapackages (secubox-full, secubox-lite)
  • Local cache build (apt-cacher-ng + repo local)

Commandes de build

# Build image ARM (MOCHAbin)
sudo bash image/build-image.sh --board mochabin

# Build image x64 pour VirtualBox
sudo bash image/build-image.sh --board vm-x64 --vdi

# Build image x64 avec cache local (plus rapide)
sudo bash image/build-image.sh --board vm-x64 --local-cache --vdi

# Créer VM VirtualBox
bash image/create-vbox-vm.sh output/secubox-vm-x64-bookworm.vdi

# Build un paquet .deb
cd packages/secubox-crowdsec && dpkg-buildpackage -us -uc -b

# Build et ajouter au repo local
bash scripts/build-add-local.sh secubox-crowdsec bookworm

Prochaines étapes

  1. Phase 1 : build-image.sh + board configs ✅ Fait
  2. Phase 2 : Infrastructure ✅ Fait
  3. Phase 3 : Core Modules (52/103) ✅ Fait
  4. Phase 4 : APT repo (apt.secubox.in) ✅ Fait
  5. Phase 5 : CSPN Hardening ✅ Fait (partiel)
  6. Phase 6 : CI/CD ✅ Fait
  7. Phase 7 : Documentation + UI Theme ✅ Fait
  8. Phase 8 : Applications ✅ 21/21 modules complete
  9. Phase 9 : System Tools ✅ 22/22 modules complete
  10. Phase 10 : Security Extensions ✅ 10/10 modules complete
  11. Tests d'intégration sur VM et hardware réel
  12. Déployer apt.secubox.in sur serveur production

Voir aussi: REMAINING-PACKAGES.md pour l'inventaire détaillé des 53 paquets restants avec classification par complexité

Build avec cache local

# Setup cache local (une fois)
sudo bash scripts/setup-local-cache.sh

# Builder tous les packages SecuBox
bash scripts/build-all-local.sh bookworm amd64

# Construire image avec cache local
sudo bash image/build-image.sh --board vm-x64 --local-cache