secubox-deb/common/secubox_core/config.py
CyberMind a19486c997
feat: cookie audit pipeline (RGPD / ePrivacy reconciler) (#159)
* feat(mitmproxy): cookie_audit addon — Set-Cookie ledger for RGPD (ref #156)

New mitmproxy addon that hooks the response flow and appends every Set-Cookie
header to /var/log/secubox/cookie-audit/server.jsonl. Cookie values are
sha256-hashed at the addon — the raw value never leaves the process.

Companion to the upcoming browser-side cookie-inventory.js + the
CookieAuditAggregator that will reconcile both streams for RGPD/ePrivacy
audit on operator-owned vhosts.

Also adds the implementation plan under docs/superpowers/plans/.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(hub): cookie-inventory.js browser snapshotter (ref #156)

Vanilla JS module that snapshots document.cookie at DOMContentLoaded, +2s,
and on visibilitychange. Names + sha256(value) are POSTed to
/api/v1/cookie-audit/ingest with credentials:'omit'. Hard-capped at 8
snapshots per page to avoid amplification.

Loaded into every HTML response via the WAF banner injection (next commit).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(waf): inject cookie-inventory.js alongside health banner (ref #156)

Extends the existing CDN banner injection block to also append a script tag
loading shared/cookie-inventory.js. Adds two new CDN-config keys:
  - cookie_inventory_url      (default: admin.gk2.secubox.in/shared/...)
  - cookie_audit_ingest_url   (default: admin.gk2.secubox.in/api/v1/...)

Both health banner and cookie inventory now load from the same injected
guard block, so a single </body> replacement covers both.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(metrics): CookieAuditAggregator + Classifier (ref #156)

Aggregator class mirroring CertStatusAggregator: tails the mitmproxy JSONL
Set-Cookie ledger and the browser-snapshot ingest dir, then reconciles per
(vhost, cookie-name).

Per-cookie source verdict:
  * http  — server only
  * js    — browser only -> RGPD violation unless strictly_necessary
  * both  — server + browser

Classifier loads regex rules from config, checks categories in order
(strictly_necessary > functional > analytics > marketing), first match wins.
Default cache file: /var/cache/secubox/metrics/cookie-audit.json.

9 unit tests covering parsing, reconciliation, persistence and malformed
input.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(metrics): wire /api/v1/cookie-audit/{ingest,report,summary} (ref #156)

Imports + initializes CookieAuditAggregator, schedules it in the FastAPI
lifespan alongside the other aggregators, opens up CORS to POST (was GET
only) for the ingest endpoint, and adds three routes:

  POST /api/v1/cookie-audit/ingest   — browser snapshot ingest (rate-bounded,
                                       credentials: omit, returns refused if
                                       audit disabled in config)
  GET  /api/v1/cookie-audit/report?host=…  — per-vhost or global report
  GET  /api/v1/cookie-audit/summary  — global rollup

Hard caps: 200 cookies/snapshot, 128B names, 128B hashes, 512B UA.

All 34 existing metrics tests + 9 new cookie-audit tests pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(core): get_cookie_audit_config + RGPD classifier baseline (ref #156)

New config helper get_cookie_audit_config() that returns the merged
[cookie_audit] section with built-in RGPD/CNIL classifier patterns.

Categories (eval order — first match wins):
  strictly_necessary  PHPSESSID, csrftoken, cart, remember_token, ...
  functional          lang, locale, theme, cookie_consent, euconsent, ...
  analytics           _ga*, _gid, _gat*, _pk_*, _hjid, _clck, _matomo*, ...
  marketing           _fbp, _fbc, __utm*, _gcl_*, _uet*, IDE, MUID, NID

Operator patterns from [cookie_audit.classifier] are MERGED on top of the
baseline by default. Set classifier_override = true to replace it entirely
(opt-out for sites that need a different taxonomy).

secubox.conf.example documents the section with commented-out examples.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: cookie-audit API + addon (ref #156)

- secubox-metrics README: new "Cookie Audit" section documenting the
  three endpoints, the source verdict semantics (http/js/both), and the
  rgpd_violation rule.
- secubox-mitmproxy README: new "Addons" section listing both
  secubox_waf and cookie_audit + the dual -s mitmdump invocation.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(claude): record cookie-audit pipeline in HISTORY+WIP (ref #156)

HISTORY 2026-05-16 entry captures scope (server ledger + browser snapshot
reconciler), the 8 implementation tasks landed in the worktree, and the
two follow-ups left (AppArmor + logrotate).

WIP gets a top entry pointing at the worktree/branch with the test
totals and "awaiting validation, no PR opened" status.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(banner): add Cookie Audit live tile + summary URL injection (ref #156)

Adds a "CookieAudit" section to the health-banner live panel alongside
VisitorOrigin / LiveHosts / CertStatus. It polls /api/v1/cookie-audit/summary
every 30s and renders:

  - host count
  - RGPD verdict row (red ⚠ when violation_count > 0, green ✓ otherwise)
  - per-category breakdown: strict / func / analytics / marketing
  - unclassified row only shown when > 0

The WAF banner-injection block now also sets
window.SECUBOX_COOKIE_AUDIT_SUMMARY (new cookie_audit_summary_url config
key) so the banner can hit the cross-origin summary endpoint when the
host page is on a non-admin vhost.

Banner bumped to v1.4.0. New section hides itself silently when the
aggregator is disabled or empty — zero visual impact on existing
deployments.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: CyberMind-FR <gandalf@Gk2.net>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-16 13:53:40 +02:00

240 lines
6.9 KiB
Python

"""
secubox_core.config — Chargeur de configuration TOML
=====================================================
Lit /etc/secubox/secubox.conf (TOML).
Fallback : ./secubox.conf.example (dev).
"""
from __future__ import annotations
import os
import subprocess
from functools import lru_cache
from pathlib import Path
try:
import tomllib # stdlib Python 3.11+
except ImportError:
import tomli as tomllib # pip install tomli pour Python 3.10
# Chemins de recherche (ordre de priorité)
_CONF_PATHS = [
Path("/etc/secubox/secubox.conf"),
Path(__file__).parents[4] / "secubox.conf.example",
]
_CONFIG: dict | None = None
def _load() -> dict:
global _CONFIG
if _CONFIG is not None:
return _CONFIG
for p in _CONF_PATHS:
if p.exists():
with open(p, "rb") as f:
_CONFIG = tomllib.load(f)
return _CONFIG
# Config minimale par défaut (dev sans fichier)
_CONFIG = {
"global": {"hostname": "secubox", "timezone": "Europe/Paris", "board": "unknown"},
"api": {"socket_dir": "/tmp/secubox", "jwt_secret": os.environ.get("SECUBOX_JWT_SECRET", "dev-secret")},
"auth": {"users": {"admin": {"password": "secubox"}}},
"crowdsec": {"lapi_url": "http://127.0.0.1:8080", "lapi_key": ""},
"dpi": {"mode": "inline", "engine": "netifyd", "interface": "eth0", "mirror_if": "ifb0"},
"wireguard": {"interface": "wg0", "listen_port": 51820},
}
return _CONFIG
def get_config(section: str = "") -> dict:
"""
Retourne la section demandée, ou la config complète si section="".
Exemple :
cfg = get_config("crowdsec")
url = cfg["lapi_url"]
"""
cfg = _load()
if not section:
return cfg
return cfg.get(section, {})
def reload_config() -> None:
"""Force le rechargement du fichier de config (post-modification)."""
global _CONFIG
_CONFIG = None
_load()
# ── Informations board ─────────────────────────────────────────────
def get_board_info() -> dict:
"""
Retourne les infos hardware du board courant.
Lit /proc/device-tree/model (DTS node name sur ARM).
"""
model = "unknown"
model_path = Path("/proc/device-tree/model")
if model_path.exists():
model = model_path.read_text(errors="replace").strip().rstrip("\x00")
# Uptime
try:
uptime_sec = float(Path("/proc/uptime").read_text().split()[0])
except Exception:
uptime_sec = 0.0
# CPU / RAM
cpu_count = os.cpu_count() or 1
mem = {}
try:
for line in Path("/proc/meminfo").read_text().splitlines():
k, v = line.split(":")
mem[k.strip()] = int(v.strip().split()[0]) # kB
except Exception:
pass
return {
"model": model,
"board": get_config("global").get("board", "unknown"),
"hostname": get_config("global").get("hostname", "secubox"),
"uptime_sec": int(uptime_sec),
"cpu_count": cpu_count,
"mem_total_mb": mem.get("MemTotal", 0) // 1024,
"mem_free_mb": mem.get("MemAvailable", 0) // 1024,
}
# ── Live-panel helpers (issue #92) ─────────────────────────────────
_VISITOR_ORIGIN_DEFAULTS = {
"enabled": False,
"window_minutes": 60,
"min_count": 5,
"top_n": 5,
"asn_db_path": "/var/lib/GeoIP/GeoLite2-ASN.mmdb",
"nft_table": "secubox_metrics",
"nft_set": "seen_src",
"nft_family": "inet",
}
_LIVE_HOSTS_DEFAULTS = {
"enabled": False,
"window_minutes": 60,
"top_n": 5,
"haproxy_socket": "/run/haproxy/admin.sock",
"frontend_filter": "*",
}
_CERT_STATUS_DEFAULTS = {
"enabled": False,
"letsencrypt_live_dir": "/etc/letsencrypt/live",
"warn_days": 30,
"critical_days": 7,
}
# Cookie audit (RGPD / ePrivacy, issue #156). See packages/secubox-metrics
# for the aggregator and packages/secubox-mitmproxy/addons/cookie_audit.py
# for the server-side ledger producer.
_COOKIE_AUDIT_DEFAULTS = {
"enabled": False,
"ledger_path": "/var/log/secubox/cookie-audit/server.jsonl",
"ingest_dir": "/var/lib/secubox/cookie-audit/ingest",
"max_ingest_age_hours": 24,
}
# Built-in classifier patterns. Override with [cookie_audit.classifier] in
# /etc/secubox/secubox.conf. Categories evaluated in order
# (strictly_necessary > functional > analytics > marketing), first match wins.
_COOKIE_CLASSIFIER_DEFAULTS = {
"strictly_necessary": [
r"^PHPSESSID$",
r"^sess(ion)?id$",
r"^csrftoken$",
r"^XSRF-TOKEN$",
r"^_csrf$",
r"^cart$",
r"^remember_token$",
r"^secubox_session$",
],
"functional": [
r"^lang$",
r"^locale$",
r"^theme$",
r"^cookie[_-]?consent$",
r"^euconsent",
],
"analytics": [
r"^_ga",
r"^_gid$",
r"^_gat",
r"^_pk_",
r"^_hjid$",
r"^_hjSession",
r"^_clck$",
r"^_clsk$",
r"^_matomo",
],
"marketing": [
r"^_fbp$",
r"^_fbc$",
r"^__utm",
r"^_gcl_",
r"^_uet",
r"^IDE$",
r"^MUID$",
r"^NID$",
],
}
def _merged(defaults: dict, section: str) -> dict:
out = dict(defaults)
out.update(get_config(section))
return out
def get_visitor_origin_config() -> dict:
"""Return [visitor_origin] merged with defaults."""
return _merged(_VISITOR_ORIGIN_DEFAULTS, "visitor_origin")
def get_live_hosts_config() -> dict:
"""Return [live_hosts] merged with defaults."""
return _merged(_LIVE_HOSTS_DEFAULTS, "live_hosts")
def get_cert_status_config() -> dict:
"""Return [cert_status] merged with defaults."""
return _merged(_CERT_STATUS_DEFAULTS, "cert_status")
def get_cookie_audit_config() -> dict:
"""Return [cookie_audit] merged with defaults, including classifier rules.
Each classifier category is the union of operator-provided patterns and
the built-in defaults — admins can extend but never silently drop the
RGPD baseline. Set ``classifier_override = true`` in
``[cookie_audit]`` to disable the merge and use operator patterns only.
"""
cfg = _merged(_COOKIE_AUDIT_DEFAULTS, "cookie_audit")
operator_cls = (get_config("cookie_audit") or {}).get("classifier") or {}
override = bool(cfg.get("classifier_override", False))
merged_cls: dict = {}
for cat, base in _COOKIE_CLASSIFIER_DEFAULTS.items():
op = operator_cls.get(cat, []) or []
if override:
merged_cls[cat] = list(op)
else:
seen = set()
out = []
for pat in list(op) + list(base):
if pat not in seen:
out.append(pat)
seen.add(pat)
merged_cls[cat] = out
cfg["classifier"] = merged_cls
return cfg