secubox-deb/common/secubox_core/whitelist.py
CyberMind-FR 27eacbcd24 feat(secubox-core+toolbox): Phase 3 transparency layer (whitelist + analysis-status + quality scoring) (ref #492)
## Goal

Make the cabine HONEST : tell the user not just what we inspected, but what
we deliberately BYPASSED and WHY (cert-pinning vendor, banking policy, E2E
opaque by design). Score each destination on passive observable signals.

This is the foundation for #493 (later) full reverse-WAF enforcement —
architecture intentionally aligned so action='block'/'redirect'/'strip' can
be added without refactoring.

## New shared code (common/secubox_core/)

  - whitelist.py : YAML loader with hot-reload. fnmatch glob patterns.
    Reads /usr/share/secubox/toolbox/whitelist-baseline.yaml +
    /etc/secubox/toolbox/whitelist.yaml (operator override). Returns
    (match dict | None) for any host.

  - classifiers/security_quality.py : grade A+/A/B/C/D/F per destination.
    Two modes :
      passive : TLS version + JA4 + SNI + ALPN + is_e2e_messaging
      active  : adds HSTS + CSP + X-Content-Type-Options + cookies attrs
                + mixed-content detection
    Returns {grade, score, mode, reasons[]} with each reason's weight.

## Curated whitelist baseline (47 patterns)

  - Apple ecosystem (push, iCloud, CDN, mzstatic, cloudkit, smoot)
  - Google/Android (googleapis, gstatic, fcm, android, play)
  - Messaging E2E (Signal, Threema, SimpleX, Matrix.org, Proton, Tutanota)
  - Banking FR (BanquePostale, SocGen, BNP, CA, CM, LCL, Boursorama, Revolut)
  - Government FR (service-public, impots.gouv, ameli, france-identite, francetravail, caf)
  - Health (Doctolib, Qare, MonEspaceSante)
  - Privacy tools (DDG, Qwant, Startpage, Proton, Tutanota, Mullvad, Tailscale)
  - Tor Project
  - OS updates (Windows Update, Microsoft, Debian, Ubuntu)

## local_store.py wiring

  - New _analysis_status(host, sni, decrypted) returns (status, reason).
    Order : decrypted > whitelist > E2E regex > pinned-failed fallback.
  - DPI events now carry analysis_status + analysis_reason fields.
  - New tls_failed_clienthello hook captures the SNI of bypassed/pinned
    flows so the report can be honest about them.

## Aggregator (secubox_toolbox/api.py)

  - New _build_transparency() builds the dict :
      breakdown        : raw counts {inspected: N, bypassed-whitelist: N, ...}
      breakdown_pct    : same as percentages
      total_events     : sum
      per_host[]       : worst-first grade table for the report UI
      whitelist_stats  : how many patterns are loaded

  - Exposed under 'transparency' key in /report and /report/me/html.
  - Backfill : legacy events (no analysis_status) get post-hoc tagged via
    whitelist match check so older sessions also show meaningful breakdowns.

## Reports

  - HTML live (report-live.html.j2) : new card 'INSPECTION : CE QUI A ETE
    REGARDE' with breakdown bar + 'QUALITE SECURITE PAR DESTINATION'
    sortable table. Inserts before the Support card.
  - PDF (reports.py) : matching sections rendered ASCII-safe via _ascii_safe.
    Top 10 worst-quality hosts.

## Verified on gk2 (2026-06-05)

  - whitelist loaded : 47 patterns, baseline OK
  - aggregator returns transparency.has_transparency=True
  - per_host populated with quality grades
  - PDF still generates clean

## Out of scope (deferred to #493 enforcement phase)

  - Action types : block/redirect/strip-cookies/downgrade
  - nftables sinkhole for known-bad destinations
  - dnsmasq hijack for whitelisted FQDN auto-direction
  - PhishTank + threat-intel feeds active blocking
  - Per-flow TLS version + headers capture (currently we only grade default
    'inspected' as C because no TLS metadata in the event yet — Phase 4 will
    extend local_store.response() to capture response headers + TLS info).

## Closes #492 (Phase 3 transparency foundation)
2026-06-05 08:13:45 +02:00

98 lines
2.9 KiB
Python

# SPDX-License-Identifier: LicenseRef-CMSD-1.0
# Copyright (c) 2026 CyberMind — Gérald Kerma <devel@cybermind.fr>
"""ToolBoX whitelist (#492) : map host/SNI -> bypass-reason + category.
YAML format (operator can override at /etc/secubox/toolbox/whitelist.yaml) :
whitelist:
- pattern: "*.push.apple.com"
reason: cert-pinning, no MITM possible
category: vendor-os
quality_expected: A+
- pattern: "signal.org"
reason: E2E messenger, transport-only observation
category: messaging-e2e
quality_expected: A+
Patterns are fnmatch-style globs (Unix shell). Phase 4 will add regex/ASN/JA4 match.
"""
from __future__ import annotations
import fnmatch
import logging
from pathlib import Path
from typing import Optional
log = logging.getLogger("secubox.whitelist")
BASELINE_PATH = Path("/usr/share/secubox/toolbox/whitelist-baseline.yaml")
OVERRIDE_PATH = Path("/etc/secubox/toolbox/whitelist.yaml")
_CACHE: dict | None = None
_CACHE_MTIME: float = 0.0
def _load_yaml(path: Path) -> list[dict]:
if not path.exists():
return []
try:
import yaml as _yaml
except ImportError:
log.warning("PyYAML unavailable, whitelist disabled")
return []
try:
data = _yaml.safe_load(path.read_text()) or {}
return data.get("whitelist", []) or []
except Exception as e:
log.warning("whitelist load failed (%s): %s", path, e)
return []
def _refresh_cache() -> dict:
"""Hot-reload : combine baseline + override, re-read if either mtime changed."""
global _CACHE, _CACHE_MTIME
mtime = 0.0
for p in (BASELINE_PATH, OVERRIDE_PATH):
if p.exists():
mtime = max(mtime, p.stat().st_mtime)
if _CACHE is not None and mtime <= _CACHE_MTIME:
return _CACHE
entries = _load_yaml(BASELINE_PATH) + _load_yaml(OVERRIDE_PATH)
# Compile : map pattern -> entry. Override wins on dup (last in list).
compiled: dict[str, dict] = {}
for e in entries:
p = e.get("pattern", "").lower().strip()
if p:
compiled[p] = e
_CACHE = {"entries": compiled, "count": len(compiled)}
_CACHE_MTIME = mtime
log.info("whitelist reloaded : %d entries", len(compiled))
return _CACHE
def match(host: str) -> Optional[dict]:
"""Return the matching whitelist entry for a host, or None."""
if not host:
return None
h = host.lower().strip()
cache = _refresh_cache()
for pattern, entry in cache["entries"].items():
if fnmatch.fnmatch(h, pattern):
return entry
return None
def is_whitelisted(host: str) -> bool:
return match(host) is not None
def stats() -> dict:
"""Returns {count, baseline_loaded, override_loaded}."""
cache = _refresh_cache()
return {
"count": cache["count"],
"baseline_loaded": BASELINE_PATH.exists(),
"override_loaded": OVERRIDE_PATH.exists(),
}