mirror of
https://github.com/CyberMind-FR/secubox-deb.git
synced 2026-07-29 09:14:33 +00:00
0ca16778cc
* docs(plan): #349 sentinelle-gsm v0.3.1 L3 + scoring + baseline (ref #349) * feat(sentinelle-gsm): GsmtapListener exposes raw_l3 payload bytes (ref #349) v0.3.0's Observation carried only header metadata (arfcn, frame_nr, channel, sub_type), which was enough to prove the GSMTAP pipe but not to drive L3-aware scoring. v0.3.1 needs the post-header payload to decode BCCH System Information (LAC/CI/MCC/MNC) and CCCH paging requests (IMSI/TMSI for the privacy-hashed subscriber digest). Slice the L3 payload from the datagram via data[hdr["hdr_len"]:] in _on_datagram() and surface it as a new Observation field `raw_l3: bytes = b""`. Default empty bytes keeps every existing caller backward compatible — only downstream decoders that opt in will consume it (wired in Task 5). Test: existing test_listener_receives_a_datagram now appends a known 18-byte payload and asserts obs.raw_l3 matches verbatim. * feat(sentinelle-gsm): l3_decode — BCCH SI + CCCH paging request parsing (ref #349) Pure-Python TLV decoder for the minimal L3 subset needed by the IMSI-catcher scoring engine: BCCH System Information Types 3/4/6 and CCCH Paging Request Types 1/2/3. No scapy dependency — the parser operates directly on the post-GSMTAP-header bytes exposed by Observation.raw_l3 (Task 1). Privacy invariant : the public dataclasses (CellInfo, PagedIdentity, ParsedPagingRequest, ParsedFrame) NEVER carry plaintext IMSI/TMSI/IMEI. The internal _try_extract_mobile_id() returns plaintext bytes; _parse_paging() immediately hands them to Anonymizer.anonymize() and only the HMAC-truncated subscriber_hash escapes into PagedIdentity. The plaintext goes out of scope at function return. Permissive parsing : every helper returns an empty CellInfo() / None on unexpected layouts rather than raising — real GSMTAP frames have surprising structural variants and exceptions in the consume loop would crash livemon ingestion. Tests : 9 cases (6 from the plan + 3 added for coverage). BCD encoder/decoder round-trip helpers build synthetic frames programmatically rather than hand-crafting bytes, which also validates the TS 24.008 §10.5.1.3/4 nibble ordering. * feat(sentinelle-gsm): baseline.py — operator-baseline learning + cell_baseline table (ref #349) Adds CellBaseline wrapper + cell_baseline table colocated in observations.db. Cells graduate to baseline once observed >= LEARN_THRESHOLD (default 3) times. Key design choices: * LEARN_THRESHOLD = 3 by default — three sightings before a cell is considered part of the operator's legitimate baseline. * set_learn_mode(seconds) skips the count entirely — every cell observed inside that explicit-learn window is INSERTed with learn_count = 3 and graduates on first sighting (used by the operator's calibration sweep). * CellBaseline accepts a raw sqlite3.Connection (NOT an ObservationsDB instance) so it stays decoupled from observations.py's public surface while still sharing the same .db file. * cell_baseline table is created idempotently (IF NOT EXISTS) in ObservationsDB.__init__, so pre-existing observations.db files keep working after upgrade. Privacy invariant: BaselineCell + cell_baseline have NO subscriber-id columns — paged identities never enter the baseline path. Tests (5/5 passing): - test_first_consider_starts_at_1_not_baseline_yet - test_three_considers_graduate_to_baseline - test_learn_mode_graduates_first_consider - test_consider_updates_metadata_via_coalesce - test_list_orders_by_last_learned_desc Full pkg sweep: 62 passing (57 prior + 5 new). * feat(sentinelle-gsm): scoring_engine — 8 heuristics + thresholds (ref #349) Replaces the v0.1 shape-only scoring.py stub with the full v0.3.1 detector brain. ScoringEngine wires against CellBaseline + L3Decode (Tasks 2-3) and runs every parsed frame through 8 heuristics, summing the triggered contributions into a CellScore clamped to [0, 100]. Heuristics + default scores (sum-clamped to 100): 1. cipher_downgrade 40 A5/X observed < baseline.cipher_a5 2. ghost_bts 35 cell unknown to baseline + paging seen 3. identity_mismatch 30 observed (mcc, mnc) != baseline pair 4. relocalization_storm 25 > 3 distinct LACs in 60s on cell 5. identity_request_abuse 35 > 2 paging reqs in 300s per hash 6. anomalous_neighbours 15 neighbour list went non-empty -> empty 7. t3212_out_of_band 15 T3212 outside [1, 60] min 8. orphan_arfcn 20 FR carrier announces ARFCN outside plan Rolling-window state (in-memory only, no DB persistence in v0.3.1): _lac_window cell_id -> deque[(ts, lac)] _idreq_window subscriber_hash -> deque[ts] _neighbour_baseline cell_id -> set[int] (cumulative ARFCNs) Thresholds: * DEFAULT_THRESHOLDS class constant carries per-heuristic config dicts. * update_thresholds() does a per-heuristic deep-merge so callers can pass partial updates like {"cipher_downgrade": {"score": 55}} without clobbering "enabled" or any other unspecified field. * thresholds() returns a deep-copy so callers can't mutate engine state. Privacy invariant kept: identity_request_abuse keys windows on subscriber_hash strings (HMAC-trunc, supplied by l3_decode.Anonymizer), never plaintext IMSI/TMSI. Reasons truncate the hash to 8 chars defensively. France GSM-900 ARFCN plan is approximate and permissive: unknown (mcc, mnc) pairs return triggered=False to avoid false-positives. Dropped lib/sentinelle_gsm/scoring.py — the v0.1.0 shape-only stub (empty Baseline + score()=0) is replaced. No other module imported it. Tests: * api/tests/test_scoring_engine.py — 11 tests, one per heuristic (plus permissive-unknown-carrier guard), one for evaluate() aggregation/clamping, one for update_thresholds() deep-merge. * Full sweep: 73 passed (was 62; +11 new). * feat(sentinelle-gsm): consume loop wires L3 + scoring + trusted match → Alert (ref #349) - Extracted `_process_observation(obs)` helper for testability — the consume coroutine now delegates per-frame work so tests can drive synthesized Observations through the full pipeline without binding UDP. - New endpoints: GET /baseline, POST /baseline/learn (sweep window), GET /scoring/thresholds, POST /scoring/thresholds (deep-merge). - Audit log entry on POST /scoring/thresholds via the existing `secubox.sentinelle-gsm.api` logger — surfaces in /journal/stream. - Alert emission decision tree: score < threshold → drop (no write) score >= threshold + match → one labeled alert per matched trusted phone score >= threshold + no match → single anomaly-only alert - ALERT_THRESHOLD configurable via SENTINELLE_ALERT_THRESHOLD env (default 60). - TrustedRegistry now hashes the IMSI as `imsi.encode("ascii").hex()` so the persisted token matches the canonical form produced by the L3 decoder's paging-request path. Existing trusted tests don't pin the hash value, so behaviour is unchanged for read-back / lookup-by-id. - 5 new integration tests at api/tests/test_alert_emission.py cover the below-threshold drop, anomaly-only emission, trusted-match labeling, paging-event hash persistence, and audit-log emission on threshold update. Full sweep 78/78 passing. * feat(sentinelle-gsm): webui baseline + scoring panels + trusted_label chip (ref #349) Adds two new webui panels and a trusted-label chip on the alerts row: - #baseline panel: Cell ID / MCC / MNC / LAC / ARFCN / cipher / learn count / last learned table backed by GET /api/v1/sensor/gsm/baseline. "Start learn (5 min)" button arms POST /baseline/learn?sweep_seconds=300 and re-polls the table every 15s for the 5-minute window so cells appear as they graduate. - #scoring panel: per-heuristic enable + score input (0..100, clamped client-side) backed by GET/POST /scoring/thresholds. Form values are cached in _thresholdState so any backend-only fields (heuristic-specific knobs) survive the POST round trip. - Alerts table: trusted_label is now rendered as a violet pill via the new .trusted-chip class (replaces the older .target-pill markup for the same data path). Local navigation gets two new anchors (Baseline, Scoring) between Observations and Actions. The 10s scan poll now also refreshes baseline while a scan is running. Wires 4 new event listeners (refresh-baseline, baseline-learn, refresh-scoring, save-scoring). No backend changes; all endpoints already exist (Tasks 1..5). Tests: 78/78 passing. * chore(sentinelle-gsm): bump 0.3.1 + extend privacy invariant tests (closes #349) Append 4 new structural privacy tests under tests/test_privacy_invariant.py to lock the v0.3.1 invariants in place: * test_l3_decode_returns_no_plaintext_fields — CellInfo, PagedIdentity, and ParsedPagingRequest dataclasses must not carry any plaintext subscriber-id field (imsi/tmsi/imei/msisdn/iccid/subscriber_id). * test_paging_request_hashes_paged_identities — feeds a synthetic Paging Request Type 1 with a known TMSI through L3Decode and asserts the plaintext TMSI bytes (as hex) NEVER appear in the resulting PagedIdentity.subscriber_hash. * test_cell_baseline_has_no_subscriber_fields — the BaselineCell dataclass must remain operator-side metadata only: no subscriber_hash, no plaintext identifier. * test_scoring_engine_reasons_contain_no_plaintext_imsi — the free-text reason strings produced by ScoringEngine.evaluate() must never echo a 15-digit token (plaintext-IMSI shape) for typical observations. These four tests pass against the v0.3.1 modules (Tasks 1-6) with no code change — the modules were designed for these invariants from the start, this commit just locks them in regression-test form. Full sweep: 78 -> 82 passing. Changelog: 0.3.0 -> 0.3.1 with the v0.3.1 entry summarising the new l3_decode, baseline, scoring_engine modules, the consume-loop wiring that emits qualified Alerts with trusted_label, the webui panels, and the updated test breakdown (l3_decode x9, baseline x5, scoring_engine x11, alert_emission x5, privacy x4 = 34 new, full sweep 82/82). --------- Co-authored-by: CyberMind-FR <gandalf@Gk2.net> |
||
|---|---|---|
| .. | ||
| architecture | ||
| assets | ||
| design | ||
| errata | ||
| eye-remote | ||
| hardware | ||
| reference | ||
| references/gst | ||
| reports | ||
| screenshots | ||
| superpowers | ||
| wiki | ||
| AI-BUILD-PROMPT.md | ||
| CACHE-PATTERN.md | ||
| FAQ-BUSYBOX-RESCUE.md | ||
| grammar.md | ||
| LED-HEALTHBUMP.md | ||
| LIVE-USB.md | ||
| MODULE-GUIDELINES.md | ||
| MODULES.md | ||
| OPENWRT-CRT-THEME.md | ||
| OPENWRT-DEBIAN-COMPARISON.md | ||
| OPENWRT-MASTERLINK.md | ||
| OPENWRT-MESH-DAEMON.md | ||
| PROMPT-BUSYBOX-TECHTIP.md | ||
| SCREENSHOTS-VM.md | ||
| SECUBOX-DEV-METHODOLOGY.md | ||
| TOOLS.md | ||
| UI-GUIDE.md | ||