Commit Graph

89 Commits

Author SHA1 Message Date
c85d0d82a0 feat(streamlit): Per-app ingest function (idempotent, history-preserving) (ref #95)
URL uses gitea@ (Gitea built-in SSH server validates against the
OS user 'gitea', not 'git'). Mirrors scripts/lib/metablog-ingest-site.sh
from sub-project B (PR #97) with the streamlit- prefix.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 16:29:34 +02:00
CyberMind
b0b6e78cab
Merge pull request #84 from CyberMind-FR/feature/license-headers-phase-a
feat(license): CMSD-1.0 header tool, CI, and conventions (Phase A)
2026-05-12 11:41:43 +02:00
b88b8ada95 fix(license): tighten detect_existing + missing-file = repo-wide (ref #81)
Two bug fixes to the CMSD-1.0 license-header tool:

1. detect_existing() previously matched the SPDX token anywhere in the
   first 10 lines, including inside docstrings and prose comments.
   Tightened the matcher to require comment markers (#, //, *, <!--)
   and whitespace before the SPDX token.

   Regression tests:
   - test_detect_existing_no_false_match_in_docstring
   - test_detect_existing_no_false_match_inline_comment_prose

2. _read_enrollment() now returns ["**"] when the allowlist file is
   absent, per spec §5.2 "missing file → repo-wide enforcement".
   Previously it returned [] (nothing enforced), making Phase C closure
   impossible without an additional file.

   Empty allowlist file still returns [] (Phase A initial), so
   test_main_empty_allowlist_passes_check is unaffected.

   New tests:
   - test_read_enrollment_missing_file_means_repo_wide
   - test_main_check_missing_allowlist_enforces_repo_wide

Suite: 49 → 53 passing. --check still exits 0 repo-wide.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 11:39:57 +02:00
ba9f6b20c1 fix(scripts): re-apply secubox-haproxy-regen-safe sub-command fix
A merge from feature/83 reverted the fix from commit 22014865, which had
already replaced `haproxyctl regen` (non-existent sub-command, printed
help text and exited 0) with the correct `haproxyctl generate`.

This commit re-applies the same fix.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 11:17:34 +02:00
2763009fad fix(license): walk() accepts repo_root for subdir invocations (ref #81)
Discovered during Phase B pilot: running `--fix packages/secubox-hub`
yielded zero files because walk() computed `rel` relative to the walk
root (the subdir) while allowlist patterns are repo-relative.

Add optional `repo_root` parameter to walk(); when present, allowlist
matching uses paths relative to it instead of the walk root.
Backwards-compatible (defaults to None, preserving existing tests).
main() passes the discovered repo_root through.

Regression test added: test_walk_subdir_with_repo_root_arg.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 11:03:32 +02:00
e31c1fa99b fix(sync-routes): preserve streamlit & custom routes from clobbering
After Session 156 fix, secubox-haproxy.service was safely restarted but
cpf.gk2.secubox.in (and other streamlit instances) immediately fell back
to "Wrong Domain" because sync-mitmproxy-routes.sh was clobbering their
container routes.

Two compounding bugs:
1. DEAD_CONTAINER_IPS included 10.100.0.50 — but that's the streamlit
   LXC's actual IP (RUNNING). fix_dead_container_routes() rewrote every
   route pointing to 10.100.0.50 (cpf:8523, etc.) to 10.100.0.1:9080.
2. The update conditional reapplied routes when the existing port didn't
   match the computed port. The computed port logic only knows metablog
   vs webui — it has no streamlit awareness, so it always tried to force
   streamlit domains to port 9080 (webui default).

Fixes:
- Drop 10.100.0.50 from DEAD_CONTAINER_IPS. The streamlit LXC at that
  address is alive; routes pointing to it are valid.
- Change "update if existing missing OR port mismatch" to "update only
  if existing is missing." The script now only ADDS routes for ACL/
  metablog domains; it never overwrites existing routes set by other
  sources (sync-all-routes.sh, manual edits, etc.). fix_dead_container_
  routes() still handles truly dead IPs separately.

Verified: cpf.gk2.secubox.in → HTTP 200 "Streamlit" persists across
sync-mitmproxy-routes.service runs. arm/admin still 200 with correct
content.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 11:02:42 +02:00
c911e8192f fix(haproxy): generator emits mitmproxy_inspector + safe regen wrapper
Triggered by cpf.gk2.secubox.in returning "Wrong Domain". Root cause: the
secubox-haproxy generators (both bash haproxyctl and Python FastAPI) emit
config rules that point to dead/wrong backends, leading to HTTPS-wide
metablog/streamlit outages when a regen runs.

Generator fixes:
- packages/secubox-haproxy/api/main.py: use_backend waf_inspector
  → use_backend mitmproxy_inspector (2 occurrences). waf_inspector
  pointed to 127.0.0.1:8890 which is not listening; mitmproxy_inspector
  is the actual healthy WAF backend at 10.100.0.60:8080.
- packages/secubox-haproxy/sbin/haproxyctl: default_backend fallback
  → default_backend mitmproxy_inspector (2 occurrences). The fallback
  backend deny_status 503's unknown hosts; switching to mitmproxy_inspector
  lets mitmproxy dispatch all hosts via its routes JSON (245 routes vs
  93 declared in haproxy.toml).

New safe regen wrapper:
- scripts/secubox-haproxy-regen-safe: snapshot → regen via haproxyctl →
  validate via haproxy -c -f → atomic swap → reload. On validation
  failure, current cfg is preserved and the broken candidate is saved
  for forensics. Prevents another silent breakage like Session 156.

Board-side state (not in repo): /etc/haproxy/haproxy.cfg patched in
place to mirror these fixes, secubox-haproxy.service stopped until
user confirms safe to re-enable. All 245 routes verified live (cpf,
arm, lldh, admin, pub, werdl, 3d, 42, zkp all HTTP 200 with correct
content).

See .claude/HISTORY.md Session 156 for full symptom chain and the
6-layer root cause analysis.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 10:52:44 +02:00
c661c4f053 fix(scripts): use issue title for PR, surface push errors as exit 4 (ref #83)
Addresses code review:
- Important #4: finish now fetches issue title via `gh issue view --json title`
  and uses it as PR title (was using the branch name). Falls back to branch
  name if the fetch fails or returns empty.
- Minor #8: explicit `return 4` on `git push` failure for consistency with
  the documented exit-code contract.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 10:22:14 +02:00
b77a49fc2f docs(scripts): document agent-worktree in scripts/README (ref #83)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 10:05:48 +02:00
4ff4830df4 feat(scripts): implement agent-worktree clean sub-command (ref #83)
Also patches gh-mock to sanitize dashes in branch-derived env var names,
and fixes --head two-arg parsing in the pr view mock handler.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 10:03:47 +02:00
9eb75676d5 docs(license): document license-headers.py in scripts/README (ref #81)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 10:01:46 +02:00
4d937995ba ci(license): add License Headers workflow (ref #81)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 10:00:54 +02:00
eba35ff5b5 feat(scripts): implement agent-worktree finish sub-command (ref #83)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 10:00:24 +02:00
e90a07a578 chore(license): self-host header on tool and tests + initial allowlist (ref #81)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 10:00:09 +02:00
4554cb991e feat(scripts): implement agent-worktree sync sub-command (ref #83)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 09:59:01 +02:00
c93cfdb639 feat(license): CLI dispatch with --check/--fix/--list/--diff (ref #81)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 09:57:19 +02:00
70e145ba62 feat(scripts): implement agent-worktree list sub-command (ref #83)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 09:57:00 +02:00
149304911d test(scripts): cover agent-worktree start edge cases (ref #83)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 09:55:17 +02:00
d5df48c5d8 fix(infra): mitmproxy route sync stability — restore metablogizer sites
Root cause: log() in sync-mitmproxy-routes.sh wrote to stdout, polluting
$(fix_dead_container_routes ...) capture → corrupted JSON → jq parse
errors → set -e abort → bad routes pushed to mitmproxy container →
mitmproxy restart-loop → HAProxy backend DOWN → HTTP 503 on every
*.gk2.secubox.in metablog vhost (arm, zkp, 3d, ~160 sites).

Fixes:
- log() now writes to stderr (>&2) — preserves stdout for command-sub
- fix_dead_container_routes returns 0 (was returning $fixed count, set -e
  killed the caller's assignment)
- sync-all-routes.sh routes metablog domains to port 8900 (was 9080,
  which is webui.conf default_server → "Wrong Domain")
- defensive 2>/dev/null || true on jq reads
- fallback to old routes_json on jq write failure (preserves last valid
  state instead of crashing)
- flock guard prevents concurrent runs on /run/sync-mitmproxy-routes.lock

Verified: systemctl start sync-mitmproxy-routes.service exits 0/SUCCESS,
244 routes valid in mitmproxy container, all four spot-checked domains
(admin.gk2, arm.gk2, zkp.gk2, 3d.gk2) return HTTP 200 with correct titles.

Note: also disabled duplicate timer secubox-route-sync.timer on the
board (executed the same script, racing on same file). systemd-level
change is on board only, not in repo.

See .claude/HISTORY.md Session 153 for full symptom chain and verification.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 09:54:52 +02:00
72733385eb feat(license): walk() with skip-list and enrollment allowlist (ref #81)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 09:54:35 +02:00
496592913b feat(scripts): implement agent-worktree start sub-command (ref #83)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 09:54:12 +02:00
33768c7854 feat(scripts): add agent-worktree CLI skeleton with help and dispatch (ref #83)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 09:51:38 +02:00
2bd965f481 feat(license): apply() for HTML (doctype) and Markdown (frontmatter), plus TOML/YAML/conf (ref #81)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 09:50:48 +02:00
a67c4c6b6e feat(scripts): add prefix_from_labels helper (ref #83)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 09:50:35 +02:00
ada5040693 feat(license): apply() for JS/TS/CSS/C/H with idempotence (ref #81)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 09:49:28 +02:00
c74e71d890 feat(scripts): add agent-worktree library with derive_slug (ref #83)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 09:49:02 +02:00
6474728dc4 feat(license): apply() for Bash with shebang-aware placement (ref #81)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 09:48:43 +02:00
6f4bd948a3 test(scripts): add gh CLI mock for agent-worktree tests (ref #83)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 09:46:20 +02:00
63e62c1d54 feat(license): apply() for plain Python, with idempotence and foreign-skip (ref #81)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 09:45:34 +02:00
72771702e9 chore(scripts): scaffold tests directory for agent-worktree (ref #83)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 09:44:34 +02:00
5b4d3e25fb feat(license): detect_existing distinguishes CMSD/foreign/none (ref #81)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 09:41:04 +02:00
0f1907df15 fix(scripts): Decouple chroot apt-update grep from pipefail (ref #80)
apt-get update may exit non-zero on unsigned/file:// warnings while
the SecuBox repo is correctly discovered. Capture the log first
(via tee, so root-owned chroot output is writeable), then grep
separately so pipefail can't mask a successful discovery.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 09:39:14 +02:00
70ac535e68 feat(license): render_header for slash/block/html comment styles (ref #81)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 09:38:20 +02:00
f08eb38073 feat(license): render_header for hash-comment languages (ref #81)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 09:34:43 +02:00
bd7975d12f chore(license): fix NotImplementedError task reference (ref #81)
Per code review: main() is implemented in Task 11 (CLI dispatch), not Task 9.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 09:33:03 +02:00
bb58789b58 feat(scripts): Add validate-staged-repo.sh (ref #80)
Four-step validation gate: reprepro check, gpg verify on InRelease,
license byte-match against project root, optional chroot apt-update
smoke test (SKIP_CHROOT=1 to disable).

Accepts French or English "Good signature" / "Bonne signature" so
the gate works under any locale.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 09:32:46 +02:00
d6fe14d535 feat(scripts): Render nginx vhost + DEPLOY.md + license artifacts (ref #80)
Generates output/repo/{nginx-apt.conf, DEPLOY.md, install.sh,
LICENCE-CMSD-1.0.md, LICENSE-CMSD-1.0.en.md}. No network operations
- artifacts are for the user to push out-of-band.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 09:32:01 +02:00
5f7b847435 feat(scripts): Add stage-apt-repo.sh orchestrator (ref #80)
Drives GPG bootstrap, reprepro init, tier-by-tier cross-build
(via build-packages.sh --filter), publish, and check gate. Halt
on tier failure unless --keep-going.

Smoke-tested on base tier (secubox-core + secubox-hub, amd64).
Release file Origin=SecuBox, InRelease GPG-verified Good signature.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 09:31:16 +02:00
c7166597f5 chore(license): scaffold license-headers tool and test module (ref #81)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 09:29:28 +02:00
3b99bcf4ec feat(scripts): Add GPG bootstrap wrapper for staged repo (ref #80)
Pins GPG_HOME=~/.gnupg/secubox (persistent), invokes existing
generate-gpg-key.sh, and writes the long fingerprint to
FINGERPRINT.txt for downstream consumption by reprepro SignWith.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 09:27:22 +02:00
52463db1e7 fix(scripts): Harden tier-manifest helper per code review (ref #80)
- Guard yaml.safe_load returning None on empty manifests
- Reject non-string entries in .packages[] with clear error
- Use RETURN trap for tmpdir cleanup (covers Ctrl-C, set -e trips)
- Document python3-yaml dependency + add friendly pre-check

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 09:24:28 +02:00
6f59de25c7 feat(scripts): Add tier-manifest helper for APT staging (ref #80)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 09:16:32 +02:00
ce82e13db1 feat(scripts): Add --filter and --dry-run to build-packages.sh (ref #80)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 09:11:14 +02:00
4529b5c11a docs(spec): CMSD-1.0 license headers across the codebase
Brainstormed design for adding the SPDX-CMSD-1.0 header to every
first-party source file (~2,170 across 6 languages), backed by a
reusable Python tool (scripts/license-headers.py), a CI check, and
a phased per-package rollout. Spec covers scope, header rendering
per language, placement rules, tool architecture, CI integration
with an enrollment allowlist, and verification steps.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 09:10:37 +02:00
bd7dda0c6f feat(secubox): complete meta-script generator Tasks 14-17
Task 14: Arch Profiles
- Add profiles/arch/arm64.yaml and profiles/arch/amd64.yaml
- Add ResolveWithArch() to merger for base → arch → tier chain
- Add tests for arch profile resolution

Task 15: Package secubox.yaml for All Packages
- Add scripts/generate-secubox-yaml.py generator script
- Generate 131 debian/secubox.yaml files with:
  - Category detection (security, network, system, etc.)
  - Tier assignment (all, lite, standard, pro)
  - API socket and UI path detection

Task 16: APT Repository Setup
- Add apt/conf/ reprepro configuration (multi-arch arm64/amd64)
- Add apt/hooks/lintian-check pre-publish validation
- Add scripts/apt-publish.sh and scripts/apt-sync.sh
- Add apt/README.md documentation

Task 17: CI Integration
- Add .github/workflows/build-secubox-cli.yml
- Build for linux-amd64 and linux-arm64
- Version injection via ldflags
- GitHub releases on tag push

Code Quality Fixes (Tasks 10-13):
- Add atomic writes for OTA boot control files (0600 perms)
- Fix NVME device extraction (nvme0n1p2 → nvme0n1)
- Add scanner.Err() checks in hardware detection
- Fix URL injection validation in fetch command
- Add proper cleanup on checksum failures

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-11 05:32:29 +02:00
128ee01209 fix(waf): Optimize mitmproxy WAF and fix dead container routes
WAF Optimizations:
- Skip WAF checks for static assets (.js, .css, .png, etc.)
- Skip WAF checks for /health, /status, /system_health
- Skip WAF checks for trusted hosts (git, admin)
- Fixed MultiDictView.to_dict() → dict() AttributeError
- CPU: 90%+ constant → 0% idle, 25-90% under load

Route Sync:
- Auto-fix routes pointing to dead containers (10.100.0.10-50)
- Dead routes redirect to webui (9080) instead of timeout
- Fixed bash arithmetic for set -e compatibility

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-09 13:27:18 +02:00
a455b1735e feat: Add route sync, enhanced export, and session management
- scripts/sync-mitmproxy-routes.sh: Auto-sync HAProxy vhosts to mitmproxy
  - Runs via systemd timer every 5 minutes
  - Fixed bash arithmetic for set -e compatibility

- secubox-metablogizer: Enhanced export ZIP package
  - Includes nginx.conf, haproxy.cfg, certificates
  - Complete README with republishing instructions
  - Site health check endpoint

- secubox-users: Added REVOKE ALL sessions panic button
  - Emergency endpoint to revoke all active sessions
  - Double confirmation UI for safety

- secubox-publish: Added health status columns
  - HTTP/HTTPS/WAF status with emoji indicators
  - Aggregated health LEDs bar

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-09 13:11:34 +02:00
e2989322f4 feat(api): Add /health endpoint to 53 modules
Auto-generated health check endpoints for sidebar status:
- Returns {status: "ok", module: "name"}
- Public endpoint (no auth required)
- Used by sidebar.js for LED status display

Added via scripts/add-health-endpoints.py

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-09 10:40:05 +02:00
eef430e123 feat(docs): Add screenshot capture system and multilingual wiki generator
- scripts/capture-screenshots.py: Auto-discovers 117 modules from packages/
  - Playwright-based capture with JWT authentication
  - 30-second delay per page for cache refresh
  - Thumbnail generation (400x225)
  - JSON manifest output

- scripts/generate-docs.py: Extended to 105 modules
  - Multilingual descriptions (EN, FR, DE, ZH)
  - --include-screenshots flag for wiki integration
  - Category index pages (CATEGORIES-XX.md)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-09 10:09:21 +02:00
919d030a58 feat(ui): Add dice icons, scribe trace, and LED pulse improvements
- Sidebar: Replace LED dots with dice icons (⚀-⚅) based on metric values
- Sidebar: Dice update dynamically with CPU/mem/disk/load/temp/net
- SOC: Add scribe trace histogram for firewall packet stats
- SOC: Remove category tag listings (keep donut metrics)
- SOC: Fix API paths for hub endpoints
- WAF: Fix JavaScript syntax errors (orphan returns, extra braces)
- WAF: Fix getSiteEmoji function structure
- HealthBump: Update LED pulse patterns (1/2/4 flashes per tier)
- HealthBump: Pulse from bright to dim (not dark to color)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-08 22:33:23 +02:00