Kaizen Dashboard

Repo	Size	Remote	Pushed	Risk
sentinel	11MB	GitHub ✓	✓ RESOLVED	F-256 closed (iter #178)
kaizen	5.5MB	NONE	—	177 iterations of improvement history
/root/scripts	288K	Git ✓	✓ (iter #187)	Has remote, committed
qa	180K	NONE	—	86+ tests, NO commits at all

Coverage: 7/12 repos fully backed to GitHub (58%)
Impact: If sentinel server dies, these 4 repos are permanently lost. Service repos (auth, billit, pulse...) survive.
Action: TODO sent to sentinel (push). Libor notified via fess.

0 unread across all 20 queues (new: hive queue). Stable since iter #181.
Resolution: Combination of sentinel execution, agent triggers, and natural consumption cleared the backlog.
Previous alert: F-230 (stagnation), F-261 (acceleration) — both RESOLVED.

Repo	7d Commits	48h	Status
Pulse	15	10	VERY ACTIVE
Billit	14	6	VERY ACTIVE
BadWolf	10	2	ACTIVE
Auth	9	0	PAUSED
Mail	5	1	MODERATE
Sentinel	3	0	STAGNANT 5d
Venom	2	0	DORMANT (F-220)

Conventional commit rate: 96% (88/91) — +28pp from 68% baseline
Kaizen-attributed commits: 10 across pulse (3), billit (4), badwolf (2), sentinel (1)
Key pattern: Security/HC fixes get implemented fast. Housekeeping (npm audit, backup expansion) stalls.

F-217	kaizen repo has NO git remote — 128 iterations, state/, reports/, simulations/ at risk of total loss
F-218	/root/scripts/ NOT version controlled — 9 critical ops scripts (relay-to-telegram, deploy.sh, qa-gate.sh, agent-trigger.sh)
F-219	sentinel has uncommitted changes — 8 files, 414 insertions, remote fetch status unknown

Git remote coverage: 7/8 repos have remote (only kaizen missing). /root/scripts/ on 3 servers but no git.
F-169 billit-web HC IPv6: VERIFIED ✓ — healthy FS=0 after fix (3 commits by billit agent).

Agent	run.sh	Cron	Unread	Status
kaizen	✓	*/15	—	AUTONOMOUS
sentinel	✓	✗	0	HAS RUNNER, NO CRON
qa	✓	✗	—	HAS RUNNER, NO CRON
venom	✗	✗	5	NO RUNNER
badwolf	✗	✗	3	NO RUNNER
mail	✗	✗	2	NO RUNNER
pulse	✗	✗	2	NO RUNNER
auth	✗	✗	1	NO RUNNER
billit	✗	✗	1	NO RUNNER

Root cause of delegation failures (F-204, F-220), unread relay messages (14 total), and sentinel stagnation.
Fix: Add sentinel + qa run.sh to cron. Create run.sh for service agents. Sim-017 blocked on Libor.

Item	Delegated To	Status
F-203 Sentinel git remote	sentinel	DONE ✓
F-174 Pulse CORS whitelist	pulse	DONE ✓
F-162 Billit API scopes	billit	DONE ✓
F-180 Pulse HC port fix	sentinel/pulse	DONE ✓
F-169 billit-web HC IPv6	billit	DONE ✓
Sim-031 QA Pipeline	QA agent	DONE ✓
F-209 Venom npm audit	venom	NOT DONE
F-210 BadWolf npm audit	badwolf	NOT DONE
F-201 Pulse M2M scope	pulse	DONE ✓
F-204 /root/scripts backup	sentinel	NOT DONE
Sim-030 P1 .env.example	sentinel/PM	NOT DONE
Sim-022 Volume backup	sentinel	NOT DONE

Pattern: Security/HC fixes (high visibility, clear impact) → get done. Housekeeping improvements (npm audit, backup expansion) → stall. Items requiring Libor → blocked until direct contact.

3-tier deploy pipeline: smoke → standard → full test coverage per service.
Architecture: sentinel runs tests/run.sh directly (no cerebro roundtrip).
qa-gate.sh proposal: wraps sentinel test framework + npm test.
Implementation delegated to sentinel.

COMPLETE	Track A (Runtime) — Mode detection, mandatory block, feature flags, credential cleanup, GraphRAG, auto memory write (7/7)
PARTIAL	Track C (deploy.sh) — deploy.sh + common.sh created. BUG: billit container names wrong (F-187).
COMPLETE	Track F (GraphRAG/Memory) — knowledge_domains in 9 YAMLs, memory audit done (51 files, dedup identified), auto memory write verified.
COMPLETE	Track H (Scaffolding) — create-project.sh exists (225 lines), templates in s60-tools, 19/19 agents consistent.

Status: 5/8 tracks complete, 1 partial (C), Track B exists differently than planned, 2 remaining.

Complete secret inventory across 11 services, 5 servers, ~40 credentials mapped.

CRITICAL	F-189: changeme123 hardcoded in 4+ locations (Redis compose, CLAUDE.md files, Neo4j)
HIGH	F-190: Zero secret rotation ever — no policy, no tooling, no audit log
HIGH	F-191: Payment credentials (Stripe/GoPay) in plaintext .env, no extra protection
HIGH	F-194: Backup secrets not encrypted at rest (rsync plaintext to argus/Hetzner)
MEDIUM	F-192: 5 .env.example files on prod-alfa — may reveal structure
MEDIUM	F-193: billit-redis runs without password (network-isolated, low risk)

Plan: Phase 1 (CLAUDE.md cleanup) delegated to PM + sentinel. Phase 2 (password rotation) requires Libor approval. Phase 3 (SOPS/age encryption) future.

Claude Code v2.1.76 feature utilization: 8% (3/39 flags). 0 plugins, 0 hooks, 2 broken MCP servers.

F-195	66 stale session files (40MB) in kaizen project — no cleanup mechanism
F-196	2 broken MCP servers (Google Calendar/Gmail) — never authenticated, useless
F-197	0/43 plugins installed despite code-review, hookify, security-guidance available
F-198	0 hooks configured — no safety guardrails on autonomous agents
F-199	Feature utilization 8% — missing --max-budget-usd, --effort, --fallback-model, --name

SELF-IMPROVED run.sh updated: --no-session-persistence, --name, --max-budget-usd 0.75, --fallback-model sonnet
Next: Plugin install (hookify, security-guidance) + MCP cleanup — propose to Libor.

Pulse: 10 commits (regression tests, error tracking, BullMQ queues, bug reporting)
Billit: 9 commits (API key scope enforcement ✓, product catalog, admin, PDF cache)
BadWolf: 7 commits (DB migration 020 ✓, BillitSync fix, RelayService)
Mail: 1 commit (CLAUDE.md update)

RESOLVED	F-162: Billit API key scopes — ApiKeyScopeGuard + default-deny deployed
RESOLVED	F-168: BadWolf missing tables — Migration 020 created courses, locations, online_courses, companies
PARTIAL	F-163: BillitSync — Staging URL hardcode removed, service created. Data backfill still needed.

Fixed: nginx IPv6 enabled, FS=0. Healthcheck passing permanently after restarts.

Iter #57 verification: tier column is varchar(20), NOT enum. Data stores hexa (3), full (3), null (2) correctly.
The pg_enum values (full_pack, monthly) are from an old unused type, not a constraint.
Billing is NOT broken. 0 runtime errors confirmed.

Iter #57 verification: billitInvoiceId column exists in credit_transactions table.
Migration 001_add_billit_invoice_id.sql (commit 278861e) was applied. 0 billing webhook errors.
NEW F-182: Pulse has no automated migration runner — only manual SQL files in migrations/. Future schema changes need manual SQL.

Iter #57 verification: billit-redis now on BOTH networks (s60-network + billit_billit-internal).
Errors: 802/h → ~3/h (99.6% reduction). 20 errors in 6h vs 4,800+ previously.
billit-api on s60-network, can now reach billit-redis via DNS. BullMQ + caching operational.
Remaining: Permanent fix needed in docker-compose.yml (current fix = manual network connect, may reset on restart).

Migration 020 created locations table WITHOUT deleted_at column. Entity expects it.
4 errors: column Location.deleted_at does not exist. Soft-delete broken.
FIX: ALTER TABLE locations ADD COLUMN IF NOT EXISTS deleted_at TIMESTAMP;

Iter #80: Pulse Dockerfile HC port 3200→3100 fixed (F-180 RESOLVED). billit-web IPv6 resolved (F-169).
Iter #81: Permanence confirmed — after night restarts (Pulse 03:36, billit 03:25 UTC), ALL containers healthy with FS=0.
Score: 9/9 services with healthchecks = 100% healthy. Only n8n + billit-redis without HC (by design).
Duration: 16 iterations of escalation to achieve. Sim-026 IMPLEMENTED ✓

PasswordAuthentication no on BOTH servers (prod-alfa + hub-alfa).
fail2ban remains ACTIVE — 0 currently banned, 369 total (brute-force now blocked at SSH level).
Combined with UFW + fail2ban = SSH fully hardened.

BillitSync service created, staging URL hardcode removed. But:
• All online_courses have company_id = NULL → sync skips 100%
• Prices not synced (unitPrice: null)
Status: Awaiting data backfill (needs Libor confirmation for company_id mapping).

19 queues total. Sentinel: 6 unread (↑↑ from 3). Minor: badwolf=2, venom=1, auth=1, mail=2.
Total messages processed: pm 316 | billit 405 | infra 197 | sentinel 193 | auth 163 | main 126 | pulse 89 | badwolf 80.
Pattern: sentinel reads kaizen messages but doesn't act on housekeeping (F-213 PG fix, F-204 scripts backup). Security/HC fixes get done, improvement tasks stall.

FIXED	UFW Firewall ACTIVE on both servers (F-130)
FIXED	Docker ports bound to 127.0.0.1 (F-132)
FIXED	s60-redis has volume + AOF (F-131)
FIXED	Nginx attack paths blocked — .php/.env/.git/wp-* → 403 (F-134)
FIXED	Billit API key scopes enforced — default-deny guard (F-162) ✓
FIXED	fail2ban active on both servers (F-166)

FIXED SSH password auth disabled — both servers (F-176, Sim-023) ✓
Implementation rate: ~43% → ~46% (↑ Sim-025 Phase 1a progress)

12/12 databases backed up successfully at 3:00 AM Mar 14.
P0 closed after 7 escalations. DO Managed PG remains as safety net.

1. Pulse Dockerfile HC port ✓ — RESOLVED (Sim-026, FS=0 permanent)
2. Pulse CORS whitelist ✓ — RESOLVED (commit 3321ed0, allowedOrigins whitelist deployed, evil.com rejected)
3. N8n stop (10 sec): docker stop s60-n8n — free 294MB RAM
4. billit-web HC fix ✓ — RESOLVED (nginx IPv6, FS=0 permanent)
5. Billit-Redis compose fix: Add billit-redis back to docker-compose.yml on s60-network (currently manual network connect)
6. Password rotation: changeme123 on s60-redis + neo4j
7. Pulse tier DB ✓ — FALSE POSITIVE (varchar, not enum) — data correct
8. Pulse billitInvoiceId ✓ — RESOLVED (column exists, migration 001 applied)
9. Billit-Redis network ✓ — RESOLVED (both networks connected, ~3 err/h)
10. SSH password disable ✓ — DONE on both servers (Sim-023)

Billit API key scopes — ApiKeyScopeGuard deployed, default-deny (F-162) ✓
BadWolf missing tables — Migration 020 deployed (F-168) ✓
NO FIREWALL — UFW ACTIVE, deny default (F-130, Sim-020) ✓
Nginx returns 200 for attacks — security-deny.conf on all sites (F-134, Sim-021) ✓
Docker ports 0.0.0.0 — All bound to 127.0.0.1 (F-132) ✓
s60-redis NO VOLUME — Named volume + AOF enabled (F-131, Sim-015 partial) ✓
Auth frontend/backend unhealthy — Both FailingStreak=0 (F-087, F-093) ✓
Docker log rotation missing — daemon.json deployed, 10m/3 files (F-151) ✓
Fess→Telegram missing — Sim-019 IMPLEMENTED, bidirectional bridge ✓
PG backup cron broken — 12/12 OK @ 3:00 Mar 14 (F-160) ✓
Pulse HC port mismatch — 3200→3100, FailingStreak 763→0 (F-173, still regresses F-180) ✓
SSH password auth enabled — Disabled on both servers (F-176, Sim-023) ✓
Pulse tier DB mismatch — FALSE POSITIVE: varchar(20), data correct (F-178 corrected iter #57) ✓
Pulse billitInvoiceId — Column exists, migration 001 applied (F-155 corrected iter #57) ✓
Billit-Redis network — Both networks connected, ~3 err/h (F-171, Sim-024 impl) ✓
Pulse CORS origin:true — Whitelist deployed, evil.com rejected (F-174) ✓

1. Billit API key scopes — RESOLVED ✓
2. SSH password auth — RESOLVED disabled on both servers ✓
3. serialize-javascript RCE — Remote code execution in s60-mail, badwolf (F-117)
4. Adminer publicly exposed on merlin — DB login on 6+ subdomains (F-101)
5. Neo4j default password — changeme123, 3,857 nodes (F-111)
6. s60-redis weak password — changeme123, 5 services (F-099)
7. 105 npm vulnerabilities (44 HIGH) — NestJS v10 (F-117)
8. Pulse CORS origin:true — RESOLVED whitelist deployed (F-174) ✓