Queue Observability Snapshot — 2026-06-07
Queue Depth
| State | Count | Notes |
|---|
| done | 832 | Healthy throughput |
| active | 65 | Normal load |
| blocked | 2 | Clean — no stale items >7d |
| queued | 4 | Low backlog |
Throughput (last 24h)
- 106 tasks completed — strong processing rate
- No significant throughput decline from prior days
DLQ Signals (failure signatures in done/ last 24h)
| Category | Count | Severity | Notes |
|---|
| API connection failures | 9 | MEDIUM | Transient infra — within normal range |
| HTTP 500 errors | 0 | ✅ CLEAN | No backend instability detected |
| Timeouts | 8 | LOW-MED | Some latency but not spiking |
| Tool missing (exit 127) | 12 | ⚠️ ELEVATED | Worker environment gaps — kubectl/binaries not in PATH |
| Unknown failure tags | 0 | ✅ CLEAN | Classifier working correctly since May 4 fix |
Stuck Tasks (>6h idle in active/)
- 3 stale tasks >72h:
cronjob-remediation, cluster-fix-calliope-kanban-bug, 4a346f5d-vt-004-mastodon
- 7 tasks between 1–3 days idle — likely completed by another run or superseded
Active Project Cards (queue/)
| Card | Priority | Status |
|---|
| queue-baseline-phase-0 | URGENT → RESOLVED | Completed May 4 |
| queue-feeder-cron | HIGH | Operational — no gaps detected |
| fix-recurring-unknown | HIGH → RESOLVED | Fixed May 6, zero unknown tags today |
| schema-drift-20260502 | MEDIUM | Queued, awaiting processing |
| project-wiki-lint-daily | LOW | Blocked (coaching.md — 1d old) |
Metrics Emitter Status
queue-baseline-metrics.jsonl: 450 lines, last modified 4.1h ago- File is still growing — emitter operational
- Post-Phase-0 stub-only tick pattern persists (known limitation)
Kanban Board: queue-ops
- 63 tasks tracked across board
- No duplicate daily session for today detected
- Previous 2026-06-07 task was from overlapping cron run — marked complete
Summary Assessment
| Metric | Status | Trend |
|---|
| Throughput | ✅ Healthy (106/24h) | Stable |
| HTTP 500 | ✅ Clean (0) | Improved (was 13 on Jun 5) |
| Tool missing | ⚠️ Elevated (12) | Persistent — PATH fix needed |
| API failures | MEDIUM (9) | Normal baseline (~8-10/day) |
| Unknown tags | ✅ Clean (0) | Fixed since May 6 |
| Stuck tasks | LOW (3 stale) | Cleanup recommended |
Action Items for pvs
- Tool missing (exit 127): 12 occurrences suggest systematic worker PATH gap. Consider adding kubectl and common CLI tools to the cron worker container image.
- Stale active cards: 3 items >72h idle in active/ should be reviewed — either archived or unblocked.
- Blocked coaching card:
project-wiki-lint-daily-coaching.md is 1 day old in blocked/ — unblock if the blocker resolved.