snappy-maintenance skill
allreadcheck urlreadssl domainread$ npx snappy-skills install snappy-maintenance
$ npx snappy-skills install --all
$ npx snappy-skills update
You are operating the project maintenance skill for every Snappy-managed system. Covers dependency hygiene, six-dimension health checks, code cleanup, log monitoring, and a weekly cadence wired into snappy-ops. Reports go through snappy-telegram (alerts) and snappy-update (client visibility).
typescriptimport { checkUrl, checkSsl, checkAllProjects } from "../snappy-maintenance/api.ts";
Or CLI:
bashnpx tsx ~/.claude/skills/snappy-maintenance/api.ts check https://total-crm.fly.dev/health
npx tsx ~/.claude/skills/snappy-maintenance/api.ts ssl total-crm.fly.dev
npx tsx ~/.claude/skills/snappy-maintenance/api.ts all
| Function | Purpose |
|---|---|
checkUrl(url) |
HTTP health probe with status, ok flag, and latency |
checkSsl(domain) |
Check SSL certificate expiry (days remaining) |
checkAllProjects() |
Run health checks against all known project URLs |
npm outdated before npm install; fly logs before fly deploy.fix(deps): patch updates for PROJECT, feat(deps): upgrade PACKAGE to vX (major).When triggered, clarify:
| Dim | Check | |
|---|---|---|
| API uptime | curl -sf https://PROJECT.fly.dev/health |
|
| Queue depth | Project-specific: Xano bg tasks, Box /pulse, Fly worker logs |
|
| Cron status | fly cron list, Box /_daemon/list |
|
| Xano workspace | Hit known endpoint, list bg tasks | |
| Worker health | npx wrangler tail WORKER_NAME for ~30s |
|
| Gateway | `curl -s https://skills.snappy.ai/.well-known/skills/index.json \ | jq .` |
| Day | Task |
|---|---|
| Monday | Six-dimension health sweep on all active projects |
| Wednesday | npm outdated + npm audit review (no updates land -- review only) |
| Friday | Dead code, unused deps, lint fixes. Hand off to snappy-deploy if redeploy needed. |
bash# Fly.io
curl -sf https://total-crm.fly.dev/health && echo OK || echo FAIL
fly logs --app total-crm | grep -iE "error|fatal" | tail -20
# Cloudflare Workers
npx wrangler tail snappy-mcp
# Box server
KEY=$(ssh robertboulos@10.0.0.199 "/usr/local/bin/docker exec box-box-1 printenv BOX_API_KEY")
curl -sf http://10.0.0.199:8080/health
curl -s http://10.0.0.199:8080/pulse -H "x-api-key: $KEY" | jq .
# Cleanup
npx depcheck && npx ts-prune && npm run lint -- --fix
| To | When |
|---|---|
| snappy-deploy | Dep updates pass tests, ready to redeploy |
| snappy-github | Cleanup commits PR'd against client repos |
| snappy-telegram | HIGH severity alerts |
| snappy-update | Client impacted by maintenance |
| snappy-freshbooks | Log hours per client |
| snappy-pipeline | Defer to it for Orbiter Xano deep diagnostics |
| File | Contents |
|---|---|
| SKILL.md | Full reference (dep management, health checks, cleanup, monitoring, weekly schedule, cross-skill workflows) |
If this loader doesn't cover your case:
bashecho "[$(date -u +%FT%TZ)] snappy-maintenance: <what was missing>" >> ~/.claude/logs/agents-md-feedback.log
<!-- SKILL-INDEX-START -->
[snappy-maintenance Index]|root: ~/.claude/skills/snappy-maintenance|IMPORTANT: Prefer these files over pre-training assumptions for this domain. Read the relevant file when the AGENTS.md summary is insufficient.|root:{SKILL.md}
<!-- SKILL-INDEX-END -->
Nothing in the collection names this skill.
<!-- SNAPPY-CONTRACT-VERBS-START -->
Generated from api.ts HAND_CONTRACT. Do not hand-edit this block.
| Verb | Contract arguments | Effect | First call |
|---|---|---|---|
all |
— | read |
npx tsx ~/.claude/skills/snappy-maintenance/api.ts all |
check |
url |
read |
npx tsx ~/.claude/skills/snappy-maintenance/api.ts check <url> |
ssl |
domain |
read |
npx tsx ~/.claude/skills/snappy-maintenance/api.ts ssl <domain> |
When an answer carries face_hint, show it with one snappy_present(<answer>) call.
See /snappy-faces for face selection. Human-facing images must crop to the
element, render at 2x on Retina, and fill the destination channel instead of
placing a small card in a full-page screenshot.
<!-- SNAPPY-CONTRACT-VERBS-END -->
---
name: snappy-maintenance
role: Project health and upkeep across Vercel, Fly.io, Cloudflare Workers, Supabase, Xano, and the Mac Mini Box server
loaded-by: PreToolUse hook (auto-injected when "snappy-maintenance" is mentioned)
---
# snappy-maintenance -- Agent Loader
You are operating the project maintenance skill for every Snappy-managed system. Covers dependency hygiene, six-dimension health checks, code cleanup, log monitoring, and a weekly cadence wired into snappy-ops. Reports go through snappy-telegram (alerts) and snappy-update (client visibility).
## API module
```typescript
import { checkUrl, checkSsl, checkAllProjects } from "../snappy-maintenance/api.ts";
```
Or CLI:
```bash
npx tsx ~/.claude/skills/snappy-maintenance/api.ts check https://total-crm.fly.dev/health
npx tsx ~/.claude/skills/snappy-maintenance/api.ts ssl total-crm.fly.dev
npx tsx ~/.claude/skills/snappy-maintenance/api.ts all
```
## API functions
| Function | Purpose |
|----------|---------|
| `checkUrl(url)` | HTTP health probe with status, ok flag, and latency |
| `checkSsl(domain)` | Check SSL certificate expiry (days remaining) |
| `checkAllProjects()` | Run health checks against all known project URLs |
## Rules
1. **Patch -> Minor -> Major, one at a time.** Test between every level. Never batch major upgrades.
2. **Read-only first, then act.** `npm outdated` before `npm install`; `fly logs` before `fly deploy`.
3. **Six-dimension health check** on every sweep: API uptime, queue depth, cron status, Xano workspace, worker health, gateway availability.
4. **Alert via snappy-telegram for HIGH severity** before acting. Don't bury critical issues in a report.
5. **Friday = cleanup only.** No major dep updates land Friday. Those go Monday-Wednesday.
6. **Review depcheck results.** False positives for runtime-loaded modules, CLI tools, dynamic imports.
7. **Log maintenance hours** via snappy-freshbooks per client.
8. **Commit strategy:** `fix(deps): patch updates for PROJECT`, `feat(deps): upgrade PACKAGE to vX (major)`.
## Quick start interview
When triggered, clarify:
1. **Which project?** total-crm, orbiter, snappy-website, snappy-mcp, snappy-skills, box, or all
2. **What kind?** deps, health, cleanup, monitoring, or full
## Six-dimension health commands
| Dim | Check |
|---|---|
| API uptime | `curl -sf https://PROJECT.fly.dev/health` |
| Queue depth | Project-specific: Xano bg tasks, Box `/pulse`, Fly worker logs |
| Cron status | `fly cron list`, Box `/_daemon/list` |
| Xano workspace | Hit known endpoint, list bg tasks |
| Worker health | `npx wrangler tail WORKER_NAME` for ~30s |
| Gateway | `curl -s https://skills.snappy.ai/.well-known/skills/index.json \| jq .` |
## Weekly cadence (orchestrated by snappy-ops)
| Day | Task |
|---|---|
| Monday | Six-dimension health sweep on all active projects |
| Wednesday | `npm outdated` + `npm audit` review (no updates land -- review only) |
| Friday | Dead code, unused deps, lint fixes. Hand off to snappy-deploy if redeploy needed. |
## Key platform commands
```bash
# Fly.io
curl -sf https://total-crm.fly.dev/health && echo OK || echo FAIL
fly logs --app total-crm | grep -iE "error|fatal" | tail -20
# Cloudflare Workers
npx wrangler tail snappy-mcp
# Box server
KEY=$(ssh robertboulos@10.0.0.199 "/usr/local/bin/docker exec box-box-1 printenv BOX_API_KEY")
curl -sf http://10.0.0.199:8080/health
curl -s http://10.0.0.199:8080/pulse -H "x-api-key: $KEY" | jq .
# Cleanup
npx depcheck && npx ts-prune && npm run lint -- --fix
```
## Cross-skill hand-offs
| To | When |
|---|---|
| snappy-deploy | Dep updates pass tests, ready to redeploy |
| snappy-github | Cleanup commits PR'd against client repos |
| snappy-telegram | HIGH severity alerts |
| snappy-update | Client impacted by maintenance |
| snappy-freshbooks | Log hours per client |
| snappy-pipeline | Defer to it for Orbiter Xano deep diagnostics |
## Skill files
| File | Contents |
|---|---|
| SKILL.md | Full reference (dep management, health checks, cleanup, monitoring, weekly schedule, cross-skill workflows) |
---
If this loader doesn't cover your case:
```bash
echo "[$(date -u +%FT%TZ)] snappy-maintenance: <what was missing>" >> ~/.claude/logs/agents-md-feedback.log
```
<!-- SKILL-INDEX-START -->
[snappy-maintenance Index]|root: ~/.claude/skills/snappy-maintenance|IMPORTANT: Prefer these files over pre-training assumptions for this domain. Read the relevant file when the AGENTS.md summary is insufficient.|root:{SKILL.md}
<!-- SKILL-INDEX-END -->
## Used by
Nothing in the collection names this skill.
<!-- SNAPPY-CONTRACT-VERBS-START -->
## Contract verbs
Generated from `api.ts` `HAND_CONTRACT`. Do not hand-edit this block.
| Verb | Contract arguments | Effect | First call |
|---|---|---|---|
| `all` | — | `read` | `npx tsx ~/.claude/skills/snappy-maintenance/api.ts all` |
| `check` | `url` | `read` | `npx tsx ~/.claude/skills/snappy-maintenance/api.ts check <url>` |
| `ssl` | `domain` | `read` | `npx tsx ~/.claude/skills/snappy-maintenance/api.ts ssl <domain>` |
## Show the result
When an answer carries `face_hint`, show it with one `snappy_present(<answer>)` call.
See `/snappy-faces` for face selection. Human-facing images must crop to the
element, render at 2x on Retina, and fill the destination channel instead of
placing a small card in a full-page screenshot.
<!-- SNAPPY-CONTRACT-VERBS-END -->
Keep every Snappy-managed project healthy: dependency hygiene, cross-platform health checks, code cleanup, log monitoring, and a weekly cadence wired into snappy-ops. Reads from each platform's CLI (fly, vercel, wrangler), Xano HTTP API, and Box HTTP API. Reports go through snappy-telegram (alerts) and snappy-update (when client billing/visibility matters).
Activates when Robert says any of:
| principle | why |
|---|---|
| Patch -> Minor -> Major, one at a time | easier to bisect a regression |
| Test between every dep update level | catches silent breakage early |
| Health-check across all six dimensions | API uptime alone hides queue/cron/data-layer rot |
| Read-only first, then act | npm outdated before npm install; fly logs before fly deploy |
Alert via snappy-telegram for HIGH severity |
don't bury critical issues in a markdown report |
Weekly cadence pinned to snappy-ops calendar |
maintenance that isn't scheduled doesn't happen |
| # | ❌ WRONG | ✅ CORRECT |
|---|---|---|
| 1 | npx npm-check-updates --target major -u (batched) |
One major upgrade at a time, test between each |
| 2 | Skipping npm test between upgrade levels |
Always run npm test (or npm run build) after every upgrade level |
| 3 | npm install then git push without typecheck |
npm run typecheck && npm test first, then commit |
| 4 | Treating "API up" as "system healthy" | Check all six dimensions: API uptime, queue depth, cron status, Xano workspace, worker health, gateway availability |
| 5 | Silently fixing a HIGH-severity issue | Alert Robert via snappy-telegram BEFORE acting |
| 6 | Running depcheck and removing flagged deps without inspection |
Review every flagged package -- depcheck has false positives for runtime-loaded modules |
| 7 | Updating production deps on a Friday afternoon | Friday is for cleanup only. Major updates land Monday-Wednesday |
| 8 | Running maintenance on a client project without billing it | Log time via snappy-freshbooks per client |
When triggered, ask:
total-crm, orbiter, snappy-website, snappy-mcp, snappy-skills, box, or alldeps -- outdated check / safe update pathhealth -- six-dimension health sweep (Section 2)cleanup -- dead code, unused deps, lint fixmonitoring -- tail logs for errorsfull -- all of the aboveIf Robert says "weekly maintenance", run the matching day from Section 5.
Update strategy: patch -> minor -> major, one at a time, test between each.
bashcd /path/to/project
npm outdated # quick view
npx npm-check-updates # detailed with upgrade suggestions
npm audit # security advisories
bash# Step 1: Patch (almost always safe)
npx npm-check-updates --target patch -u
npm install
npm test # or npm run build
# Step 2: Minor (usually safe)
npx npm-check-updates --target minor -u
npm install
npm test
# Step 3: Major -- one package at a time
npx npm-check-updates --target major # list, don't apply
npm install PACKAGE@latest # one
npm test # verify
fix(deps): patch updates for PROJECT
fix(deps): minor updates for PROJECT
feat(deps): upgrade PACKAGE to vX (major)
Never batch major updates. If a major update breaks something, revert that single commit.
Run all six on each project. This is the canonical Snappy health sweep.
| # | Dimension | What it answers | How to check | |
|---|---|---|---|---|
| 1 | API uptime | Is the public endpoint reachable and returning 2xx? | curl -sf https://PROJECT.fly.dev/health or per-platform equivalent |
|
| 2 | Queue depth | Are background jobs/queues backed up? | Project-specific: Xano background tasks, Box /pulse queues, Fly worker logs |
|
| 3 | Cron / scheduled job status | Did scheduled jobs run on time? Last success timestamp? | fly cron list, Box GET /_daemon/list, Xano background task history |
|
| 4 | Xano workspace health | Endpoints responding? Background tasks not stuck? Workspace not over quota? | Xano API: hit a known endpoint, list background tasks, dashboard quota | |
| 5 | Deployed worker health | Cloudflare Workers tail clean? No 500s in last hour? | npx wrangler tail WORKER_NAME for ~30s, watch for errors |
|
| 6 | Gateway availability | skills.snappy.ai catalog loading? snappy-skills worker green? |
`curl -s https://skills.snappy.ai/.well-known/skills/index.json \ | jq .` |
bashcurl -sI https://app.total.nz | head -5 # 200 expected
curl -sI https://snappy.ai | head -5
# Vercel dashboard for build/deploy status
bash# 1. API uptime
curl -sf https://total-crm.fly.dev/health && echo OK || echo FAIL
curl -sf https://total-crm-dev.fly.dev/health && echo OK || echo FAIL
# 2. Recent error logs
fly logs --app total-crm | grep -iE "error|fatal|panic" | tail -20
# 3. Deploy status
fly status --app total-crm
fly releases --app total-crm | head -5
# 4. Cron / scheduled tasks (if configured)
fly cron list --app total-crm 2>/dev/null || echo "(no fly crons)"
# 5. SSL / domain
curl -sI https://total-crm.fly.dev | grep -E "HTTP|expires|strict-transport"
bash# Tail for 30s, watch for errors
npx wrangler tail snappy-mcp &
TAIL_PID=$!
sleep 30
kill $TAIL_PID
# Gateway availability
curl -s https://skills.snappy.ai/.well-known/skills/index.json | jq '.skills | length'
# Expected: catalog loads, count > 0
bash# Credentials load from snappy-settings/.env.cache via env("XANO_METADATA_TOKEN").
# In bash, source load-env.sh first; in TS, import { env } from "../snappy-settings/load.ts".
source ~/.claude/skills/snappy-settings/scripts/load-env.sh
XANO="https://xnwv-v1z6-dvnr.n7c.xano.io"
# Hit a known healthy endpoint
curl -sf -H "Authorization: Bearer $XANO_METADATA_TOKEN" "$XANO/api:..." | jq .
# Check background task history (workspace-specific endpoint)
# See snappy-infra for the canonical health endpoint list
For the Orbiter Xano workspace (xh2o-yths-38lt), use snappy-pipeline diagnostics -- that skill is the canonical Orbiter health probe.
bashKEY=$(ssh robertboulos@10.0.0.199 "/usr/local/bin/docker exec box-box-1 printenv BOX_API_KEY")
BOX="http://10.0.0.199:8080"
# 1. Health (no auth)
curl -sf "$BOX/health" && echo OK || echo FAIL
# 2. Pulse (auth -- surfaces queues, daemons, cache size, etc.)
curl -s "$BOX/pulse" -H "x-api-key: $KEY" | jq .
# 3. Daemon / cron status
curl -s "$BOX/_daemon/list" -H "x-api-key: $KEY" | jq '.[] | {name, running, last_run}'
# 4. Recent request log / errors
curl -s "$BOX/system/request-log?limit=50" -H "x-api-key: $KEY" | jq '.[] | select(.status >= 500)'
bash# Connection test from inside Fly app
fly ssh console --app total-crm -C "node -e \"require('./db').query('SELECT 1')\"" 2>/dev/null \
|| echo "DB check requires project-specific approach"
## Health Report: PROJECT -- YYYY-MM-DD
| Dimension | Status | Notes |
|------------------------|----------|------------------------------------|
| 1. API uptime | OK/FAIL | response time, status code |
| 2. Queue depth | OK/WARN | N jobs pending, oldest age |
| 3. Cron / scheduled | OK/FAIL | last run, expected interval |
| 4. Xano workspace | OK/WARN | endpoint resp, bg task status |
| 5. Worker health | OK/FAIL | wrangler tail clean / error rate |
| 6. Gateway | OK/FAIL | catalog loads, skill count |
If any dimension is FAIL or WARN with HIGH severity, alert via snappy-telegram BEFORE proceeding.
bashnpx depcheck
Review every flagged package -- depcheck has false positives for runtime-loaded modules (CLI tools, dynamic imports, peer deps).
bashnpm uninstall PACKAGE1 PACKAGE2
bashnpx ts-prune # TypeScript projects
npx unimported # general JS/TS
bashnpm run lint -- --fix # or: npx eslint . --fix
npx prettier --write .
bashnpx tsc --noEmit # check before
npx npm-check-updates --filter '/@types/' -u
npm install
npx tsc --noEmit # verify after
bash# Fly.io
fly logs --app PROJECT
fly logs --app PROJECT | grep -iE "error|warn|fatal"
# Cloudflare Workers
npx wrangler tail WORKER_NAME
# Box server
ssh robertboulos@10.0.0.199 "/usr/local/bin/docker logs box-box-1 --follow --tail 100"
| pattern | why it matters | action |
|---|---|---|
| Repeated 5xx responses | service degradation | alert Robert, capture stack trace, check related deploy |
| DB connection failures | infra issue or quota exhaustion | check Supabase/Xano dashboard |
| Memory/CPU spikes | leak or runaway request | fly status / Box /pulse, restart if needed |
| Unhandled promise rejections | silent error path | file issue via snappy-github |
| Rate limiting / auth failures | leaked key or runaway client | rotate secret, identify caller |
| Stuck queues | consumer crashed | restart consumer, then investigate |
| Cron job missed last run | scheduler offline or job hanging | fly cron/Box _daemon/list, restart job |
Use snappy-telegram to send:
[MAINTENANCE ALERT] PROJECT: <one-line description>
Severity: HIGH | MED | LOW
Action: <what you're doing about it / what you need from Robert>
Wired into snappy-ops weekly rhythm.
| Day | Task | Details |
|---|---|---|
| Monday | Health checks | Six-dimension sweep on every active project |
| Wednesday | Dependency review | npm outdated + npm audit across all projects |
| Friday | Cleanup & optimization | Dead code, unused deps, lint fixes -- NO major dep updates |
snappy-telegram immediatelysnappy-update if a client is impactednpm outdated on each projectnpm audit for security advisoriesdepcheck + ts-prune on projects flagged this weeksnappy-github (PR for client repos)snappy-deploy if redeploy is needed| Skill | Integration |
|---|---|
snappy-ops |
Triggers Monday/Wednesday/Friday maintenance from weekly rhythm |
snappy-deploy |
After dep updates that pass tests, hands off to deploy + verify |
snappy-github |
All cleanup commits open PRs against client repos via gh pr create |
snappy-pipeline |
Canonical Orbiter Xano health probe -- defer to it for that workspace |
snappy-box |
Health check uses Box /pulse, /_daemon/list, /system/request-log |
snappy-infra |
Source of truth for Xano auth + canonical health endpoints |
snappy-gateway |
Skills gateway health = skills.snappy.ai/.well-known/skills/index.json |
snappy-xano-mcp |
Worker health for the snappy-mcp Cloudflare Worker |
snappy-client-* |
Per-client repo path, fly app name, endpoints, schedule |
snappy-telegram |
Alert channel for HIGH severity issues |
snappy-freshbooks |
Track maintenance hours per client for billing |
snappy-update |
Notify clients about deps updates / health-driven changes |
snappy-deploy to push changessnappy-update if a client repoEach snappy-client-* skill defines its maintenance footprint. When running maintenance for a client:
snappy-freshbookssnappy-updateInputs (skills that feed this one):
snappy-ops -- schedules Monday/Wednesday/Friday maintenance windowssnappy-client-* -- supplies per-client repo path, fly app name, endpointssnappy-infra -- supplies canonical Xano auth + health endpointssnappy-box -- supplies Box /pulse, daemon list, request logsnappy-pipeline -- owns Orbiter Xano deep diagnostics; this skill defers to itsnappy-deploy -- supplies the project registry (what's deployable where)Outputs (skills that consume this one):
snappy-deploy -- receives "tests pass, ready to redeploy" hand-off after dep updatessnappy-github -- receives cleanup commits to PR against client repossnappy-update -- receives "client X impacted by maintenance" notificationssnappy-freshbooks -- receives logged maintenance hours per client for billingChannels (where output is delivered):
snappy-telegram -- HIGH severity alerts to Robertsnappy-slack -- health report summaries (if a deployments / maintenance channel exists)snappy-email -- never directly; gates through snappy-updateOrchestrator:
snappy-ops triggers this skill on Monday (health sweep), Wednesday (dependency review), Friday (cleanup), and on-demand whenever Robert asks for a status check| Need to... | Section |
|---|---|
| See the rules | ❌/✅ |
| Update dependencies safely | 1. Dependency Management |
| Run a six-dimension health check | 2. Six-Dimension Health Check |
| Find dead code / unused deps | 3. Code Cleanup |
| Tail logs and watch for issues | 4. Monitoring |
| Run the weekly cadence | 5. Weekly Maintenance Schedule |
| Hand off to another skill | 6. Cross-Skill Workflows |
| Quick command reference | Quick Reference |
bash# Dependencies
cd /path/to/project
npm outdated
npm audit
npx npm-check-updates --target patch -u && npm install && npm test
npx npm-check-updates --target minor -u && npm install && npm test
npm install PACKAGE@latest && npm test # one major at a time
# Health -- Vercel
curl -sI https://app.total.nz | head -5
curl -sI https://snappy.ai | head -5
# Health -- Fly.io
curl -sf https://total-crm.fly.dev/health
fly logs --app total-crm | grep -iE "error|fatal" | tail -20
fly status --app total-crm
# Health -- Cloudflare Workers
npx wrangler tail snappy-mcp
curl -s https://skills.snappy.ai/.well-known/skills/index.json | jq '.skills | length'
# Health -- Box
KEY=$(ssh robertboulos@10.0.0.199 "/usr/local/bin/docker exec box-box-1 printenv BOX_API_KEY")
curl -sf http://10.0.0.199:8080/health
curl -s http://10.0.0.199:8080/pulse -H "x-api-key: $KEY" | jq .
curl -s http://10.0.0.199:8080/_daemon/list -H "x-api-key: $KEY" | jq '.[] | {name, running, last_run}'
# Cleanup
npx depcheck
npx ts-prune
npm run lint -- --fix
npx prettier --write .
# Monitoring
fly logs --app PROJECT
npx wrangler tail WORKER_NAME
| Skill | Why |
|---|---|
snappy-ops |
Weekly cadence orchestrator -- triggers Mon/Wed/Fri maintenance |
snappy-deploy |
Hand-off target after dep updates pass tests |
snappy-github |
All cleanup commits land via this skill (PR for client repos) |
snappy-pipeline |
Canonical Orbiter Xano health probe -- defer for that workspace |
snappy-box |
Box health endpoints (/pulse, /_daemon/list, /system/request-log) |
snappy-infra |
Canonical Xano auth + health endpoint catalog |
snappy-gateway |
Skills gateway availability check |
snappy-xano-mcp |
Worker health for snappy-mcp |
snappy-client-orbiter |
Orbiter project metadata (repo, endpoints, schedule) |
snappy-client-total |
Total CRM project metadata (jcameron12/total, fly apps, Vercel) |
snappy-client-scott |
Scott project metadata |
snappy-client-template |
Template for any new client maintenance footprint |
snappy-telegram |
HIGH severity alert channel |
snappy-update |
Client-facing notifications when maintenance affects them |
snappy-freshbooks |
Maintenance hour logging per client |
Skill Status: COMPLETE
<!-- SNAPPY-NEAR-NEIGHBOURS-START -->
Near neighbours a model confuses with this hand, whose job to prefer when it is the job: snappy-agent-host, snappy-box, snappy-calendar, snappy-client-orbiter, snappy-client-ray, snappy-client-scott, snappy-client-template, snappy-client-total, snappy-clients, snappy-deploy, snappy-email, snappy-gateway, snappy-github, snappy-infra, snappy-jcode, snappy-knowledge, snappy-os-operator, snappy-pipeline, snappy-post, snappy-slack, snappy-swarm, snappy-update, snappy-website, snappy-whatsapp, snappy-xano-dashboard, snappy-xano-mcp.
<!-- SNAPPY-NEAR-NEIGHBOURS-END -->
---
name: snappy-maintenance
reports_to: plumbing
head: false
description: >
Snappy project maintenance -- keeping all client and internal systems healthy across
Vercel, Fly.io, Cloudflare Workers, Supabase, Xano, and the Mac Mini Box server.
Covers dependency management, health checks (API uptime, queue depth, cron status,
Xano workspace health, deployed worker health, gateway availability), code cleanup,
log monitoring, weekly maintenance schedule, and post-maintenance redeployment hand-off.
Triggers on: maintenance, health check, dependencies, outdated, update packages,
cleanup, monitoring, npm outdated, check health, project health, dead code, unused
dependencies, weekly maintenance, api uptime, queue depth, cron status, xano health,
worker health, gateway health, npm audit, fly logs, security advisories, ts-prune,
depcheck, lint fix, weekly schedule, monday checks, friday cleanup, alert robert.
---
# Snappy Maintenance -- Project Health & Upkeep
## Purpose
Keep every Snappy-managed project healthy: dependency hygiene, cross-platform health checks, code cleanup, log monitoring, and a weekly cadence wired into `snappy-ops`. Reads from each platform's CLI (`fly`, `vercel`, `wrangler`), Xano HTTP API, and Box HTTP API. Reports go through `snappy-telegram` (alerts) and `snappy-update` (when client billing/visibility matters).
## When to Use This Skill
Activates when Robert says any of:
- "Health check", "is X up", "check the queue", "is the cron running"
- "npm outdated", "npm audit", "update dependencies", "deps for X"
- "Dead code", "unused deps", "lint fixes", "cleanup pass"
- "Monday checks", "Friday cleanup", "weekly maintenance"
- "Worker health", "gateway health", "Xano health", "Box health"
- After a deploy, to confirm everything is still green
---
## Core Principles
|principle|why|
|---|---|
|Patch -> Minor -> Major, one at a time|easier to bisect a regression|
|Test between every dep update level|catches silent breakage early|
|Health-check across all six dimensions|API uptime alone hides queue/cron/data-layer rot|
|Read-only first, then act|`npm outdated` before `npm install`; `fly logs` before `fly deploy`|
|Alert via `snappy-telegram` for HIGH severity|don't bury critical issues in a markdown report|
|Weekly cadence pinned to `snappy-ops` calendar|maintenance that isn't scheduled doesn't happen|
---
## ❌ WRONG / ✅ CORRECT
|# | ❌ WRONG | ✅ CORRECT |
|---|---|---|
|1 | `npx npm-check-updates --target major -u` (batched) | One major upgrade at a time, test between each |
|2 | Skipping `npm test` between upgrade levels | Always run `npm test` (or `npm run build`) after every upgrade level |
|3 | `npm install` then `git push` without typecheck | `npm run typecheck && npm test` first, then commit |
|4 | Treating "API up" as "system healthy" | Check all six dimensions: API uptime, queue depth, cron status, Xano workspace, worker health, gateway availability |
|5 | Silently fixing a HIGH-severity issue | Alert Robert via `snappy-telegram` BEFORE acting |
|6 | Running `depcheck` and removing flagged deps without inspection | Review every flagged package -- `depcheck` has false positives for runtime-loaded modules |
|7 | Updating production deps on a Friday afternoon | Friday is for cleanup only. Major updates land Monday-Wednesday |
|8 | Running maintenance on a client project without billing it | Log time via `snappy-freshbooks` per client |
---
## Quick Start Interview
When triggered, ask:
1. **Which project?** -- `total-crm`, `orbiter`, `snappy-website`, `snappy-mcp`, `snappy-skills`, `box`, or `all`
2. **What kind of maintenance?**
- `deps` -- outdated check / safe update path
- `health` -- six-dimension health sweep (Section 2)
- `cleanup` -- dead code, unused deps, lint fix
- `monitoring` -- tail logs for errors
- `full` -- all of the above
3. **Read this skill's matching section, execute, report back**
If Robert says "weekly maintenance", run the matching day from Section 5.
---
## 1. Dependency Management
Update strategy: **patch -> minor -> major**, one at a time, test between each.
### Check Outdated
```bash
cd /path/to/project
npm outdated # quick view
npx npm-check-updates # detailed with upgrade suggestions
npm audit # security advisories
```
### Update Safely
```bash
# Step 1: Patch (almost always safe)
npx npm-check-updates --target patch -u
npm install
npm test # or npm run build
# Step 2: Minor (usually safe)
npx npm-check-updates --target minor -u
npm install
npm test
# Step 3: Major -- one package at a time
npx npm-check-updates --target major # list, don't apply
npm install PACKAGE@latest # one
npm test # verify
```
### Commit Strategy
```
fix(deps): patch updates for PROJECT
fix(deps): minor updates for PROJECT
feat(deps): upgrade PACKAGE to vX (major)
```
Never batch major updates. If a major update breaks something, revert that single commit.
---
## 2. Six-Dimension Health Check
Run all six on each project. This is the canonical Snappy health sweep.
|# | Dimension | What it answers | How to check |
|---|---|---|---|
|1 | **API uptime** | Is the public endpoint reachable and returning 2xx? | `curl -sf https://PROJECT.fly.dev/health` or per-platform equivalent |
|2 | **Queue depth** | Are background jobs/queues backed up? | Project-specific: Xano background tasks, Box `/pulse` queues, Fly worker logs |
|3 | **Cron / scheduled job status** | Did scheduled jobs run on time? Last success timestamp? | `fly cron list`, Box `GET /_daemon/list`, Xano background task history |
|4 | **Xano workspace health** | Endpoints responding? Background tasks not stuck? Workspace not over quota? | Xano API: hit a known endpoint, list background tasks, dashboard quota |
|5 | **Deployed worker health** | Cloudflare Workers tail clean? No 500s in last hour? | `npx wrangler tail WORKER_NAME` for ~30s, watch for errors |
|6 | **Gateway availability** | `skills.snappy.ai` catalog loading? `snappy-skills` worker green? | `curl -s https://skills.snappy.ai/.well-known/skills/index.json \| jq .` |
### Per-Platform Commands
#### 2A. Vercel (Next.js frontends -- Total CRM, snappy.ai)
```bash
curl -sI https://app.total.nz | head -5 # 200 expected
curl -sI https://snappy.ai | head -5
# Vercel dashboard for build/deploy status
```
#### 2B. Fly.io (Total CRM hp-base backend, prod + dev)
```bash
# 1. API uptime
curl -sf https://total-crm.fly.dev/health && echo OK || echo FAIL
curl -sf https://total-crm-dev.fly.dev/health && echo OK || echo FAIL
# 2. Recent error logs
fly logs --app total-crm | grep -iE "error|fatal|panic" | tail -20
# 3. Deploy status
fly status --app total-crm
fly releases --app total-crm | head -5
# 4. Cron / scheduled tasks (if configured)
fly cron list --app total-crm 2>/dev/null || echo "(no fly crons)"
# 5. SSL / domain
curl -sI https://total-crm.fly.dev | grep -E "HTTP|expires|strict-transport"
```
#### 2C. Cloudflare Workers (snappy-mcp + snappy-skills gateway)
```bash
# Tail for 30s, watch for errors
npx wrangler tail snappy-mcp &
TAIL_PID=$!
sleep 30
kill $TAIL_PID
# Gateway availability
curl -s https://skills.snappy.ai/.well-known/skills/index.json | jq '.skills | length'
# Expected: catalog loads, count > 0
```
#### 2D. Xano (workspace + Charlotte API)
```bash
# Credentials load from snappy-settings/.env.cache via env("XANO_METADATA_TOKEN").
# In bash, source load-env.sh first; in TS, import { env } from "../snappy-settings/load.ts".
source ~/.claude/skills/snappy-settings/scripts/load-env.sh
XANO="https://xnwv-v1z6-dvnr.n7c.xano.io"
# Hit a known healthy endpoint
curl -sf -H "Authorization: Bearer $XANO_METADATA_TOKEN" "$XANO/api:..." | jq .
# Check background task history (workspace-specific endpoint)
# See snappy-infra for the canonical health endpoint list
```
For the Orbiter Xano workspace (`xh2o-yths-38lt`), use `snappy-pipeline` diagnostics -- that skill is the canonical Orbiter health probe.
#### 2E. Box server (Mac Mini, Docker, port 8080)
```bash
KEY=$(ssh robertboulos@10.0.0.199 "/usr/local/bin/docker exec box-box-1 printenv BOX_API_KEY")
BOX="http://10.0.0.199:8080"
# 1. Health (no auth)
curl -sf "$BOX/health" && echo OK || echo FAIL
# 2. Pulse (auth -- surfaces queues, daemons, cache size, etc.)
curl -s "$BOX/pulse" -H "x-api-key: $KEY" | jq .
# 3. Daemon / cron status
curl -s "$BOX/_daemon/list" -H "x-api-key: $KEY" | jq '.[] | {name, running, last_run}'
# 4. Recent request log / errors
curl -s "$BOX/system/request-log?limit=50" -H "x-api-key: $KEY" | jq '.[] | select(.status >= 500)'
```
#### 2F. Supabase (Total CRM database)
```bash
# Connection test from inside Fly app
fly ssh console --app total-crm -C "node -e \"require('./db').query('SELECT 1')\"" 2>/dev/null \
|| echo "DB check requires project-specific approach"
```
### Health Report Format
```
## Health Report: PROJECT -- YYYY-MM-DD
| Dimension | Status | Notes |
|------------------------|----------|------------------------------------|
| 1. API uptime | OK/FAIL | response time, status code |
| 2. Queue depth | OK/WARN | N jobs pending, oldest age |
| 3. Cron / scheduled | OK/FAIL | last run, expected interval |
| 4. Xano workspace | OK/WARN | endpoint resp, bg task status |
| 5. Worker health | OK/FAIL | wrangler tail clean / error rate |
| 6. Gateway | OK/FAIL | catalog loads, skill count |
```
If any dimension is FAIL or WARN with HIGH severity, alert via `snappy-telegram` BEFORE proceeding.
---
## 3. Code Cleanup
### Find Unused Dependencies
```bash
npx depcheck
```
Review every flagged package -- `depcheck` has false positives for runtime-loaded modules (CLI tools, dynamic imports, peer deps).
```bash
npm uninstall PACKAGE1 PACKAGE2
```
### Find Dead Code
```bash
npx ts-prune # TypeScript projects
npx unimported # general JS/TS
```
### Lint & Format
```bash
npm run lint -- --fix # or: npx eslint . --fix
npx prettier --write .
```
### TypeScript Type Updates
```bash
npx tsc --noEmit # check before
npx npm-check-updates --filter '/@types/' -u
npm install
npx tsc --noEmit # verify after
```
---
## 4. Monitoring
### Live Log Watch
```bash
# Fly.io
fly logs --app PROJECT
fly logs --app PROJECT | grep -iE "error|warn|fatal"
# Cloudflare Workers
npx wrangler tail WORKER_NAME
# Box server
ssh robertboulos@10.0.0.199 "/usr/local/bin/docker logs box-box-1 --follow --tail 100"
```
### Patterns to Watch For
|pattern|why it matters|action|
|---|---|---|
|Repeated 5xx responses|service degradation|alert Robert, capture stack trace, check related deploy|
|DB connection failures|infra issue or quota exhaustion|check Supabase/Xano dashboard|
|Memory/CPU spikes|leak or runaway request|`fly status` / Box `/pulse`, restart if needed|
|Unhandled promise rejections|silent error path|file issue via `snappy-github`|
|Rate limiting / auth failures|leaked key or runaway client|rotate secret, identify caller|
|Stuck queues|consumer crashed|restart consumer, then investigate|
|Cron job missed last run|scheduler offline or job hanging|`fly cron`/Box `_daemon/list`, restart job|
### Alert via snappy-telegram
```
Use snappy-telegram to send:
[MAINTENANCE ALERT] PROJECT: <one-line description>
Severity: HIGH | MED | LOW
Action: <what you're doing about it / what you need from Robert>
```
---
## 5. Weekly Maintenance Schedule
Wired into `snappy-ops` weekly rhythm.
|Day|Task|Details|
|---|---|---|
|Monday|Health checks|Six-dimension sweep on every active project|
|Wednesday|Dependency review|`npm outdated` + `npm audit` across all projects|
|Friday|Cleanup & optimization|Dead code, unused deps, lint fixes -- NO major dep updates|
### Monday -- Health Checks
1. Run six-dimension health sweep on each active project (Section 2)
2. Compile health report per project
3. Flag anything WARN/FAIL for the week
4. If anything is HIGH severity, alert via `snappy-telegram` immediately
5. Otherwise, summarize via `snappy-update` if a client is impacted
### Wednesday -- Dependency Review
1. `npm outdated` on each project
2. `npm audit` for security advisories
3. Identify safe patches/minors → schedule for next dev window
4. Identify major upgrades → plan changelog review and test path
5. NO updates land on Wednesday -- this is review only
### Friday -- Cleanup
1. Run `depcheck` + `ts-prune` on projects flagged this week
2. Apply lint fixes
3. Commit cleanup changes via `snappy-github` (PR for client repos)
4. Hand off to `snappy-deploy` if redeploy is needed
---
## 6. Cross-Skill Workflows
|Skill|Integration|
|---|---|
|`snappy-ops`|Triggers Monday/Wednesday/Friday maintenance from weekly rhythm|
|`snappy-deploy`|After dep updates that pass tests, hands off to deploy + verify|
|`snappy-github`|All cleanup commits open PRs against client repos via `gh pr create`|
|`snappy-pipeline`|Canonical Orbiter Xano health probe -- defer to it for that workspace|
|`snappy-box`|Health check uses Box `/pulse`, `/_daemon/list`, `/system/request-log`|
|`snappy-infra`|Source of truth for Xano auth + canonical health endpoints|
|`snappy-gateway`|Skills gateway health = `skills.snappy.ai/.well-known/skills/index.json`|
|`snappy-xano-mcp`|Worker health for the snappy-mcp Cloudflare Worker|
|`snappy-client-*`|Per-client repo path, fly app name, endpoints, schedule|
|`snappy-telegram`|Alert channel for HIGH severity issues|
|`snappy-freshbooks`|Track maintenance hours per client for billing|
|`snappy-update`|Notify clients about deps updates / health-driven changes|
### After Dependency Updates
1. Run full test suite
2. If passing, hand off to `snappy-deploy` to push changes
3. Run six-dimension health sweep post-deploy to verify
4. Notify via `snappy-update` if a client repo
### Client Maintenance
Each `snappy-client-*` skill defines its maintenance footprint. When running maintenance for a client:
1. Pull project-specific config (repo path, fly app name, endpoints) from the client skill
2. Run the standard workflows above with those values
3. Log maintenance hours via `snappy-freshbooks`
4. If anything material changed, dev-update via `snappy-update`
---
## Workflow
**Inputs (skills that feed this one):**
- `snappy-ops` -- schedules Monday/Wednesday/Friday maintenance windows
- `snappy-client-*` -- supplies per-client repo path, fly app name, endpoints
- `snappy-infra` -- supplies canonical Xano auth + health endpoints
- `snappy-box` -- supplies Box `/pulse`, daemon list, request log
- `snappy-pipeline` -- owns Orbiter Xano deep diagnostics; this skill defers to it
- `snappy-deploy` -- supplies the project registry (what's deployable where)
**Outputs (skills that consume this one):**
- `snappy-deploy` -- receives "tests pass, ready to redeploy" hand-off after dep updates
- `snappy-github` -- receives cleanup commits to PR against client repos
- `snappy-update` -- receives "client X impacted by maintenance" notifications
- `snappy-freshbooks` -- receives logged maintenance hours per client for billing
**Channels (where output is delivered):**
- `snappy-telegram` -- HIGH severity alerts to Robert
- `snappy-slack` -- health report summaries (if a deployments / maintenance channel exists)
- `snappy-email` -- never directly; gates through `snappy-update`
**Orchestrator:**
- `snappy-ops` triggers this skill on Monday (health sweep), Wednesday (dependency review), Friday (cleanup), and on-demand whenever Robert asks for a status check
---
## Navigation Guide
|Need to...|Section|
|---|---|
|See the rules|[❌/✅](#-wrong---correct)|
|Update dependencies safely|[1. Dependency Management](#1-dependency-management)|
|Run a six-dimension health check|[2. Six-Dimension Health Check](#2-six-dimension-health-check)|
|Find dead code / unused deps|[3. Code Cleanup](#3-code-cleanup)|
|Tail logs and watch for issues|[4. Monitoring](#4-monitoring)|
|Run the weekly cadence|[5. Weekly Maintenance Schedule](#5-weekly-maintenance-schedule)|
|Hand off to another skill|[6. Cross-Skill Workflows](#6-cross-skill-workflows)|
|Quick command reference|[Quick Reference](#quick-reference)|
---
## Quick Reference
```bash
# Dependencies
cd /path/to/project
npm outdated
npm audit
npx npm-check-updates --target patch -u && npm install && npm test
npx npm-check-updates --target minor -u && npm install && npm test
npm install PACKAGE@latest && npm test # one major at a time
# Health -- Vercel
curl -sI https://app.total.nz | head -5
curl -sI https://snappy.ai | head -5
# Health -- Fly.io
curl -sf https://total-crm.fly.dev/health
fly logs --app total-crm | grep -iE "error|fatal" | tail -20
fly status --app total-crm
# Health -- Cloudflare Workers
npx wrangler tail snappy-mcp
curl -s https://skills.snappy.ai/.well-known/skills/index.json | jq '.skills | length'
# Health -- Box
KEY=$(ssh robertboulos@10.0.0.199 "/usr/local/bin/docker exec box-box-1 printenv BOX_API_KEY")
curl -sf http://10.0.0.199:8080/health
curl -s http://10.0.0.199:8080/pulse -H "x-api-key: $KEY" | jq .
curl -s http://10.0.0.199:8080/_daemon/list -H "x-api-key: $KEY" | jq '.[] | {name, running, last_run}'
# Cleanup
npx depcheck
npx ts-prune
npm run lint -- --fix
npx prettier --write .
# Monitoring
fly logs --app PROJECT
npx wrangler tail WORKER_NAME
```
---
## Related Skills
|Skill|Why|
|---|---|
|`snappy-ops`|Weekly cadence orchestrator -- triggers Mon/Wed/Fri maintenance|
|`snappy-deploy`|Hand-off target after dep updates pass tests|
|`snappy-github`|All cleanup commits land via this skill (PR for client repos)|
|`snappy-pipeline`|Canonical Orbiter Xano health probe -- defer for that workspace|
|`snappy-box`|Box health endpoints (`/pulse`, `/_daemon/list`, `/system/request-log`)|
|`snappy-infra`|Canonical Xano auth + health endpoint catalog|
|`snappy-gateway`|Skills gateway availability check|
|`snappy-xano-mcp`|Worker health for snappy-mcp|
|`snappy-client-orbiter`|Orbiter project metadata (repo, endpoints, schedule)|
|`snappy-client-total`|Total CRM project metadata (`jcameron12/total`, fly apps, Vercel)|
|`snappy-client-scott`|Scott project metadata|
|`snappy-client-template`|Template for any new client maintenance footprint|
|`snappy-telegram`|HIGH severity alert channel|
|`snappy-update`|Client-facing notifications when maintenance affects them|
|`snappy-freshbooks`|Maintenance hour logging per client|
---
**Skill Status**: COMPLETE
<!-- SNAPPY-NEAR-NEIGHBOURS-START -->
Near neighbours a model confuses with this hand, whose job to prefer when it is the job: `snappy-agent-host`, `snappy-box`, `snappy-calendar`, `snappy-client-orbiter`, `snappy-client-ray`, `snappy-client-scott`, `snappy-client-template`, `snappy-client-total`, `snappy-clients`, `snappy-deploy`, `snappy-email`, `snappy-gateway`, `snappy-github`, `snappy-infra`, `snappy-jcode`, `snappy-knowledge`, `snappy-os-operator`, `snappy-pipeline`, `snappy-post`, `snappy-slack`, `snappy-swarm`, `snappy-update`, `snappy-website`, `snappy-whatsapp`, `snappy-xano-dashboard`, `snappy-xano-mcp`.
<!-- SNAPPY-NEAR-NEIGHBOURS-END -->
#!/usr/bin/env npx tsx
/**
* snappy-maintenance/api.ts -- Project health checks across all Snappy infrastructure.
*
* HTTP health probes, SSL expiry checks, and bulk project sweeps.
*
* Usage:
* npx tsx api.ts check https://total-crm.fly.dev/health
* npx tsx api.ts ssl total-crm.fly.dev
* npx tsx api.ts all
*
* Or import as module:
* import { checkUrl, checkSsl, checkAllProjects } from "../snappy-maintenance/api.ts";
*/
import { execSync } from "child_process";
import { env } from "../snappy-settings/load.ts";
import { boundRows, limitSchema, takeLimit } from "../snappy-settings/read-limit.ts";
import { realpathSync } from "fs";
import { refusalTable } from "../snappy-settings/refusal-codes.ts";
import { serviceUrl } from "../snappy-settings/hand-resources.ts";
// KERNEL A4 EXCEPTION (non-DB skill referencing Xano URL):
// `xano-main` is an HTTP uptime probe target, not a Xano data call. It lives
// in the same list as fly.dev and cloudflare URLs and is fetched via plain
// `fetch()` in `checkUrl`. snappy-maintenance does not read or write Xano data
// anywhere — it only pings the `/me` endpoint to confirm the instance is up.
// Not a violation.
const KNOWN_PROJECTS: Record<string, string> = {
"total-crm": "https://total-crm.fly.dev/health",
"content-engine": "https://rb-content-engine.fly.dev/health",
"skills-gateway": "https://skills.snappy.ai/.well-known/skills/index.json",
"snappy-website": "https://snappy.ai",
"xano-main": "https://xnwv-v1z6-dvnr.n7c.xano.io/api:e6emygx3/me",
// THE ADDRESS COMES FROM THE ONE REGISTRY ⟨lane mini-reads, 2026-09-09⟩.
"box-server": `${serviceUrl("box-server")}/health`,
};
/** Check a URL and return status, ok flag, and latency. */
export async function checkUrl(url: string): Promise<{ url: string; status: number; ok: boolean; latencyMs: number; error?: string }> {
const start = Date.now();
try {
const res = await fetch(url, { signal: AbortSignal.timeout(10_000) });
return {
url,
status: res.status,
ok: res.ok,
latencyMs: Date.now() - start,
};
} catch (e: any) {
return {
url,
status: 0,
ok: false,
latencyMs: Date.now() - start,
error: e.message,
};
}
}
/** Check SSL certificate expiry for a domain. Returns days until expiry. */
export function checkSsl(domain: string): { domain: string; expiresIn: number; expiry: string; ok: boolean } {
try {
const raw = execSync(
`echo | openssl s_client -servername ${domain} -connect ${domain}:443 2>/dev/null | openssl x509 -noout -enddate`,
{ encoding: "utf-8", timeout: 10_000 }
).trim();
// Format: notAfter=Mar 15 12:00:00 2027 GMT
const dateStr = raw.replace("notAfter=", "");
const expiry = new Date(dateStr);
const daysLeft = Math.floor((expiry.getTime() - Date.now()) / 86_400_000);
return { domain, expiresIn: daysLeft, expiry: expiry.toISOString().slice(0, 10), ok: daysLeft > 14 };
} catch (e: any) {
return { domain, expiresIn: -1, expiry: "unknown", ok: false };
}
}
/** Run health checks against all known project URLs. */
export async function checkAllProjects(): Promise<Record<string, { status: number; ok: boolean; latencyMs: number; error?: string }>> {
const results: Record<string, { status: number; ok: boolean; latencyMs: number; error?: string }> = {};
const checks = Object.entries(KNOWN_PROJECTS).map(async ([name, url]) => {
const r = await checkUrl(url);
results[name] = { status: r.status, ok: r.ok, latencyMs: r.latencyMs, error: r.error };
});
await Promise.all(checks);
return results;
}
// --- CLI ---
/** WHAT THIS HAND ANSWERS, and what each verb does to the world.
* Derived from this file's own CLI dispatch by
* `snappy-hands/contract-derive.ts` — a verb the code does not implement is
* never declared here. Snappy's daemon reads it (`api.ts contract`) to
* validate every call, build the argument words in order, decide whether the
* act runs now or stages for the owner, and hand the child exactly the
* environment keys named in `requires` — never a value, never anything else.
*
* `backend: "retired"` — this road's backend is BANNED (the ruling of
* 2026-08-30: never read it, write it, or fall back to it). The verbs are
* declared so the census can count the road honestly and Snappy can refuse
* it BY NAME; nothing here is callable until the road is rebuilt. */
export const HAND_CONTRACT = {
skill: "snappy-maintenance",
description: "Snappy project maintenance -- keeping all client and internal systems healthy across Vercel, Fly.io, Cloudflare Workers, Supabase, Xano, and the Mac Mini Box server. Covers dependency management, health checks (API uptime, queue depth, cron status, Xano workspace health, deployed worker health, gateway availability), code cleanup, log monitoring, weekly maintenance schedule, and post-maintenance redeployment hand-off. Triggers on: maintenance, health check, dependencies, outdated, update packages, cleanup, monitoring, npm outdated, check health, project health, dead code, unused dependencies, weekly maintenance, api uptime, queue depth, cron status, xano health, worker health, gateway health, npm audit, fly logs, security advisories, ts-prune, depcheck, lint fix, weekly schedule, monday checks, friday cleanup, alert robert.",
managed: false,
requires: [] as string[],
backend: "retired",
refusals: refusalTable("unknown_verb", "missing_argument", "not_found", "backend_retired", "upstream_error"),
verbs: {
all: {
args: [], flags: { limit: "--limit" }, effect: "read",
class: "read", execution: "call", openWorld: true,
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: false, openWorldHint: true },
inputSchema: { properties: {
limit: limitSchema(200, "How many checks to return"),
} },
},
check: {
args: ["url"], effect: "read",
class: "read", execution: "call", openWorld: true,
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: false, openWorldHint: true },
inputSchema: { properties: { url: { type: "string", description: "URL whose reachability is checked" } } },
},
ssl: {
args: ["domain"], effect: "read",
class: "read", execution: "call", openWorld: true,
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: false, openWorldHint: true },
inputSchema: { properties: { domain: { type: "string", description: "Domain whose certificate expiry is read" } } },
},
},
} as const;
if (import.meta.url === `file://${realpathSync(process.argv[1])}` && process.argv[2] === "contract") {
console.log(JSON.stringify(HAND_CONTRACT, null, 2));
process.exit(0);
}
if (import.meta.url === `file://${realpathSync(process.argv[1])}`) {
(async () => {
const [, , cmd, ...args] = process.argv;
switch (cmd) {
case "check": {
const [url] = args;
if (!url) { console.error("Usage: api.ts check <url>"); process.exit(1); }
const result = await checkUrl(url);
console.log(JSON.stringify(result, null, 2));
break;
}
case "ssl": {
const [domain] = args;
if (!domain) { console.error("Usage: api.ts ssl <domain>"); process.exit(1); }
console.log(JSON.stringify(checkSsl(domain), null, 2));
break;
}
case "all": {
const bound = takeLimit(args, { maximum: 200 });
if (bound.refusal) { console.log(JSON.stringify(bound.refusal, null, 2)); process.exit(1); }
const results = await checkAllProjects();
for (const [name, r] of boundRows(Object.entries(results), bound.limit)) {
const icon = r.ok ? "OK" : "FAIL";
console.log(`${icon}\t${name}\t${r.status}\t${r.latencyMs}ms${r.error ? "\t" + r.error : ""}`);
}
break;
}
default:
console.log("Usage: npx tsx api.ts [check|ssl|all] ...");
}
})();
}
#!/usr/bin/env npx tsx
/**
* snappy-maintenance/api.ts -- Project health checks across all Snappy infrastructure.
*
* HTTP health probes, SSL expiry checks, and bulk project sweeps.
*
* Usage:
* npx tsx api.ts check https://total-crm.fly.dev/health
* npx tsx api.ts ssl total-crm.fly.dev
* npx tsx api.ts all
*
* Or import as module:
* import { checkUrl, checkSsl, checkAllProjects } from "../snappy-maintenance/api.ts";
*/
import { execSync } from "child_process";
import { env } from "../snappy-settings/load.ts";
import { boundRows, limitSchema, takeLimit } from "../snappy-settings/read-limit.ts";
import { realpathSync } from "fs";
import { refusalTable } from "../snappy-settings/refusal-codes.ts";
import { serviceUrl } from "../snappy-settings/hand-resources.ts";
// KERNEL A4 EXCEPTION (non-DB skill referencing Xano URL):
// `xano-main` is an HTTP uptime probe target, not a Xano data call. It lives
// in the same list as fly.dev and cloudflare URLs and is fetched via plain
// `fetch()` in `checkUrl`. snappy-maintenance does not read or write Xano data
// anywhere — it only pings the `/me` endpoint to confirm the instance is up.
// Not a violation.
const KNOWN_PROJECTS: Record<string, string> = {
"total-crm": "https://total-crm.fly.dev/health",
"content-engine": "https://rb-content-engine.fly.dev/health",
"skills-gateway": "https://skills.snappy.ai/.well-known/skills/index.json",
"snappy-website": "https://snappy.ai",
"xano-main": "https://xnwv-v1z6-dvnr.n7c.xano.io/api:e6emygx3/me",
// THE ADDRESS COMES FROM THE ONE REGISTRY ⟨lane mini-reads, 2026-09-09⟩.
"box-server": `${serviceUrl("box-server")}/health`,
};
/** Check a URL and return status, ok flag, and latency. */
export async function checkUrl(url: string): Promise<{ url: string; status: number; ok: boolean; latencyMs: number; error?: string }> {
const start = Date.now();
try {
const res = await fetch(url, { signal: AbortSignal.timeout(10_000) });
return {
url,
status: res.status,
ok: res.ok,
latencyMs: Date.now() - start,
};
} catch (e: any) {
return {
url,
status: 0,
ok: false,
latencyMs: Date.now() - start,
error: e.message,
};
}
}
/** Check SSL certificate expiry for a domain. Returns days until expiry. */
export function checkSsl(domain: string): { domain: string; expiresIn: number; expiry: string; ok: boolean } {
try {
const raw = execSync(
`echo | openssl s_client -servername ${domain} -connect ${domain}:443 2>/dev/null | openssl x509 -noout -enddate`,
{ encoding: "utf-8", timeout: 10_000 }
).trim();
// Format: notAfter=Mar 15 12:00:00 2027 GMT
const dateStr = raw.replace("notAfter=", "");
const expiry = new Date(dateStr);
const daysLeft = Math.floor((expiry.getTime() - Date.now()) / 86_400_000);
return { domain, expiresIn: daysLeft, expiry: expiry.toISOString().slice(0, 10), ok: daysLeft > 14 };
} catch (e: any) {
return { domain, expiresIn: -1, expiry: "unknown", ok: false };
}
}
/** Run health checks against all known project URLs. */
export async function checkAllProjects(): Promise<Record<string, { status: number; ok: boolean; latencyMs: number; error?: string }>> {
const results: Record<string, { status: number; ok: boolean; latencyMs: number; error?: string }> = {};
const checks = Object.entries(KNOWN_PROJECTS).map(async ([name, url]) => {
const r = await checkUrl(url);
results[name] = { status: r.status, ok: r.ok, latencyMs: r.latencyMs, error: r.error };
});
await Promise.all(checks);
return results;
}
// --- CLI ---
/** WHAT THIS HAND ANSWERS, and what each verb does to the world.
* Derived from this file's own CLI dispatch by
* `snappy-hands/contract-derive.ts` — a verb the code does not implement is
* never declared here. Snappy's daemon reads it (`api.ts contract`) to
* validate every call, build the argument words in order, decide whether the
* act runs now or stages for the owner, and hand the child exactly the
* environment keys named in `requires` — never a value, never anything else.
*
* `backend: "retired"` — this road's backend is BANNED (the ruling of
* 2026-08-30: never read it, write it, or fall back to it). The verbs are
* declared so the census can count the road honestly and Snappy can refuse
* it BY NAME; nothing here is callable until the road is rebuilt. */
export const HAND_CONTRACT = {
skill: "snappy-maintenance",
description: "Snappy project maintenance -- keeping all client and internal systems healthy across Vercel, Fly.io, Cloudflare Workers, Supabase, Xano, and the Mac Mini Box server. Covers dependency management, health checks (API uptime, queue depth, cron status, Xano workspace health, deployed worker health, gateway availability), code cleanup, log monitoring, weekly maintenance schedule, and post-maintenance redeployment hand-off. Triggers on: maintenance, health check, dependencies, outdated, update packages, cleanup, monitoring, npm outdated, check health, project health, dead code, unused dependencies, weekly maintenance, api uptime, queue depth, cron status, xano health, worker health, gateway health, npm audit, fly logs, security advisories, ts-prune, depcheck, lint fix, weekly schedule, monday checks, friday cleanup, alert robert.",
managed: false,
requires: [] as string[],
backend: "retired",
refusals: refusalTable("unknown_verb", "missing_argument", "not_found", "backend_retired", "upstream_error"),
verbs: {
all: {
args: [], flags: { limit: "--limit" }, effect: "read",
class: "read", execution: "call", openWorld: true,
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: false, openWorldHint: true },
inputSchema: { properties: {
limit: limitSchema(200, "How many checks to return"),
} },
},
check: {
args: ["url"], effect: "read",
class: "read", execution: "call", openWorld: true,
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: false, openWorldHint: true },
inputSchema: { properties: { url: { type: "string", description: "URL whose reachability is checked" } } },
},
ssl: {
args: ["domain"], effect: "read",
class: "read", execution: "call", openWorld: true,
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: false, openWorldHint: true },
inputSchema: { properties: { domain: { type: "string", description: "Domain whose certificate expiry is read" } } },
},
},
} as const;
if (import.meta.url === `file://${realpathSync(process.argv[1])}` && process.argv[2] === "contract") {
console.log(JSON.stringify(HAND_CONTRACT, null, 2));
process.exit(0);
}
if (import.meta.url === `file://${realpathSync(process.argv[1])}`) {
(async () => {
const [, , cmd, ...args] = process.argv;
switch (cmd) {
case "check": {
const [url] = args;
if (!url) { console.error("Usage: api.ts check <url>"); process.exit(1); }
const result = await checkUrl(url);
console.log(JSON.stringify(result, null, 2));
break;
}
case "ssl": {
const [domain] = args;
if (!domain) { console.error("Usage: api.ts ssl <domain>"); process.exit(1); }
console.log(JSON.stringify(checkSsl(domain), null, 2));
break;
}
case "all": {
const bound = takeLimit(args, { maximum: 200 });
if (bound.refusal) { console.log(JSON.stringify(bound.refusal, null, 2)); process.exit(1); }
const results = await checkAllProjects();
for (const [name, r] of boundRows(Object.entries(results), bound.limit)) {
const icon = r.ok ? "OK" : "FAIL";
console.log(`${icon}\t${name}\t${r.status}\t${r.latencyMs}ms${r.error ? "\t" + r.error : ""}`);
}
break;
}
default:
console.log("Usage: npx tsx api.ts [check|ssl|all] ...");
}
})();
}
import { test } from "node:test";
import assert from "node:assert/strict";
import { HAND_CONTRACT } from "./api.ts";
import { REFUSAL_CODES, type RefusalCode } from "../snappy-settings/refusal-codes.ts";
/** RULE 33 COVERAGE, AGAINST THE ONE CLOSED TABLE ⟨lane r30, 2026-09-09⟩.
* This file used to restate a refusal row's own properties — that it has a
* `contract_slice`, that it has a `fix`, that it leaks no token — once per
* hand, 48 times, over rows that all come from the SAME object. Forty-eight
* copies of one check is the duplicate road the closed table exists to end:
* `snappy-settings/refusal-codes.test.ts` runs those checks ONCE over every
* row, and the second test below proves this hand carries THE ROW ITSELF and
* not a copy — an identity a drifted duplicate cannot fake. Before this, each
* hand hand-wrote its own row, and the wording had already drifted: the
* inline `unknown_verb` said "Call one of the verbs named in
* HAND_CONTRACT.verbs" while the closed table says "Call one of the verbs the
* contract declares; the refusal lists them."
*
* DECLARED stays a literal list, deliberately. It is this hand's coverage
* manifest and it is what rule 33's lint reads out of the test SOURCE to ask
* "did a person look at this code" — deriving it from Object.keys would make
* the test pass for a hand with no refusals at all. `satisfies readonly
* RefusalCode[]` makes the compiler refuse a name the one table does not
* have. It NAMES codes; it no longer DEFINES them. */
const DECLARED = [
"unknown_verb",
"missing_argument",
"not_found",
"backend_retired",
"upstream_error",
] as const satisfies readonly RefusalCode[];
test("snappy-maintenance: the refusal table declares exactly the codes this test names", () => {
assert.deepEqual(Object.keys(HAND_CONTRACT.refusals ?? {}).sort(), [...DECLARED].sort());
});
test("snappy-maintenance: every declared refusal is the one closed table's own row, never a copy", () => {
const declared = Object.entries(HAND_CONTRACT.refusals ?? {});
assert.ok(declared.length > 0, "HAND_CONTRACT.refusals is empty");
for (const [code, row] of declared) {
assert.ok(code in REFUSAL_CODES, `${code} is not a row of snappy-settings/refusal-codes.ts`);
assert.equal(row, REFUSAL_CODES[code as RefusalCode], `${code} is a copy of the closed table's row, not the row itself`);
}
});
import { test } from "node:test";
import assert from "node:assert/strict";
import { HAND_CONTRACT } from "./api.ts";
import { REFUSAL_CODES, type RefusalCode } from "../snappy-settings/refusal-codes.ts";
/** RULE 33 COVERAGE, AGAINST THE ONE CLOSED TABLE ⟨lane r30, 2026-09-09⟩.
* This file used to restate a refusal row's own properties — that it has a
* `contract_slice`, that it has a `fix`, that it leaks no token — once per
* hand, 48 times, over rows that all come from the SAME object. Forty-eight
* copies of one check is the duplicate road the closed table exists to end:
* `snappy-settings/refusal-codes.test.ts` runs those checks ONCE over every
* row, and the second test below proves this hand carries THE ROW ITSELF and
* not a copy — an identity a drifted duplicate cannot fake. Before this, each
* hand hand-wrote its own row, and the wording had already drifted: the
* inline `unknown_verb` said "Call one of the verbs named in
* HAND_CONTRACT.verbs" while the closed table says "Call one of the verbs the
* contract declares; the refusal lists them."
*
* DECLARED stays a literal list, deliberately. It is this hand's coverage
* manifest and it is what rule 33's lint reads out of the test SOURCE to ask
* "did a person look at this code" — deriving it from Object.keys would make
* the test pass for a hand with no refusals at all. `satisfies readonly
* RefusalCode[]` makes the compiler refuse a name the one table does not
* have. It NAMES codes; it no longer DEFINES them. */
const DECLARED = [
"unknown_verb",
"missing_argument",
"not_found",
"backend_retired",
"upstream_error",
] as const satisfies readonly RefusalCode[];
test("snappy-maintenance: the refusal table declares exactly the codes this test names", () => {
assert.deepEqual(Object.keys(HAND_CONTRACT.refusals ?? {}).sort(), [...DECLARED].sort());
});
test("snappy-maintenance: every declared refusal is the one closed table's own row, never a copy", () => {
const declared = Object.entries(HAND_CONTRACT.refusals ?? {});
assert.ok(declared.length > 0, "HAND_CONTRACT.refusals is empty");
for (const [code, row] of declared) {
assert.ok(code in REFUSAL_CODES, `${code} is not a row of snappy-settings/refusal-codes.ts`);
assert.equal(row, REFUSAL_CODES[code as RefusalCode], `${code} is a copy of the closed table's row, not the row itself`);
}
});
#!/usr/bin/env bash
set -euo pipefail
# ============================================================================
# health-sweep.sh -- Multi-dimension health sweep for Snappy-managed projects
# Part of snappy-maintenance skill
# Usage: ./health-sweep.sh [--project NAME]
# ============================================================================
# --- Project registry ---
# Each entry: NAME|LOCAL_PATH|HEALTH_URL|PLATFORM|FLY_APP
declare -a PROJECTS=(
"total-crm|/Users/robertboulos/dev/total/hp-base|https://total-crm.fly.dev/health|fly|total-crm"
"total-crm-dev|/Users/robertboulos/dev/total/hp-base|https://total-crm-dev.fly.dev/health|fly|total-crm-dev"
"total-frontend|/Users/robertboulos/dev/total/total-v2|https://app.total.nz|vercel|"
"snappy-website|/Users/robertboulos/dev/snappy/snappy-website|https://snappy.ai|vercel|"
"snappy-mcp|/Users/robertboulos/dev/snappy/snappy-mcp|https://skills.snappy.ai/.well-known/skills/index.json|cloudflare|"
"snappy-skills|/Users/robertboulos/dev/snappy/snappy-skills|https://skills.snappy.ai/.well-known/skills/index.json|cloudflare|"
"box|/Users/robertboulos/dev/snappy/box|http://10.0.0.199:8080/health|box|"
)
# --- Config ---
REPORT_DATE=$(date -u +%Y-%m-%d)
TIMEOUT=10
TARGET_PROJECT=""
REPORT_LINES=()
ISSUES=() # severity|project|dimension|message
# --- Colors (for terminal) ---
RED='\033[0;31m'
YELLOW='\033[0;33m'
GREEN='\033[0;32m'
NC='\033[0m'
# --- Parse args ---
while [[ $# -gt 0 ]]; do
case $1 in
--project)
TARGET_PROJECT="$2"
shift 2
;;
--help|-h)
echo "Usage: health-sweep.sh [--project NAME]"
echo "Projects: total-crm, total-crm-dev, total-frontend, snappy-website, snappy-mcp, snappy-skills, box"
echo "Omit --project to sweep all."
exit 0
;;
*)
echo "Unknown arg: $1" >&2
exit 1
;;
esac
done
# --- Helpers ---
log_issue() {
local severity="$1" project="$2" dimension="$3" message="$4"
ISSUES+=("${severity}|${project}|${dimension}|${message}")
}
parse_project() {
local entry="$1"
IFS='|' read -r P_NAME P_PATH P_URL P_PLATFORM P_FLY_APP <<< "$entry"
}
print_status() {
local status="$1"
case "$status" in
OK) printf "${GREEN}OK${NC}" ;;
WARN) printf "${YELLOW}WARN${NC}" ;;
FAIL) printf "${RED}FAIL${NC}" ;;
esac
}
# ============================================================================
# Dimension checks
# ============================================================================
check_uptime() {
local name="$1" url="$2"
if [[ -z "$url" ]]; then
echo "SKIP|No health URL configured"
return
fi
local http_code response_time
http_code=$(curl -sf -o /dev/null -w "%{http_code}" --max-time "$TIMEOUT" "$url" 2>/dev/null) || http_code="000"
response_time=$(curl -sf -o /dev/null -w "%{time_total}" --max-time "$TIMEOUT" "$url" 2>/dev/null) || response_time="timeout"
if [[ "$http_code" =~ ^2 ]]; then
echo "OK|HTTP ${http_code}, ${response_time}s"
elif [[ "$http_code" == "000" ]]; then
log_issue "HIGH" "$name" "API uptime" "Unreachable (timeout or DNS failure)"
echo "FAIL|Unreachable (timeout/DNS)"
else
log_issue "HIGH" "$name" "API uptime" "HTTP ${http_code}"
echo "FAIL|HTTP ${http_code}, ${response_time}s"
fi
}
check_ssl() {
local name="$1" url="$2"
# Extract host from URL
local host
host=$(echo "$url" | sed -E 's|https?://([^/:]+).*|\1|')
# Skip non-HTTPS
if [[ ! "$url" =~ ^https ]]; then
echo "SKIP|Not HTTPS"
return
fi
local expiry_str expiry_epoch now_epoch days_left
expiry_str=$(echo | openssl s_client -servername "$host" -connect "${host}:443" 2>/dev/null | openssl x509 -noout -enddate 2>/dev/null | cut -d= -f2) || true
if [[ -z "$expiry_str" ]]; then
log_issue "MED" "$name" "SSL cert" "Could not read SSL cert"
echo "WARN|Could not read cert"
return
fi
# macOS date parsing
if date -j -f "%b %d %T %Y %Z" "$expiry_str" +%s &>/dev/null; then
expiry_epoch=$(date -j -f "%b %d %T %Y %Z" "$expiry_str" +%s)
else
# Linux fallback
expiry_epoch=$(date -d "$expiry_str" +%s 2>/dev/null) || { echo "WARN|Could not parse expiry"; return; }
fi
now_epoch=$(date +%s)
days_left=$(( (expiry_epoch - now_epoch) / 86400 ))
if [[ "$days_left" -lt 7 ]]; then
log_issue "HIGH" "$name" "SSL cert" "Expires in ${days_left} days"
echo "FAIL|Expires in ${days_left} days"
elif [[ "$days_left" -lt 30 ]]; then
log_issue "MED" "$name" "SSL cert" "Expires in ${days_left} days"
echo "WARN|Expires in ${days_left} days"
else
echo "OK|Expires in ${days_left} days"
fi
}
check_git_status() {
local name="$1" path="$2"
if [[ ! -d "$path/.git" ]]; then
echo "SKIP|No git repo at ${path}"
return
fi
local uncommitted behind
uncommitted=$(git -C "$path" status --porcelain 2>/dev/null | wc -l | tr -d ' ')
# Fetch quietly to check remote
git -C "$path" fetch --quiet 2>/dev/null || true
local local_ref remote_ref
local_ref=$(git -C "$path" rev-parse HEAD 2>/dev/null) || local_ref=""
remote_ref=$(git -C "$path" rev-parse '@{upstream}' 2>/dev/null) || remote_ref=""
behind=0
if [[ -n "$local_ref" && -n "$remote_ref" && "$local_ref" != "$remote_ref" ]]; then
behind=$(git -C "$path" rev-list --count HEAD..@{upstream} 2>/dev/null || echo "0")
fi
local status="OK" notes=""
if [[ "$uncommitted" -gt 0 ]]; then
notes="${uncommitted} uncommitted files"
status="WARN"
log_issue "LOW" "$name" "Git status" "$notes"
fi
if [[ "$behind" -gt 0 ]]; then
[[ -n "$notes" ]] && notes="${notes}, "
notes="${notes}${behind} commits behind remote"
status="WARN"
log_issue "MED" "$name" "Git status" "${behind} commits behind remote"
fi
[[ -z "$notes" ]] && notes="Clean, up to date"
echo "${status}|${notes}"
}
check_deps_freshness() {
local name="$1" path="$2"
local pkg="${path}/package.json"
if [[ ! -f "$pkg" ]]; then
echo "SKIP|No package.json"
return
fi
# Check package-lock age
local lock="${path}/package-lock.json"
local target="$pkg"
[[ -f "$lock" ]] && target="$lock"
local mod_epoch now_epoch days_old
if stat -f %m "$target" &>/dev/null; then
# macOS
mod_epoch=$(stat -f %m "$target")
else
# Linux
mod_epoch=$(stat -c %Y "$target")
fi
now_epoch=$(date +%s)
days_old=$(( (now_epoch - mod_epoch) / 86400 ))
# Also count outdated if npm is available
local outdated_count=""
if command -v npm &>/dev/null && [[ -f "$lock" ]]; then
outdated_count=$(cd "$path" && npm outdated --json 2>/dev/null | python3 -c "import sys,json; print(len(json.load(sys.stdin)))" 2>/dev/null) || outdated_count=""
fi
local status="OK" notes="Lock file ${days_old}d old"
if [[ -n "$outdated_count" && "$outdated_count" -gt 0 ]]; then
notes="${notes}, ${outdated_count} outdated"
fi
if [[ "$days_old" -gt 90 ]]; then
status="WARN"
log_issue "MED" "$name" "Deps freshness" "Lock file ${days_old} days old"
fi
if [[ -n "$outdated_count" && "$outdated_count" -gt 20 ]]; then
status="WARN"
log_issue "MED" "$name" "Deps freshness" "${outdated_count} outdated packages"
fi
echo "${status}|${notes}"
}
check_deploy_status() {
local name="$1" platform="$2" fly_app="$3"
case "$platform" in
fly)
if ! command -v fly &>/dev/null; then
echo "SKIP|fly CLI not installed"
return
fi
local status_out
status_out=$(fly status --app "$fly_app" 2>&1) || { echo "FAIL|fly status failed"; log_issue "HIGH" "$name" "Deploy" "fly status failed"; return; }
local running
running=$(echo "$status_out" | grep -c "running" || true)
if [[ "$running" -gt 0 ]]; then
local version
version=$(echo "$status_out" | grep -oE 'v[0-9]+' | head -1 || echo "?")
echo "OK|${running} instance(s) running, ${version}"
else
log_issue "HIGH" "$name" "Deploy" "No running instances"
echo "FAIL|No running instances"
fi
;;
vercel)
# Vercel has no simple CLI status -- uptime check covers it
echo "OK|Verified via uptime check"
;;
cloudflare)
echo "OK|Verified via uptime check"
;;
box)
echo "OK|Verified via uptime check"
;;
*)
echo "SKIP|Unknown platform: ${platform}"
;;
esac
}
# ============================================================================
# Run sweep for one project
# ============================================================================
sweep_project() {
local entry="$1"
parse_project "$entry"
echo ""
echo "## ${P_NAME} (${P_PLATFORM})"
echo ""
echo "| Dimension | Status | Details |"
echo "|-----------|--------|---------|"
# 1. API uptime
IFS='|' read -r s d <<< "$(check_uptime "$P_NAME" "$P_URL")"
echo "| API uptime | ${s} | ${d} |"
# 2. SSL cert
IFS='|' read -r s d <<< "$(check_ssl "$P_NAME" "$P_URL")"
echo "| SSL cert | ${s} | ${d} |"
# 3. Git status
IFS='|' read -r s d <<< "$(check_git_status "$P_NAME" "$P_PATH")"
echo "| Git status | ${s} | ${d} |"
# 4. Deps freshness
IFS='|' read -r s d <<< "$(check_deps_freshness "$P_NAME" "$P_PATH")"
echo "| Deps freshness | ${s} | ${d} |"
# 5. Deploy status
IFS='|' read -r s d <<< "$(check_deploy_status "$P_NAME" "$P_PLATFORM" "$P_FLY_APP")"
echo "| Deploy status | ${s} | ${d} |"
}
# ============================================================================
# Main
# ============================================================================
echo "# Health Sweep Report -- ${REPORT_DATE}"
echo ""
echo "Generated: $(date -u +%Y-%m-%dT%H:%M:%SZ)"
for entry in "${PROJECTS[@]}"; do
parse_project "$entry"
if [[ -n "$TARGET_PROJECT" && "$P_NAME" != "$TARGET_PROJECT" ]]; then
continue
fi
sweep_project "$entry"
done
# If a specific project was requested but not found
if [[ -n "$TARGET_PROJECT" ]]; then
found=false
for entry in "${PROJECTS[@]}"; do
IFS='|' read -r pn _ _ _ _ <<< "$entry"
[[ "$pn" == "$TARGET_PROJECT" ]] && found=true
done
if [[ "$found" == false ]]; then
echo ""
echo "ERROR: Project '${TARGET_PROJECT}' not found in registry."
echo "Known projects: $(for e in "${PROJECTS[@]}"; do IFS='|' read -r n _ _ _ _ <<< "$e"; printf "%s " "$n"; done)"
exit 1
fi
fi
# ============================================================================
# Issue summary (severity-ranked)
# ============================================================================
echo ""
echo "---"
echo ""
echo "## Issue Summary"
echo ""
if [[ ${#ISSUES[@]} -eq 0 ]]; then
echo "No issues detected. All checks passed."
else
echo "| Severity | Project | Dimension | Details |"
echo "|----------|---------|-----------|---------|"
# Sort: HIGH first, then MED, then LOW
for sev in HIGH MED LOW; do
for issue in "${ISSUES[@]}"; do
IFS='|' read -r is ip id im <<< "$issue"
if [[ "$is" == "$sev" ]]; then
echo "| **${is}** | ${ip} | ${id} | ${im} |"
fi
done
done
# Count by severity
high_count=0; med_count=0; low_count=0
for issue in "${ISSUES[@]}"; do
case "${issue%%|*}" in
HIGH) ((high_count++)) ;;
MED) ((med_count++)) ;;
LOW) ((low_count++)) ;;
esac
done
echo ""
echo "**Totals:** ${high_count} HIGH, ${med_count} MED, ${low_count} LOW"
if [[ "$high_count" -gt 0 ]]; then
echo ""
echo "> **ACTION REQUIRED:** ${high_count} HIGH severity issue(s) detected. Alert via snappy-telegram before proceeding."
fi
fi
#!/usr/bin/env bash
set -euo pipefail
# ============================================================================
# health-sweep.sh -- Multi-dimension health sweep for Snappy-managed projects
# Part of snappy-maintenance skill
# Usage: ./health-sweep.sh [--project NAME]
# ============================================================================
# --- Project registry ---
# Each entry: NAME|LOCAL_PATH|HEALTH_URL|PLATFORM|FLY_APP
declare -a PROJECTS=(
"total-crm|/Users/robertboulos/dev/total/hp-base|https://total-crm.fly.dev/health|fly|total-crm"
"total-crm-dev|/Users/robertboulos/dev/total/hp-base|https://total-crm-dev.fly.dev/health|fly|total-crm-dev"
"total-frontend|/Users/robertboulos/dev/total/total-v2|https://app.total.nz|vercel|"
"snappy-website|/Users/robertboulos/dev/snappy/snappy-website|https://snappy.ai|vercel|"
"snappy-mcp|/Users/robertboulos/dev/snappy/snappy-mcp|https://skills.snappy.ai/.well-known/skills/index.json|cloudflare|"
"snappy-skills|/Users/robertboulos/dev/snappy/snappy-skills|https://skills.snappy.ai/.well-known/skills/index.json|cloudflare|"
"box|/Users/robertboulos/dev/snappy/box|http://10.0.0.199:8080/health|box|"
)
# --- Config ---
REPORT_DATE=$(date -u +%Y-%m-%d)
TIMEOUT=10
TARGET_PROJECT=""
REPORT_LINES=()
ISSUES=() # severity|project|dimension|message
# --- Colors (for terminal) ---
RED='\033[0;31m'
YELLOW='\033[0;33m'
GREEN='\033[0;32m'
NC='\033[0m'
# --- Parse args ---
while [[ $# -gt 0 ]]; do
case $1 in
--project)
TARGET_PROJECT="$2"
shift 2
;;
--help|-h)
echo "Usage: health-sweep.sh [--project NAME]"
echo "Projects: total-crm, total-crm-dev, total-frontend, snappy-website, snappy-mcp, snappy-skills, box"
echo "Omit --project to sweep all."
exit 0
;;
*)
echo "Unknown arg: $1" >&2
exit 1
;;
esac
done
# --- Helpers ---
log_issue() {
local severity="$1" project="$2" dimension="$3" message="$4"
ISSUES+=("${severity}|${project}|${dimension}|${message}")
}
parse_project() {
local entry="$1"
IFS='|' read -r P_NAME P_PATH P_URL P_PLATFORM P_FLY_APP <<< "$entry"
}
print_status() {
local status="$1"
case "$status" in
OK) printf "${GREEN}OK${NC}" ;;
WARN) printf "${YELLOW}WARN${NC}" ;;
FAIL) printf "${RED}FAIL${NC}" ;;
esac
}
# ============================================================================
# Dimension checks
# ============================================================================
check_uptime() {
local name="$1" url="$2"
if [[ -z "$url" ]]; then
echo "SKIP|No health URL configured"
return
fi
local http_code response_time
http_code=$(curl -sf -o /dev/null -w "%{http_code}" --max-time "$TIMEOUT" "$url" 2>/dev/null) || http_code="000"
response_time=$(curl -sf -o /dev/null -w "%{time_total}" --max-time "$TIMEOUT" "$url" 2>/dev/null) || response_time="timeout"
if [[ "$http_code" =~ ^2 ]]; then
echo "OK|HTTP ${http_code}, ${response_time}s"
elif [[ "$http_code" == "000" ]]; then
log_issue "HIGH" "$name" "API uptime" "Unreachable (timeout or DNS failure)"
echo "FAIL|Unreachable (timeout/DNS)"
else
log_issue "HIGH" "$name" "API uptime" "HTTP ${http_code}"
echo "FAIL|HTTP ${http_code}, ${response_time}s"
fi
}
check_ssl() {
local name="$1" url="$2"
# Extract host from URL
local host
host=$(echo "$url" | sed -E 's|https?://([^/:]+).*|\1|')
# Skip non-HTTPS
if [[ ! "$url" =~ ^https ]]; then
echo "SKIP|Not HTTPS"
return
fi
local expiry_str expiry_epoch now_epoch days_left
expiry_str=$(echo | openssl s_client -servername "$host" -connect "${host}:443" 2>/dev/null | openssl x509 -noout -enddate 2>/dev/null | cut -d= -f2) || true
if [[ -z "$expiry_str" ]]; then
log_issue "MED" "$name" "SSL cert" "Could not read SSL cert"
echo "WARN|Could not read cert"
return
fi
# macOS date parsing
if date -j -f "%b %d %T %Y %Z" "$expiry_str" +%s &>/dev/null; then
expiry_epoch=$(date -j -f "%b %d %T %Y %Z" "$expiry_str" +%s)
else
# Linux fallback
expiry_epoch=$(date -d "$expiry_str" +%s 2>/dev/null) || { echo "WARN|Could not parse expiry"; return; }
fi
now_epoch=$(date +%s)
days_left=$(( (expiry_epoch - now_epoch) / 86400 ))
if [[ "$days_left" -lt 7 ]]; then
log_issue "HIGH" "$name" "SSL cert" "Expires in ${days_left} days"
echo "FAIL|Expires in ${days_left} days"
elif [[ "$days_left" -lt 30 ]]; then
log_issue "MED" "$name" "SSL cert" "Expires in ${days_left} days"
echo "WARN|Expires in ${days_left} days"
else
echo "OK|Expires in ${days_left} days"
fi
}
check_git_status() {
local name="$1" path="$2"
if [[ ! -d "$path/.git" ]]; then
echo "SKIP|No git repo at ${path}"
return
fi
local uncommitted behind
uncommitted=$(git -C "$path" status --porcelain 2>/dev/null | wc -l | tr -d ' ')
# Fetch quietly to check remote
git -C "$path" fetch --quiet 2>/dev/null || true
local local_ref remote_ref
local_ref=$(git -C "$path" rev-parse HEAD 2>/dev/null) || local_ref=""
remote_ref=$(git -C "$path" rev-parse '@{upstream}' 2>/dev/null) || remote_ref=""
behind=0
if [[ -n "$local_ref" && -n "$remote_ref" && "$local_ref" != "$remote_ref" ]]; then
behind=$(git -C "$path" rev-list --count HEAD..@{upstream} 2>/dev/null || echo "0")
fi
local status="OK" notes=""
if [[ "$uncommitted" -gt 0 ]]; then
notes="${uncommitted} uncommitted files"
status="WARN"
log_issue "LOW" "$name" "Git status" "$notes"
fi
if [[ "$behind" -gt 0 ]]; then
[[ -n "$notes" ]] && notes="${notes}, "
notes="${notes}${behind} commits behind remote"
status="WARN"
log_issue "MED" "$name" "Git status" "${behind} commits behind remote"
fi
[[ -z "$notes" ]] && notes="Clean, up to date"
echo "${status}|${notes}"
}
check_deps_freshness() {
local name="$1" path="$2"
local pkg="${path}/package.json"
if [[ ! -f "$pkg" ]]; then
echo "SKIP|No package.json"
return
fi
# Check package-lock age
local lock="${path}/package-lock.json"
local target="$pkg"
[[ -f "$lock" ]] && target="$lock"
local mod_epoch now_epoch days_old
if stat -f %m "$target" &>/dev/null; then
# macOS
mod_epoch=$(stat -f %m "$target")
else
# Linux
mod_epoch=$(stat -c %Y "$target")
fi
now_epoch=$(date +%s)
days_old=$(( (now_epoch - mod_epoch) / 86400 ))
# Also count outdated if npm is available
local outdated_count=""
if command -v npm &>/dev/null && [[ -f "$lock" ]]; then
outdated_count=$(cd "$path" && npm outdated --json 2>/dev/null | python3 -c "import sys,json; print(len(json.load(sys.stdin)))" 2>/dev/null) || outdated_count=""
fi
local status="OK" notes="Lock file ${days_old}d old"
if [[ -n "$outdated_count" && "$outdated_count" -gt 0 ]]; then
notes="${notes}, ${outdated_count} outdated"
fi
if [[ "$days_old" -gt 90 ]]; then
status="WARN"
log_issue "MED" "$name" "Deps freshness" "Lock file ${days_old} days old"
fi
if [[ -n "$outdated_count" && "$outdated_count" -gt 20 ]]; then
status="WARN"
log_issue "MED" "$name" "Deps freshness" "${outdated_count} outdated packages"
fi
echo "${status}|${notes}"
}
check_deploy_status() {
local name="$1" platform="$2" fly_app="$3"
case "$platform" in
fly)
if ! command -v fly &>/dev/null; then
echo "SKIP|fly CLI not installed"
return
fi
local status_out
status_out=$(fly status --app "$fly_app" 2>&1) || { echo "FAIL|fly status failed"; log_issue "HIGH" "$name" "Deploy" "fly status failed"; return; }
local running
running=$(echo "$status_out" | grep -c "running" || true)
if [[ "$running" -gt 0 ]]; then
local version
version=$(echo "$status_out" | grep -oE 'v[0-9]+' | head -1 || echo "?")
echo "OK|${running} instance(s) running, ${version}"
else
log_issue "HIGH" "$name" "Deploy" "No running instances"
echo "FAIL|No running instances"
fi
;;
vercel)
# Vercel has no simple CLI status -- uptime check covers it
echo "OK|Verified via uptime check"
;;
cloudflare)
echo "OK|Verified via uptime check"
;;
box)
echo "OK|Verified via uptime check"
;;
*)
echo "SKIP|Unknown platform: ${platform}"
;;
esac
}
# ============================================================================
# Run sweep for one project
# ============================================================================
sweep_project() {
local entry="$1"
parse_project "$entry"
echo ""
echo "## ${P_NAME} (${P_PLATFORM})"
echo ""
echo "| Dimension | Status | Details |"
echo "|-----------|--------|---------|"
# 1. API uptime
IFS='|' read -r s d <<< "$(check_uptime "$P_NAME" "$P_URL")"
echo "| API uptime | ${s} | ${d} |"
# 2. SSL cert
IFS='|' read -r s d <<< "$(check_ssl "$P_NAME" "$P_URL")"
echo "| SSL cert | ${s} | ${d} |"
# 3. Git status
IFS='|' read -r s d <<< "$(check_git_status "$P_NAME" "$P_PATH")"
echo "| Git status | ${s} | ${d} |"
# 4. Deps freshness
IFS='|' read -r s d <<< "$(check_deps_freshness "$P_NAME" "$P_PATH")"
echo "| Deps freshness | ${s} | ${d} |"
# 5. Deploy status
IFS='|' read -r s d <<< "$(check_deploy_status "$P_NAME" "$P_PLATFORM" "$P_FLY_APP")"
echo "| Deploy status | ${s} | ${d} |"
}
# ============================================================================
# Main
# ============================================================================
echo "# Health Sweep Report -- ${REPORT_DATE}"
echo ""
echo "Generated: $(date -u +%Y-%m-%dT%H:%M:%SZ)"
for entry in "${PROJECTS[@]}"; do
parse_project "$entry"
if [[ -n "$TARGET_PROJECT" && "$P_NAME" != "$TARGET_PROJECT" ]]; then
continue
fi
sweep_project "$entry"
done
# If a specific project was requested but not found
if [[ -n "$TARGET_PROJECT" ]]; then
found=false
for entry in "${PROJECTS[@]}"; do
IFS='|' read -r pn _ _ _ _ <<< "$entry"
[[ "$pn" == "$TARGET_PROJECT" ]] && found=true
done
if [[ "$found" == false ]]; then
echo ""
echo "ERROR: Project '${TARGET_PROJECT}' not found in registry."
echo "Known projects: $(for e in "${PROJECTS[@]}"; do IFS='|' read -r n _ _ _ _ <<< "$e"; printf "%s " "$n"; done)"
exit 1
fi
fi
# ============================================================================
# Issue summary (severity-ranked)
# ============================================================================
echo ""
echo "---"
echo ""
echo "## Issue Summary"
echo ""
if [[ ${#ISSUES[@]} -eq 0 ]]; then
echo "No issues detected. All checks passed."
else
echo "| Severity | Project | Dimension | Details |"
echo "|----------|---------|-----------|---------|"
# Sort: HIGH first, then MED, then LOW
for sev in HIGH MED LOW; do
for issue in "${ISSUES[@]}"; do
IFS='|' read -r is ip id im <<< "$issue"
if [[ "$is" == "$sev" ]]; then
echo "| **${is}** | ${ip} | ${id} | ${im} |"
fi
done
done
# Count by severity
high_count=0; med_count=0; low_count=0
for issue in "${ISSUES[@]}"; do
case "${issue%%|*}" in
HIGH) ((high_count++)) ;;
MED) ((med_count++)) ;;
LOW) ((low_count++)) ;;
esac
done
echo ""
echo "**Totals:** ${high_count} HIGH, ${med_count} MED, ${low_count} LOW"
if [[ "$high_count" -gt 0 ]]; then
echo ""
echo "> **ACTION REQUIRED:** ${high_count} HIGH severity issue(s) detected. Alert via snappy-telegram before proceeding."
fi
fi