OR Key
drop another .md file to compare - side-by-side diff against distill-approvals

distill-approvals

Learns from what you approve so future drafts match your taste.
description: "Triggers on prompt mention of 'distill-approvals'."
personal 2 files 10 recent evals

What it does for you

Learns from what you approve so future drafts match your taste.

What it produces

A recent result, so you can see the kind of work it returns.

loading…

How to get it

These run inside the Snappy workspace. Want this working in your business? I set skills like this up with you, in one focused week.

Work with me
For developers how this skill is built, graded, and how it runs

at a glance- the short version

eval modeauto
categorySystem
dependsdrafts

what's inside - the parts that make up a skill 2/4 present

A skill is just a few plain-text files. Only the main one is required. The rest are optional, added as the work needs them. This is what the skill is made of; how it runs is just below.

The skill
state/skills/distill-approvals/SKILL.md present
the skill itself, in plain text
The main file. It says what the skill is and lays out the steps in plain English.
Code
state/lib/distill-approvals.ts not present
code the skill can run
Optional. Many skills are just words and need no code at all.
Scripts
state/bin/distill-approvals/ not present
helper scripts
Optional. Added when a skill has a few commands to run.
Loader
state/skills/distill-approvals/AGENTS.md present
what the AI loads on the fly
Loaded automatically the moment this skill is needed. Kept short on purpose.

how it runs - the shared frame every skill uses 3/5 present

Every skill runs the same way. One part does the work, a separate part checks it, and a short loader hands the AI exactly what it needs for the job. Anything this skill doesn't use shows a one-line note saying why, on purpose, not by accident.

makes the work The worker
not present

No work step here. This is probably a skill that reads or coordinates, not one that produces something.

checks the work The reviewer
not present

No separate check found. Without one, the part that makes the work could end up approving its own work, worth a closer look.

frame
learns Self-correction
present
fixes itself learns from gaps
When a run hits a gap, the skill gets edited on the spot [FIXED] or queued for a bigger rewrite [LOGGED], so it keeps getting better.
tidies up Background fixes
present
queued for rewrite runs in the background
Bigger fixes that can't be made on the spot get queued and rewritten in the background later.
remembers Run history
present
state/log/evals.ndjson auto runs
Every run is written down here, so the next time this skill is used it already knows how the last runs went.
Critical rules the things this skill must not get wrong
  1. READ-ONLY against state/log/approvals.ndjson. Never write back to that file — it's the source of truth for "Robert clicked X."
  2. Output goes to TWO files (both, not either): state/log/taste-hints.ndjson (newline-append, machine-readable for next-dispatch) AND state/log/distill-approvals/<utc-date>.json (structured per-day rollup for cockpit).
  3. Distinct from distill (human-authored journal). This is automation-scale taste — never write to state/observations.md or state/memory/. Per feedback_memory_human_scale_only, don't pollute the journal layer with cluster counts.
  4. Empty approvals.ndjson ⇒ exit 0, score: 0.5, primary_issue: "no-approvals-yet". Not a failure — the loop just hasn't produced taste signal yet.
  5. Single-bucket case (every (skill, shape) has n<2) ⇒ exit 0, score: 0.5, primary_issue: "insufficient-signal". Wait for more.
  6. Shape inference is deterministic (no LLM dispatch) AND lives in state/lib/draft-shape.ts so writer (action-item-draft) and reader (here) stay in lockstep. Drift between the two breaks the loop silently.
  7. +4 more in AGENTS.md →

what it has learned - fixes written back in over time sample

When a run hits something this skill didn't handle, the fix gets written back into the skill so it doesn't happen again. FIXED means it was corrected on the spot. LOGGED means it's queued for a bigger rewrite. Either way, the skill gets a little better and never makes the same mistake twice.

  1. Loading feedback rows…

SKILL.md- the skill, written out in plain English

Backed by: state/lib/log.ts, state/bin/distill-approvals/run.ts

distill-approvals

The taste loop closer. Approvals were a write-only ledger until this skill existed - POST /draft/decision wrote rows to state/log/approvals.ndjson and nothing read them. This skill is the reader: every Approve/Dismiss click teaches the system what kind of drafts Robert keeps vs throws away, and the next draft attempt is shaped by that taste.

Distinct from distill (which scans the human-authored journal at state/memory/) - this is automation-scale taste signal, not human-scale narrative. Per the human-scale rule, journal observations stay separate; approval patterns live in their own ledger.

Steps

The sidecar (state/bin/distill-approvals/run.ts) does the work end-to-end. Invoked by /run-skill distill-approvals or auto-fired by the server after each POST /draft/decision.

  1. Read state/log/approvals.ndjson. If empty, score 0.5 with

primary_issue: "no-approvals-yet" and exit - the loop hasn't produced a taste signal yet.

  1. For each row, infer the artifact shape from the draft body that

row points at: linkedin-post, email, message, or other. Heuristic: read the path from the row, open the draft, scan first 400 chars for shape markers (Subject:, Dear, "post", "tweet", etc.). Cache shape inference into the row to avoid re-reading.

  1. Bucket: (source_skill, shape) → {accept_count, reject_count, edit_count, sample_titles, dismissal_reasons[]}.
  1. Compute per-bucket stats: accept_rate = accepts / (accepts + rejects),

confidence band based on sample size (n<3 low, n<10 medium, n>=10 high).

  1. Build a single-line taste-hint for each (skill, shape) bucket

with n>=2 and write to state/log/taste-hints.ndjson. Schema: {ts, source_skill, shape, accept_rate, n, confidence, hint}. The hint is the natural-language hint the next dispatch reads, e.g. "Recent: 3/4 LinkedIn drafts accepted, 2/2 email drafts dismissed — default to LinkedIn shape unless the action item is clearly a 1:1 message."

  1. Write a structured per-day summary at

state/log/distill-approvals/<utc-date>.json with the full bucketing for the cockpit to render later.

  1. Score the eval row (distinct actor + auditor session ids per

CONSTITUTION #3) with artifact_path set to the summary file.

Eval

auto. Shape gate: if approvals.ndjson has rows, the summary file exists AND taste-hints.ndjson got at least one new line. Score 1 when all true; 0.5 when approvals exist but everything is single-bucket (n<2 everywhere - not enough signal yet); 0.5 with reason no-approvals-yet when the source ledger is empty.

Why

Per the four-agent debate (cockpit-UX, eval-semantics, leverage-map, meta - 2026-04-26): the approval primitive shipped without a reader. Approvals were write-only theater. This skill is the bridge between "Robert clicked Approve/Dismiss" and "the next draft attempt is better-shaped because of it." Without this, the cockpit produces drafts shaped by guesses; with this, it produces drafts shaped by Robert's actual past decisions.

Auto-fired by the server's POST /draft/decision handler so the loop closes in real time - no cron lag.

AGENTS.md- what the AI loads when this skill comes up

distill-approvals - loader

Per-turn rules. Full reference: state/skills/distill-approvals/SKILL.md. Do not skip these.

Critical Rules

  • READ-ONLY against state/log/approvals.ndjson. Never write back to that file - it's the source of truth for "Robert clicked X."
  • Output goes to TWO files (both, not either): state/log/taste-hints.ndjson (newline-append, machine-readable for next-dispatch) AND state/log/distill-approvals/<utc-date>.json (structured per-day rollup for cockpit).
  • Distinct from distill (human-authored journal). This is automation-scale taste - never write to state/observations.md or state/memory/. Per feedback_memory_human_scale_only, don't pollute the journal layer with cluster counts.
  • Empty approvals.ndjson ⇒ exit 0, score: 0.5, primary_issue: "no-approvals-yet". Not a failure - the loop just hasn't produced taste signal yet.
  • Single-bucket case (every (skill, shape) has n<2) ⇒ exit 0, score: 0.5, primary_issue: "insufficient-signal". Wait for more.
  • Shape inference is deterministic (no LLM dispatch) AND lives in state/lib/draft-shape.ts so writer (action-item-draft) and reader (here) stay in lockstep. Drift between the two breaks the loop silently.
  • Path resolution: approval rows store paths state-relative (log/action-item-draft/...) matching the /drafts endpoint convention the cockpit speaks. Resolve via join(ROOT/state, path). Drafts written by skill code may emit state/log/... in their JSON draft_path field - that's the dispatch result, NOT the approval row. Don't store state/log/... in approvals.ndjson; distill double-prefixes and silently classifies every row as "other".
  • Latency tracking: per-bucket p50/p90 of accept_latencies_ms and reject_latencies_ms lives in the rollup. Hint copy includes "Median accept latency Xs" when ≥1 accept has timing - fast accepts mean the shape lands; slow accepts after a long pause are weaker signal.
  • Actor ≠ auditor (CONSTITUTION #3). actor_session_id = sessionId(); auditor_session_id = "distill-approvals-aud-<ts>-<hex>". Build independently.
  • artifact_path MUST point at the per-day summary file written this run, so the eval row links back to its output.

Commands

| ui dashboard | state/skills/distill-approvals/resources/ui.openui |

purposeinvocation
invoke (cockpit)POST /dispatch {key:"/run-skill", scope:"distill-approvals"}
invoke (cli)npx tsx state/bin/distill-approvals/run.ts
auto-fireserver's POST /draft/decision handler async-spawns this after writing the approval row
read sourcereadFileSync("state/log/approvals.ndjson", "utf8") - split on \n, JSON.parse each line
read draft body for shape inferencereadFileSync(row.path, "utf8").slice(0, 400) after stripping frontmatter
write hintsappend-only newline-JSON to state/log/taste-hints.ndjson
write rollupstate/log/distill-approvals/<utc-date>.json (overwrite - last run of the day wins)
scoreimport { score, sessionId } from "../lib/eval.ts" - pass artifact_path = rollup path
consumerdrafting skills (currently action-item-draft) read latest hint per (source_skill, shape) and inject into prompt
eval logstate/log/evals.ndjson (skill: "distill-approvals")

Output schemas

state/log/taste-hints.ndjson - one row per (skill, shape) bucket per run:

{ "ts":"<utc-iso>", "source_skill":"action-item-draft",
  "shape":"linkedin-post"|"email"|"message"|"other",
  "accept_rate":0.75, "n":4, "confidence":"low"|"medium"|"high",
  "hint":"<one-sentence guidance for next dispatch>" }

state/log/distill-approvals/<utc-date>.json - full structured rollup:

{ "computed_at":"<utc-iso>", "rows_scanned":12,
  "by_bucket": {
    "action-item-draft|linkedin-post": {
      "accepts":3, "rejects":1, "edits":0,
      "accept_rate":0.75, "n":4, "confidence":"low",
      "sample_accepted_titles":[...], "sample_rejected_titles":[...],
      "dismissal_reasons":[...] } } }

Shape inference heuristic (deterministic, in order)

matchshape
Subject: line OR Dear greeting in first 400 charsemail
[at] mentions OR LinkedIn mention OR "post about X" framing OR body < 1500 chars totallinkedin-post
Direct second-person ("you") with no greetingmessage
anything elseother

Confidence band

nband
n < 3low
3 ≤ n < 10medium
n ≥ 10high

accept_rate = accepts / (accepts + rejects). Only emit a hint row when n ≥ 2.

Recoverable conditions (exit 0, not 1)

conditionscoreprimary_issue
approvals.ndjson empty/missing0.5no-approvals-yet
every bucket has n<20.5insufficient-signal
a row's path points at a deleted draft0.5draft-missing (skip the row, continue)

Self-Test

Reading this loader, an agent should:

  1. [ ] Read approvals.ndjson without writing back to it.
  2. [ ] Write BOTH taste-hints.ndjson AND distill-approvals/<date>.json.
  3. [ ] Use the deterministic shape table (no LLM in shape inference).
  4. [ ] Score 0.5 with no-approvals-yet on empty source - not 0.
  5. [ ] Score 0.5 with insufficient-signal when every bucket is n<2.
  6. [ ] NOT touch state/observations.md or state/memory/.
  7. [ ] Use distinct actor/auditor session ids and pass artifact_path = rollup path.

Self-report

If this loader fell short:

echo "[$(date -u +%FT%TZ)] distill-approvals: <what was missing>" >> state/log/loader-feedback.log

<!-- kernel-ok: historical attribution for the PID-loader pattern. -->

Self-correcting loader (PID feedback)

This is the rule that makes loaders learn. Ported from snappy-kernel, where it demonstrably worked: the snappy-dom-cartographer skill mapped Skool admin UI once, every later ship-agent inherited the full map for free.

Before you finish, do two things:

1. Fix gaps (P - proportional). MANDATORY. If this loader didn't cover your case - if you had to read another file, run an undocumented command, work around a wrong selector, or discover a quirk - you MUST attempt an Edit to this AGENTS.md before you log.

  • Default action: edit this file. Surgical. Add the missing line, table

row, or code example where it belongs. No restructuring.

  • Only LOGGED is allowed when: the fix needs >10 lines, spans multiple

files, or requires a structural rewrite. In that case the state/regen/drain.sh queue picks it up asynchronously.

  • "I didn't have time" / "it's minor" / "the next agent will figure it out"

are NOT valid reasons. Minor gaps compound - that's the whole point of this loop.

  • Concrete example of a valid P-fix: discover agent-browser screenshot

takes path positionally not as --output <path>. One line in a table. Edit it in. Do not just log it.

  • The goal: the next agent never has to leave the loader.

2. Log the result.

echo "[$(date -u +%FT%TZ)] distill-approvals: <what was missing or fixed> [FIXED|LOGGED]" >> state/log/loader-feedback.log
  • FIXED = you patched this loader inline (P-fix).
  • LOGGED = too large for inline; the PostToolUse enqueue + Stop-hook drain

will rewrite the loader from scratch on next session-end.

Do not skip this. Every agent run must leave the system better than it found it. The loader is the setpoint; you are the sensor; the gap is the error signal; closing the gap is the correction.

OpenUI Resource

  • Skill-owned OpenUI Lang resource: state/skills/distill-approvals/resources/ui.openui. Read it before rendering or editing this skill's generated component surface.
  • Treat this resource as a first-class artifact of the skill, not a generic chat response. Improve it when the skill's user-facing output needs to become richer.
  • System resources compose OpenUI primitives and inherit SnappyChat tokens. Use ui_contract: branded in SKILL.md only for deliberate platform or client visuals.

api.ts- the code it can call

⚠ no api.ts - this skill has no typed action surface

scripts- helper scripts it can run

prose-only skill - no sidecar under state/bin/ yet. Steps, if any, are described in SKILL.md.

how we check it- the checks, plus the last 10 runs

rubric auto no rubric declared
recent mean 0.90 · 10 runs actor/auditor: unverifiable
deps drafts
timestamp verb score primary_issue artifact
2026-04-27 02:21Z - 1.00 - -
2026-04-27 02:21Z - 1.00 - -
2026-04-27 02:16Z - 1.00 - -
2026-04-27 02:15Z - 1.00 - -
2026-04-27 02:14Z - 1.00 - -
2026-04-27 02:09Z - 0.50 - -
2026-04-27 01:43Z - 0.50 - -
2026-04-27 02:21Z - 1.00 - -
2026-04-27 02:21Z - 1.00 - -
2026-04-27 02:16Z - 1.00 - -