OR Key
drop another .md file to compare - side-by-side diff against dogfood-loop

dogfood-loop

Has your assistant test your app like a real user and fix what falls short.
personal 2 files

What it does for you

Has your assistant test your app like a real user and fix what falls short.

What it produces

A recent result, so you can see the kind of work it returns.

loading…

How to get it

These run inside the Snappy workspace. Want this working in your business? I set skills like this up with you, in one focused week.

Work with me
For developers how this skill is built, graded, and how it runs

at a glance- the short version

actorDogfood-running subagent
auditorNext agent (or Robert)
eval modeauto-shape
stages1

what's inside - the parts that make up a skill 3/4 present

A skill is just a few plain-text files. Only the main one is required. The rest are optional, added as the work needs them. This is what the skill is made of; how it runs is just below.

The skill
state/skills/dogfood-loop/SKILL.md present
the skill itself, in plain text
The main file. It says what the skill is and lays out the steps in plain English.
Code
state/lib/dogfood-loop.ts present
code the skill can run
Reusable code this skill can call when it needs to.
Scripts
state/bin/dogfood-loop/ not present
helper scripts
Optional. Added when a skill has a few commands to run.
Loader
state/skills/dogfood-loop/AGENTS.md present
what the AI loads on the fly
Loaded automatically the moment this skill is needed. Kept short on purpose.

how it's graded - what counts as a good run 4 criteria · 3 deterministic · 1 judge

Each row is one thing a good run has to get right. deterministic means a quick check decides, pass or fail. judge means the AI reads the result and rates it. Grading each piece on its own (instead of one overall score) shows exactly where a run fell short, so the fix is obvious.

name
kind
check
report_present
deterministic
/tmp/dogfood-loop-report.md exists and contains a Summary section.
scenarios_min_three
deterministic
Report contains at least three Scenario blocks, each with Pre, Post, Card rendered, Render quality, Action, Notes lines.
every_fail_has_action
judge
Every scenario marked 'fail' carries either a commit SHA in Action OR an entry appended to state/log/snappy-chat-backlog.md - no orphan failures.
no_external_sideeffects
deterministic
No scenario triggered a real outbound send (no LinkedIn URN, no Gmail draft, no Slack post). The dispatch backend stays text-only or generative-shape only.

how it runs - the shared frame every skill uses 4/5 present

Every skill runs the same way. One part does the work, a separate part checks it, and a short loader hands the AI exactly what it needs for the job. Anything this skill doesn't use shows a one-line note saying why, on purpose, not by accident.

makes the work The worker
present
Dogfood-running subagent the worker
Does the actual work. Whatever it produces is what gets checked next.
checks the work The reviewer
present
Next agent (or Robert) the checker
A separate checker grades the work, so the part that made it can't approve its own work.
frame
learns Self-correction
not present

This skill doesn't fix its own gaps yet.

tidies up Background fixes
present
queued for rewrite runs in the background
Bigger fixes that can't be made on the spot get queued and rewritten in the background later.
remembers Run history
present
state/log/evals.ndjson auto-shape runs
Every run is written down here, so the next time this skill is used it already knows how the last runs went.
Critical rules the things this skill must not get wrong
No must-not-break rules called out for this skill. Anything important lives in the writeup below.

what it has learned - fixes written back in over time sample

When a run hits something this skill didn't handle, the fix gets written back into the skill so it doesn't happen again. FIXED means it was corrected on the spot. LOGGED means it's queued for a bigger rewrite. Either way, the skill gets a little better and never makes the same mistake twice.

  1. Loading feedback rows…

how the work flows- who makes it, who checks it

actor Dogfood-running subagent
auditor Next agent (or Robert)
1 data
npx
npx tsx state/bin/dogfood-loop/run.ts \

SKILL.md- the skill, written out in plain English

dogfood-loop

The dogfood loop is how Snappy OS gets sharper between releases. A subagent is BOTH a user (drives the app via the chat-drive queue) AND a developer (commits fixes to Snappy OS when a rendered card falls below world-class). Closing this loop is what unlocks Robert's generative-UI leverage.

The deliverable IS the playbook below. There is no backing lib - the prose is the executable. Producer skills (agent-recap, bundle-snapshot, chat-drive) supply the primitives; this skill stitches them into a cadence.

The primitives stack

[ subagent in any context ]
        │
        ▼
state/lib/chat-drive.ts  ─ dispatchInChatUI(text)  ──┐
                          resetChatUI()              │
                                                     ▼
state/lib/desktop.ts     ─ captureScreen(path)
                          captureWindow(app, path)
                          listWindows(app?)

Snappy OS polls 127.0.0.1:3147/chat-inject-pop every 500ms. When dispatchInChatUI pushes text to /chat-inject-push, the React polling effect injects via the React-controlled-component setter dance and fires submit. resetChatUI pushes a control message that flips setHasThread(false) + setChatKey(k+1), returning the welcome surface for the next scenario.

Steps

The 7-step cadence below is automated by the sidecar at state/bin/dogfood-loop/run.ts. Drive a full pass with:

npx tsx state/bin/dogfood-loop/run.ts \
  --scenarios state/skills/dogfood-loop/scenarios/default.json \
  --out /tmp/dogfood-loop-report.md

The sidecar handles steps 1-4 + 7 (reset, pre-shot, push, post-shot, append to report) for every scenario, then exits. Steps 5 and 6 (visual grading, fix-or-log) stay manual - actor (the sidecar) captures; auditor (the agent that runs it) reads each /tmp/loop-N-post.png against the eval checklist below and decides commit-vs-backlog.

For each scenario:

  1. Reset - await resetChatUI({waitMs: 1500}). Welcome surface should

remount.

  1. Pre-shot - await captureScreen('/tmp/loop-N-pre.png'). Confirms

reset succeeded.

  1. Push intent - `await dispatchInChatUI(intent, {waitForFirstFrame:

14000})`.

  1. Post-shot - await captureScreen('/tmp/loop-N-post.png').
  2. Read both shots with the Read tool. Evaluate against the eval

checklist below.

  1. Fix or log - if anything is below world-class, edit

web/src/styles.css or the relevant component, redeploy via bash scripts/build-app.sh --install, re-run the scenario to verify. If the fix needs >25 LOC or spans 3+ files, stop and append to ~/projects/snappy-os/state/log/snappy-os-backlog.md instead.

  1. Append to report - /tmp/dogfood-loop-report.md per the format at

the bottom of this doc.

Eval checklist - is this card world-class?

For every rendered scenario, check all of these:

Render integrity

  • [ ] Welcome surface unmounted, thread state visible
  • [ ] No console errors (open DevTools via menubar, or check WKWebView log

if available)

  • [ ] User pill right-anchored at the actual right edge of the message

column

  • [ ] User pill bg = --surface-elevated, border = subtle, radius ~12px,

max-width ≤540px, content fit

  • [ ] Assistant content visible (text or generative-UI card)

Card rendering (when a generative-UI shape fires)

  • [ ] Card chrome present: 1px border, soft shadow, 14px radius, no

side-stripe

  • [ ] Channel hairline color present (3-4px gradient strip on top edge)
  • [ ] Header row: verb chip (mono 11px) + channel chip (mono 10.5px) +

count if applicable

  • [ ] Body text 14px sans, line-height 1.6, color = --text (not

secondary)

  • [ ] Action row only present if the shape uses one (Approve/Edit/Reject

for DispatchCard)

  • [ ] No clipped content, no overflow

Layout quality

  • [ ] Composer at bottom with Queue pill (copper-filled, "Queue" label)
  • [ ] Mode chips inline on left of action-bar, model right-pushed
  • [ ] Mic button is a quiet outline (not copper-filled) when idle
  • [ ] Right panel headers in sentence-case sans, NOT uppercase mono
  • [ ] Sidebar agent rows compact, sentence-case section labels

Behavior

  • [ ] Dispatch completed (no "Unknown dispatch error" or stuck spinner)
  • [ ] Streaming actually streamed (not all-at-once dump)
  • [ ] Approve/Edit/Reject buttons on DispatchCard are clickable

If ALL pass → scenario is pass. If any fail → scenario is fail - <reason>. Don't half-pass.

Standard scenario set (ranked by render-richness)

Run as many as fit in budget. Each tests a different generative-UI shape; each is also a contract test for the producer skill that owns it:

  1. compose a short project update about today's snappy-os build session

DispatchCard with a message channel, counter fold, Approve/Edit/Reject. Producer: action-item-draft.

  1. show me what the agents did today → ProgressList. Producer:

agent-recap.

  1. what is the snappy-os bundle context → ContextPanel with connectors.

Producer: bundle-snapshot.

  1. open the file state/skills/desktop/SKILL.md → WorkingFolder with file

pill. Producer: desktop.

  1. give me feedback on the welcome screen design → FeedbackForm if

plumbed; else text.

  1. are you sure you want to fire snappy-verify? → ConfirmDialog if plumbed;

else text.

DO NOT run anything that side-effects externally - no real LinkedIn posts, no emails, no Slack sends. The default backend is openrouter / gemini so most replies are text-only unless the prompt explicitly invokes a generative shape via the dispatcher.

Failure modes - what to do when

SymptomCauseAction
dispatchInChatUI succeeds but composer never receives textPolling stopped or React un-mountedRestart snappy-chat: `pgrep -af /Applications/SnappyChat.app/Contents/MacOS/SnappyChat \awk '{print $1}' \xargs kill; sleep 1; open /Applications/SnappyChat.app`
Welcome surface stays after dispatchReact controlled-component setter didn't fireConfirm injectIntoComposer ran by reading App.tsx; check if textarea is display:none (offsetParent null on both variants)
Dispatch returns "Unknown dispatch error"/dispatch/chat 5xx, often dispatch.json mis-routedcurl -XPOST 127.0.0.1:3147/dispatch-config -d '{"chat":{"backend":"openrouter","model":"gemini"}}' to reset axis
Build fails on bash scripts/build-app.sh --installTS error or stale ResourcesRead the error; never bypass with --no-verify
Screenshot shows blank/whitepeekaboo targeted the wrong windowUse captureScreen (full display) instead of captureWindow
Stuck process at /private/tmp/.../Snappy OSOld build still runningpkill -f /private/tmp.*SnappyChat; sleep 0.5; open /Applications/SnappyChat.app
Multiple npx tsx processes piling upSubagent's previous calls leakedpkill -f "npx tsx" is safe - the desktop helpers are stateless

What goes where

  • Quick fix (≤25 LOC, ≤2 files, in snappy-os) → commit immediately

with chat: prefix and explicit pathspec.

  • Bigger fix (>25 LOC OR ≥3 files OR cross-repo) → append to

~/projects/snappy-os/state/log/snappy-chat-backlog.md as - [ ] <title> - <one-line scope>. Don't commit. Robert routes the larger work.

  • Reference observation (e.g. "the welcome state's spark is rendering

blurry at 22px") → also goes to backlog. Cosmetic findings batch better.

  • New skill needed (e.g. discovered we need a notification-toast

generative-UI shape) → write a finding in the backlog AND mention it in the loop report's "next" section.

Report format

/tmp/dogfood-loop-report.md:

## Scenario N: <intent>
- Pre: /tmp/loop-N-pre.png
- Post: /tmp/loop-N-post.png
- Card rendered: <name | "text-only" | "none">
- Render quality: pass | fail  -  <reason>
- Action: <commit SHA> | "no fix needed" | "logged to backlog"
- Notes: <one or two sentences>

## Summary
- Scenarios run: N
- Clean passes: M
- Commits: <SHAs>
- Findings to backlog: <count>
- Next biggest gap: <one sentence>

Hard constraints

  • PRODUCT.md tone is absolute. No emojis. No em dashes. No exclamation

points in user-visible copy.

  • DESIGN.md tokens are the only color source. Every styles.css value

pulls from existing OKLCH tokens.

  • Don't modify snappy-os skills/libs in a dogfood pass. That's a

separate axis.

  • Don't side-effect externally. No real outbound messages.
  • Don't run more than 6 minutes per scenario. If it eats more, log a

finding and move on.

Reference shots

Claude Desktop "Cowork" reference shots live in refs/cd-{01..23}.png - Robert's design target. See refs/README.md for the per-shot index. cd-01..06 = welcome state, cd-13/14/21 = thread state, cd-16..20 = customize / skills directory, cd-08..12 = projects + scheduled, cd-22..23 = artifacts.

Eval

Actor: the dogfood-running subagent. Captures screenshots, evaluates against the checklist, commits fixes, appends to report. Auditor: the next agent (or Robert) reads /tmp/dogfood-loop-report.md and verifies that scenarios actually passed by re-reading the post-shots. Distinct context = actor ≠ auditor.

OutcomeScore
Report present, ≥3 scenarios, every fail has a fix-or-backlog action1.0
Report present but a scenario flagged fail with no follow-through0.5
No report or report missing scenario shots / actions0.0

Rubric

criteria:
  - name: report_present
    kind: deterministic
    check: "/tmp/dogfood-loop-report.md exists and contains a Summary section."
  - name: scenarios_min_three
    kind: deterministic
    check: "Report contains at least three Scenario blocks, each with Pre, Post, Card rendered, Render quality, Action, Notes lines."
  - name: every_fail_has_action
    kind: judge
    check: "Every scenario marked 'fail' carries either a commit SHA in Action OR an entry appended to state/log/snappy-chat-backlog.md  -  no orphan failures."
  - name: no_external_sideeffects
    kind: deterministic
    check: "No scenario triggered a real outbound send (no LinkedIn URN, no Gmail draft, no Slack post). The dispatch backend stays text-only or generative-shape only."

AGENTS.md- what the AI loads when this skill comes up

dogfood-loop loader

Compact runtime loader for Snappy OS QA. The full contract lives in state/skills/dogfood-loop/SKILL.md; do not inject historical feedback logs or consolidated issue inventories into chat dispatch.

When invoked

Use the real /Applications/SnappyChat.app, not only /dispatch/chat. Drive it through POST 127.0.0.1:3147/chat-inject-push, reset with POST /chat-inject-control {"action":"reset"}, and capture screenshots from the running app. Screenshots are evidence; curl 200s are not enough.

Cadence

  1. Flush/reset the queue, then confirm the UI is ready.
  2. Push one concrete user intent.
  3. Wait for render completion or a visible failure.
  4. Capture the app screenshot.
  5. Grade: render integrity, card/canvas quality, layout quality, behavior.
  6. Fix narrow issues directly, redeploy, and re-run the same intent.
  7. If the fix is large or cross-repo, record a precise backlog item instead.

Five exchanges are required for a full dogfood pass when time allows. In an emergency, run the smallest loop that proves the reported failure is fixed.

Non-negotiables

  • Actor is not auditor: do not treat the thing that generated output as the

final grade. Independently inspect the screenshot and logs.

  • No blank right rail, empty canvas, placeholder, dead button, or silent

provider failure is acceptable demo behavior.

  • Do not perform real outbound side effects such as posting, sending email, or

firing production agents unless the prompt explicitly asks and apply:true is present.

  • Keep snappy-os fixes narrow. Build/install before claiming the UI is live:

bash ~/projects/snappy-os-app/apps/snappy-os/scripts/build-app.sh --install.

Useful commands

curl -sS -XPOST 127.0.0.1:3147/chat-inject-flush
curl -sS -XPOST 127.0.0.1:3147/chat-inject-control -H 'content-type: application/json' -d '{"action":"reset"}'
curl -sS -XPOST 127.0.0.1:3147/chat-inject-push -H 'content-type: application/json' -d '{"text":"<intent>","newThread":true}'
pgrep -af "/Applications/SnappyChat.app/Contents/MacOS/SnappyChat"
bash ~/projects/snappy-os-app/apps/snappy-os/scripts/build-app.sh --install

Report shape

Write the result plainly: intent, screenshot path, card/canvas rendered, pass or fail with reason, changed files or backlog item, and the next highest-risk gap.

api.ts- the code it can call

/**
 * snappy-dogfood-loop/api.ts -- typed scenario API for the dogfood loop.
 *
 * Today the dogfood-loop sidecar lives at state/bin/dogfood-loop/run.ts
 * and reads scenarios from a JSON file. This lib is the typed surface a
 * subagent (or any caller) should adopt instead of shell-driving scenarios.
 *
 * The loop runs autonomously — the curriculum from
 * feedback_loop_graduation is about NOT auto-promoting to cron until
 * trust is established, not about "the operator runs each tick by hand."
 * This lib provides the TYPED contract for any caller (subagent, cron,
 * test, ad-hoc).
 *
 *   import { runScenario, type Scenario } from "../lib/dogfood-loop.ts";
 *   const result = await runScenario({
 *     name: "linkedin-schedule",
 *     prompt: "show me linkedin schedule",
 *     waitMs: 14000,
 *     successPatterns: ["linkedin", "schedule"],
 *   });
 */

export interface Scenario {
  /** Short name for the scenario; used in screenshot filenames + reports. */
  name: string;
  /** Prompt to push into the cockpit chat. */
  prompt: string;
  /** Wait after dispatch before the post-shot. Default 14000ms. */
  waitMs?: number;
  /** Substrings to look for in the captured surface. */
  successPatterns?: string[];
  /** Substrings whose presence means failure (e.g., a stale-canvas marker). */
  failPatterns?: string[];
}

export interface ScenarioResult {
  scenario: Scenario;
  passed: boolean;
  preShot: string | null;
  postShot: string | null;
  /** Substrings actually found in the post-shot OCR / DOM dump. */
  matchedPatterns: string[];
  /** Substrings expected but not found. */
  missingPatterns: string[];
  durationMs: number;
  error?: string;
}

export interface RunOptions {
  /** Bridge endpoint for chat dispatch. Default 127.0.0.1:3147/chat-inject-push. */
  injectUrl?: string;
  /** Bridge endpoint for chat reset. Default 127.0.0.1:3147/chat-inject-pop (reset variant). */
  resetUrl?: string;
  /** Capture function. Returns path to captured PNG, or null on failure. */
  capture?: (label: string) => Promise<string | null>;
  /** OCR function. Reads a PNG path; returns text content. Default no-op. */
  readShot?: (path: string) => Promise<string>;
}

/**
 * Run one scenario. Pure plumbing — reset → pre-shot → push → wait → post-shot
 * → eval. Producers (chat-drive bridge, screencapture, OCR) are passed in;
 * the lib doesn't import them so this stays portable to a remote runner.
 */
export async function runScenario(scenario: Scenario, opts: RunOptions = {}): Promise<ScenarioResult> {
  const start = Date.now();
  const waitMs = scenario.waitMs ?? 14_000;
  const inject = opts.injectUrl ?? "http://127.0.0.1:3147/chat-inject-push";
  const capture = opts.capture ?? (async () => null);
  const readShot = opts.readShot ?? (async () => "");

  let preShot: string | null = null;
  let postShot: string | null = null;
  const matchedPatterns: string[] = [];
  const missingPatterns: string[] = [];

  try {
    preShot = await capture(`${scenario.name}-pre`);
    const r = await fetch(inject, {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({ text: scenario.prompt }),
    });
    if (!r.ok) {
      return {
        scenario,
        passed: false,
        preShot,
        postShot: null,
        matchedPatterns: [],
        missingPatterns: scenario.successPatterns ?? [],
        durationMs: Date.now() - start,
        error: `inject HTTP ${r.status}`,
      };
    }
    await new Promise((resolve) => setTimeout(resolve, waitMs));
    postShot = await capture(`${scenario.name}-post`);
    const text = postShot ? (await readShot(postShot)).toLowerCase() : "";
    for (const p of scenario.successPatterns ?? []) {
      if (text.includes(p.toLowerCase())) matchedPatterns.push(p);
      else missingPatterns.push(p);
    }
    let passed = (scenario.successPatterns ?? []).length === 0 || missingPatterns.length === 0;
    for (const p of scenario.failPatterns ?? []) {
      if (text.includes(p.toLowerCase())) {
        passed = false;
        missingPatterns.push(`(unexpected) ${p}`);
      }
    }
    return {
      scenario,
      passed,
      preShot,
      postShot,
      matchedPatterns,
      missingPatterns,
      durationMs: Date.now() - start,
    };
  } catch (e) {
    return {
      scenario,
      passed: false,
      preShot,
      postShot,
      matchedPatterns,
      missingPatterns,
      durationMs: Date.now() - start,
      error: String(e),
    };
  }
}

/**
 * Run a list of scenarios sequentially. Returns one result per scenario.
 */
export async function runAll(scenarios: Scenario[], opts: RunOptions = {}): Promise<ScenarioResult[]> {
  const out: ScenarioResult[] = [];
  for (const s of scenarios) {
    out.push(await runScenario(s, opts));
  }
  return out;
}

scripts- helper scripts it can run

prose-only skill - 4 inline code blocks live in SKILL.md above (no state/bin/ sidecar yet).

how we check it- the checks, plus the last 10 runs

rubric auto-shape no rubric declared
recent no runs actor/auditor: unverifiable
deps none declared

no recent runs logged - the eval contract is declared but nothing has been graded yet