chat-drive

Push text into the Snappy OS composer from any agent, anywhere. The text flows through the same OpenUI submit path the human types into: processMessage → /dispatch/chat → AG-UI stream → generative-UI cards rendered in the live React tree. Same store, same surface, same eyes.

This is the missing primitive for closed-loop Snappy OS dogfood: an agent can now type intent and watch real cards stream in, then audit by screenshot.

What it's for

• Dogfood loops. A subagent pushes a stress-test intent, screenshots the

result, grades the rendered card. The actor (push) and the auditor (read the screenshot) are necessarily distinct - the contract holds for free.

• Automated UX QA. Verify the welcome surface unmounts on first message,

user-pill alignment, dispatch-card variants, etc., end-to-end through the rendered DOM.

• Recursive subagent dispatch. A long-running agent can re-enter the chat

surface mid-task by pushing a follow-up intent. The chat is the agent's outbox.

When NOT to use it

• Anything that doesn't need the rendered UI. If you don't care about the

React tree, call the head-screen server's /dispatch/chat directly, or use the dispatch skill. Running through the chat surface adds streaming latency for no reason.

• As a synthesis transport. The bridge is a queue, not an RPC channel -

there's no callback when streaming finishes. Use /dispatch/chat directly when you need the response programmatically.

Steps

1. Verify the head-screen server is up. The bridge endpoints live on it.

   bash ~/projects/snappy-os/state/bin/head-screen/launch.sh   # idempotent

1. Verify Snappy OS is running and on screen so the polled push lands somewhere.

  pgrep -af "/Applications/SnappyChat.app/Contents/MacOS/SnappyChat"

If it's not, build + install:

   cd ~/projects/snappy-os-app/apps/snappy-os && bash scripts/build-app.sh --install

1. Push the intent.

   npx tsx -e "
   import { dispatchInChatUI } from './state/lib/chat-drive.ts';
   await dispatchInChatUI('what did the agents do today', { waitForFirstFrame: 12000 });
   "

The default waitForFirstFrame is 8000ms. Pass a larger value when the target backend is slow (Claude Code: 12-15s; openrouter/gemini: 6-10s).

1. Audit by screenshot. The bridge has no completion callback - actor ≠ auditor.

   npx tsx -e "
   import { captureScreen } from './state/lib/desktop.ts';
   const path = await captureScreen('/tmp/chat-drive-verify.png');
   console.log(path);
   "

Then Read the PNG. Welcome surface unmounted + user pill on the right + assistant card streaming = bridge working.

Library API

state/lib/chat-drive.ts exports three functions. Importable from any TS agent code; also runnable as a CLI smoke.

export async function dispatchInChatUI(
  text: string,
  opts?: { waitForFirstFrame?: number }   // default 8000ms
): Promise<void>;

export async function resetChatUI(
  opts?: { waitMs?: number }               // default 1500ms
): Promise<void>;

export async function chatDriveAvailable(): Promise<boolean>;

resetChatUI is for multi-scenario dogfood loops: clears the thread and brings the welcome surface back so the next dispatchInChatUI lands in a clean state. Same FIFO as text pushes (single ordering), distinct /chat-inject-control endpoint so wire shape is unambiguous.

CLI:

npx tsx state/lib/chat-drive.ts "say hello in three words"

HEAD_SCREEN_URL env var overrides the default http://127.0.0.1:3147.

Architecture (one paragraph)

dispatchInChatUI POSTs to the head-screen server's POST /chat-inject-push endpoint, which appends the text to a 50-slot in-memory FIFO with FIFO eviction on overflow. The snappy-os React app mounts a polling effect that hits GET /chat-inject-pop every 500ms; on hit, it locates whichever OpenUI composer is currently visible (welcome OR thread variant) and writes through React's native value setter to trigger the textarea's onChange, then clicks the submit button. From there, the real processMessage path takes over - same code as a human typing.

For QA probes and dogfood agents, include "newThread": true in the push body. The app resets to a fresh chat before dispatching that item so probe traffic does not land in Robert's active thread. Omit it only when the intent is deliberately a follow-up in the current conversation.

Eval

Actor: dispatchInChatUI(text) - pushes onto the queue. Auditor: the audit harness re-reads the lib's exported function shape (present, async, two parameters); the user-facing audit is "is the rendered card correct" via screenshot, deliberately outside the lib.

Eval kind: shape. Mechanical: import the lib, assert dispatchInChatUI and chatDriveAvailable exist as functions, type-check passes. Logged as the skill's eval row in state/log/evals.ndjson.

Pitfalls

• The head-screen server must be alive. The bridge IS the head-screen

server. If /healthz doesn't answer, push will throw. chatDriveAvailable() is the cheap precheck.

• No completion callback. waitForFirstFrame is the only synchronization

knob. Tune it per backend, then screenshot.

• Restart drops queued pushes. The queue is in-memory by design - restart

= empty. If a dogfood loop relies on durability across restarts, you're using the wrong primitive.

• The bridge is loopback only. No external network exposure. The CORS

headers are wide so file:// origins (WKWebView) work; the listener is bound to 127.0.0.1.

Files

• state/lib/chat-drive.ts - the API (importable + CLI).
• state/bin/head-screen/server.ts - owns the queue endpoints

(POST /chat-inject-push, POST /chat-inject-control, GET /chat-inject-pop).

• ~/projects/snappy-os-app/apps/snappy-os/web/src/App.tsx - the React polling effect and

composer-injection helper that drains the queue.

chat-drive - loader

Push text into the Snappy OS composer. Routes through same OpenUI path as human typing: processMessage -> /dispatch/chat -> AG-UI stream -> live cards. Closed-loop dogfood: agent pushes intent, auditor screenshots result. Actor != auditor by design.

Full reference: state/skills/chat-drive/SKILL.md. Lib: state/lib/chat-drive.ts. Server: state/bin/head-screen/server.ts. Consumer: ~/projects/snappy-os-app/apps/snappy-os/web/src/App.tsx (poll 1000ms).

Critical Rules

1. Actor (push) ≠ auditor (screenshot). dispatchInChatUI() queues text; lib never reports "did card render." Audit via npx tsx state/lib/desktop.ts capture-screen /tmp/path.png + Read. No completion callback.
2. waitForFirstFrame is only sync knob. Default 8000ms. Budget: 8s dispatch + 1s React poll lag = 9s minimum. Tune per backend: Claude Code 12-15s, openrouter/gemini 6-10s. After push, screenshot.
3. Head-screen server MUST be alive. Pre-flight: chatDriveAvailable() or bash state/bin/head-screen/launch.sh (idempotent). Verify: curl http://127.0.0.1:3147/healthz answers 200.
4. Snappy OS must be running and visible. Push -> in-memory FIFO. No polling = silent eviction. Confirm: pgrep -af "/Applications/SnappyChat.app" shows process. Build if missing: cd ~/projects/snappy-os-app/apps/snappy-os && bash scripts/build-app.sh --install.
5. Queue is in-memory FIFO: 50-item cap, 30s TTL per item, wiped on server restart. Never durability-dependent. Pre-drain before QA: curl -XPOST http://127.0.0.1:3147/chat-inject-flush (returns {flushed:N}).
6. React polls /chat-inject-pop every 1000ms (App.tsx:337). Each push incurs up to 1000ms before composer sees it. Factor into waitForFirstFrame budget.
7. tsx never hot-reloads server.ts. After any server.ts edit, kill + restart the server. Verify: pgrep -af server.ts + git log --oneline -1 match. Without restart, edits invisible.
8. Don't push faster than dispatcher streams. Wait for RUN_FINISHED before next text push. Use resetChatUI() (not manual reset) between scenarios.
9. resetChatUI() does NOT flush. It sends control items for nav reset (welcome). Pre-flush with /chat-inject-flush if stale pushes queued.
10. Activate app before screenshot on secondary Space. Snappy OS off-screen = screencapture captures wallpaper. Run osascript -e 'tell application "Snappy Chat" to activate' with 1-2s wait.
11. React pre-fetches /chat-inject-pop items. Polling effect calls pop() before dispatching. Manual curl drain competes with React. Use /chat-inject-flush endpoint only; never manual pop drain.
12. Concurrent claude -p / claude --continue compete for queue. Symptom: pushes queue but never dispatch (other session consumed them). Mitigation: serialize QA sessions or use /dispatch/chat POST directly (isolated, independent thread).
13. Server crashes wipe in-memory queue. Watch for FATAL evalLeaderboardRegex undefined in state/log/head-screen.log. Verify uptime: ps -p $(pgrep -f server.ts | head -1) -o etime. If elapsed time reset, queue is gone; re-push after restart.
14. Force fresh crypto.randomUUID() for each messageId. Reusing any id (especially OpenUI's optimistic user-message id) collides in store reducer = duplicate-render bug. App.tsx processMessage must call crypto.randomUUID() per injected message. Do not remove.
15. Direct /dispatch/chat POST requires intent field. curl -XPOST http://127.0.0.1:3147/dispatch/chat -d '{"intent":"text","threadId":"<id>"}' (NOT flat messages[] array). Returns 400 intent required if missing.
16. TCC screencapture blocked in subagent context. If capture-screen fails with exit:1, use the bundled Computer/helper path or a live appshot as the audit surface; do not route through external helper daemons.
17. __snappyNav does NOT navigate to Live Apps. window.__snappyNav('#/chat/live-apps') navigates to the chat view, not the Live Apps surface. Do not use it for Live Apps navigation.
18. JS textarea injection does NOT submit messages. Injecting text into the OpenUI composer textarea via document.querySelector(...).value = ... or .dispatchEvent(new Event(...)) fills the field visually but does not trigger React's controlled-component submission handler. Use /chat-inject-push instead.
19. System Events keystrokes do NOT reach WKWebView. AppleScript System Events keystroke commands target native AppKit views; WKWebView content lives in a separate web process and does not receive synthetic key events this way.
20. Live Apps navigation requires custom events via /test-eval. The Live Apps surface listens for snappy:open-artifacts and snappy:live-apps-tab DOM events. Fire them via the /test-eval endpoint (see Commands). /chat-inject-control view-artifacts navigates to the artifacts sidebar tab, not the Live Apps surface.

Commands

WKWebView Hard Failures (do not retry these)

Self-Test

An agent reading this should correctly:

1. [ ] Pre-flight chatDriveAvailable() before push?
2. [ ] Tune waitForFirstFrame for backend + 1s React poll lag?
3. [ ] Audit by screenshot Read, NOT lib return values?
4. [ ] Keep both snappy-os and head-screen alive?
5. [ ] Use resetChatUI() (not manual) between scenarios?
6. [ ] Restart server after any server.ts edit (tsx no hot-reload)?
7. [ ] Serialize pushes; wait for RUN_FINISHED before next?
8. [ ] Drain /chat-inject-flush before QA tests?
9. [ ] Know React pre-fetches (no completion callback)?
10. [ ] Activate app via osascript before screenshot on secondary Space?
11. [ ] Know resetChatUI() doesn't flush (pre-flush if stale)?
12. [ ] Check pgrep claude.*-p for competing consumers?
13. [ ] Know server crashes wipe queue; verify uptime via ps?
14. [ ] Force fresh crypto.randomUUID() per messageId?
15. [ ] Use bundled Computer/appshot or local desktop capture for screenshots when needed.
16. [ ] Know TCC screencapture blocked in subagent context (use bridge)?
17. [ ] Know __snappyNav does NOT navigate to Live Apps?
18. [ ] Know JS textarea injection does NOT submit (use /chat-inject-push)?
19. [ ] Know System Events keystrokes do NOT reach WKWebView content?
20. [ ] Use snappy:open-artifacts event via /test-eval for Live Apps navigation?
21. [ ] Use snappy:live-apps-tab event via /test-eval to switch Live Apps tabs?

Found a gap? Edit this file. <!-- footer-injection-point -->