drop another .md file to compare - side-by-side diff against ai-models

ai-models

Connects your assistant to OpenAI so it can write, answer, and reason.

description: "Triggers on prompt mention of 'ai-models'."

personal 2 files 10 recent evals

Export

What it does for you

Connects your assistant to OpenAI so it can write, answer, and reason.

What it produces

A recent result, so you can see the kind of work it returns.

loading…

How to get it

These run inside the Snappy workspace. Want this working in your business? I set skills like this up with you, in one focused week.

Work with me

For developers how this skill is built, graded, and how it runs

at a glance- the short version

actorExported functions in state/lib/ai-models.ts.

auditorNone wired yet - eval is manual (Robert review).

eval modeshape

categorySystem

stages3

dependssettings

what's inside - the parts that make up a skill 3/4 present

A skill is just a few plain-text files. Only the main one is required. The rest are optional, added as the work needs them. This is what the skill is made of; how it runs is just below.

The skill

state/skills/ai-models/SKILL.md present

the skill itself, in plain text

The main file. It says what the skill is and lays out the steps in plain English.

Code

state/lib/ai-models.ts present

code the skill can run

Reusable code this skill can call when it needs to.

Scripts

state/bin/ai-models/ not present

helper scripts

Optional. Added when a skill has a few commands to run.

Loader

state/skills/ai-models/AGENTS.md present

what the AI loads on the fly

Loaded automatically the moment this skill is needed. Kept short on purpose.

how it's graded - what counts as a good run 4 criteria · 3 deterministic · 1 judge

Each row is one thing a good run has to get right. deterministic means a quick check decides, pass or fail. judge means the AI reads the result and rates it. Grading each piece on its own (instead of one overall score) shows exactly where a run fell short, so the fix is obvious.

name

kind

check

chat_completion_response_valid

deterministic

The output of chatCompletion() must be a valid JSON object matching the expected OpenAI chat completion schema.

embedding_output_structure

deterministic

The output of embed() must be an array of numbers, representing the embedding vector, or an object containing such a vector.

image_generation_url_present

deterministic

The output of generateOpenAIImage() must contain a URL pointing to the generated image.

api_error_handling

judge

The skill should gracefully handle common OpenAI API errors (e.g., rate limits, invalid input, authentication failures) and return informative error messages or codes.

how it runs - the shared frame every skill uses 5/5 present

Every skill runs the same way. One part does the work, a separate part checks it, and a short loader hands the AI exactly what it needs for the job. Anything this skill doesn't use shows a one-line note saying why, on purpose, not by accident.

makes the work The worker

present

Exported functions in state/lib/ai-models.ts. the worker

Does the actual work. Whatever it produces is what gets checked next.

checks the work The reviewer

present

None wired yet - eval is manual (Robert review). the checker

A separate checker grades the work, so the part that made it can't approve its own work.

frame

learns Self-correction

present

fixes itself learns from gaps

When a run hits a gap, the skill gets edited on the spot [FIXED] or queued for a bigger rewrite [LOGGED], so it keeps getting better.

tidies up Background fixes

present

queued for rewrite runs in the background

Bigger fixes that can't be made on the spot get queued and rewritten in the background later.

remembers Run history

present

state/log/pending-eval.ndjson pending runs

Every run is written down here, then reviewed by hand each week.

Critical rules the things this skill must not get wrong

ALWAYS prefer state/lib/ai-models.ts (chatCompletion, embed, generateOpenAIImage) over rolling a fresh OpenAI client
For image generation Robert prefers Nano Banana via state/bin/image/; only use generateOpenAIImage if that's explicitly the brief
For cheap-labor dispatch use state/lib/dispatch.ts (pi gateway), not OpenAI directly

what it has learned - fixes written back in over time sample

When a run hits something this skill didn't handle, the fix gets written back into the skill so it doesn't happen again. FIXED means it was corrected on the spot. LOGGED means it's queued for a bigger rewrite. Either way, the skill gets a little better and never makes the same mistake twice.

Loading feedback rows…

how the work flows- who makes it, who checks it

inputs settings

actor Exported functions in state/lib/ai-models.ts.

1 generator

invoke

actor = Exported functions in state/lib/ai-models.ts.

import { chatCompletion, embed, generateOpenAIImage } from "state/lib/ai-models.ts"

+ eval for this step

auditor None wired yet - eval is manual (Robert review).

2 auditor

inspect

auditor = None wired yet - eval is manual (Robert review).

npx tsx -e 'import("/Users/robertboulos/projects/snappy-os/state/lib/ai-models.ts").then(m => console.log(Object.keys(m)))'

+ eval for this step

3 data

eval log

`state/log/pending-eval.ndjson` (skill: "ai-models")

+ eval for this step

SKILL.md- the skill, written out in plain English

ai-models

Direct OpenAI API for all snappy-* skills.

Ported from kernel snappy-ai-models in Phase 0.5. See state/lib/ai-models.ts for the full API surface.

Steps

chatCompletion() - see state/lib/ai-models.ts
embed() - see state/lib/ai-models.ts
generateOpenAIImage() - see state/lib/ai-models.ts

Eval

Actor: the exported functions in state/lib/ai-models.ts. Auditor: none wired yet - eval is manual (Robert review). File a state/log/pending-eval.ndjson row on each run.

Score convention:

Outcome	Score
Pass on first try	1.0
Failed first, auto-fix applied, re-check passed	0.5
Still failing or unrecoverable	0.0

Gotchas

via the Phase 0.5 driver. Only these rewrites were applied: already in state/lib/)

realpathSync(process.argv[1]) CLI guard wrapped in try/catch

See the kernel SKILL.md for the original long-form guidance if you need it

(read-only reference at the kernel path above).

Graduation

This skill is prose. Graduate by defining a deterministic auditor and flipping eval: auto.

Rubric

criteria:
  - name: chat_completion_response_valid
    kind: deterministic
    check: "The output of chatCompletion() must be a valid JSON object matching the expected OpenAI chat completion schema."
  - name: embedding_output_structure
    kind: deterministic
    check: "The output of embed() must be an array of numbers, representing the embedding vector, or an object containing such a vector."
  - name: image_generation_url_present
    kind: deterministic
    check: "The output of generateOpenAIImage() must contain a URL pointing to the generated image."
  - name: api_error_handling
    kind: judge
    check: "The skill should gracefully handle common OpenAI API errors (e.g., rate limits, invalid input, authentication failures) and return informative error messages or codes."

AGENTS.md- what the AI loads when this skill comes up

ai-models - loader

Per-turn rules for the ai-models skill. Full reference: state/skills/ai-models/SKILL.md. Do not skip these.

Critical Rules

_(no failures recorded yet - Phase 0.5 mechanical port from kernel snappy-ai-models. Read state/skills/ai-models/SKILL.md and state/lib/ai-models.ts before invoking.)_

ALWAYS prefer state/lib/ai-models.ts (chatCompletion, embed, generateOpenAIImage) over rolling a fresh OpenAI client
For image generation Robert prefers Nano Banana via state/bin/image/; only use generateOpenAIImage if that's explicitly the brief
For cheap-labor dispatch use state/lib/dispatch.ts (pi gateway), not OpenAI directly

Commands

| ui dashboard | state/skills/ai-models/resources/ui.openui | |invoke: import { chatCompletion, embed, generateOpenAIImage } from "state/lib/ai-models.ts" |verify: npx tsx -e 'import("/Users/robertboulos/projects/snappy-os/state/lib/ai-models.ts").then(m => console.log(Object.keys(m)))' |eval log: state/log/pending-eval.ndjson (skill: "ai-models")

OpenUI Resource

Skill-owned OpenUI Lang resource: state/skills/ai-models/resources/ui.openui. Read it before rendering or editing this skill's generated component surface.
Treat this resource as a first-class artifact of the skill, not a generic chat response. Improve it when the skill's user-facing output needs to become richer.
System resources compose OpenUI primitives and inherit SnappyChat tokens. Use ui_contract: branded in SKILL.md only for deliberate platform or client visuals.

Known Pitfalls

Phase 0.5 port - settings/load path patched to ./env.ts, no behavior changes
Credentials read via env("OPENAI_API_KEY") - never hardcode

Self-Test

An agent reading this should correctly:

[ ] Use the lib functions, not raw fetch() to OpenAI
[ ] Pick Nano Banana / state/bin/image/ for image work unless told otherwise
[ ] Read the API key via env(), not bash fallback

Self-report

If this loader fell short, append a line:

echo "[$(date -u +%FT%TZ)] ai-models: <what was missing>" >> state/log/loader-feedback.log

Self-correcting loader (PID feedback)

This is the rule that makes loaders learn. Ported from snappy-kernel, where it demonstrably worked: the snappy-dom-cartographer skill mapped Skool admin UI once, every later ship-agent inherited the full map for free.

Before you finish, do two things:

1. Fix gaps (P - proportional). MANDATORY. If this loader didn't cover your case - if you had to read another file, run an undocumented command, work around a wrong selector, or discover a quirk - you MUST attempt an Edit to this AGENTS.md before you log.

Default action: edit this file. Surgical. Add the missing line, table

row, or code example where it belongs. No restructuring.

Only LOGGED is allowed when: the fix needs >10 lines, spans multiple

files, or requires a structural rewrite. In that case the state/regen/drain.sh queue picks it up asynchronously.

"I didn't have time" / "it's minor" / "the next agent will figure it out"

are NOT valid reasons. Minor gaps compound - that's the whole point of this loop.

Concrete example of a valid P-fix: discover agent-browser screenshot

takes path positionally not as --output <path>. One line in a table. Edit it in. Do not just log it.

The goal: the next agent never has to leave the loader.

2. Log the result.

echo "[$(date -u +%FT%TZ)] ai-models: <what was missing or fixed> [FIXED|LOGGED]" >> state/log/loader-feedback.log

FIXED = you patched this loader inline (P-fix).
LOGGED = too large for inline; the PostToolUse enqueue + Stop-hook drain

will rewrite the loader from scratch on next session-end.

Do not skip this. Every agent run must leave the system better than it found it. The loader is the setpoint; you are the sensor; the gap is the error signal; closing the gap is the correction.

api.ts- the code it can call

#!/usr/bin/env npx tsx
/**
 * snappy-ai-models/api.ts -- Direct OpenAI API for all snappy-* skills.
 *
 * Uses OPENAI_API_KEY from snappy-settings/.env.cache.
 * Direct OpenAI REST API -- chat completions, embeddings, image generation.
 *
 * Usage:
 *   npx tsx api.ts chat "Explain quantum tunneling"
 *   npx tsx api.ts chat "Summarize this" --model gpt-4o
 *   npx tsx api.ts embed "What is Xano?"
 *
 * Or import as module:
 *   import { chatCompletion, embed, generateOpenAIImage } from "./ai-models.ts";
 */

import { env } from "./env.ts";
import { realpathSync } from "fs";

const BASE = "https://api.openai.com/v1";
const DEFAULT_MODEL = "gpt-4o-mini";

interface ChatOptions {
  model?: string;
  systemPrompt?: string;
  temperature?: number;
  maxTokens?: number;
}

interface ImageOptions {
  model?: string;
  size?: string;
}

async function openai(path: string, body: Record<string, unknown>): Promise<unknown> {
  const res = await fetch(`${BASE}${path}`, {
    method: "POST",
    headers: {
      Authorization: `Bearer ${env("OPENAI_API_KEY")}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify(body),
  });
  if (!res.ok) {
    const err = await res.text();
    throw new Error(`OpenAI failed (${res.status}): ${err}`);
  }
  return res.json();
}

// --- Public API ---

export async function chatCompletion(prompt: string, opts: ChatOptions = {}): Promise<{ text: string; raw: unknown }> {
  const model = opts.model || DEFAULT_MODEL;
  const messages: { role: string; content: string }[] = [];
  if (opts.systemPrompt) messages.push({ role: "system", content: opts.systemPrompt });
  messages.push({ role: "user", content: prompt });

  const data = await openai("/chat/completions", {
    model,
    messages,
    ...(opts.temperature != null ? { temperature: opts.temperature } : {}),
    ...(opts.maxTokens ? { max_tokens: opts.maxTokens } : {}),
  }) as any;
  const text = data.choices?.[0]?.message?.content || "";
  return { text, raw: data };
}

export async function embed(text: string, model = "text-embedding-3-small"): Promise<{ vector: number[]; raw: unknown }> {
  const data = await openai("/embeddings", {
    model,
    input: text,
  }) as any;
  const vector = data.data?.[0]?.embedding || [];
  return { vector, raw: data };
}

export async function generateOpenAIImage(prompt: string, opts: ImageOptions = {}): Promise<{ url: string; raw: unknown }> {
  const model = opts.model || "gpt-image-1";
  const data = await openai("/images/generations", {
    model,
    prompt,
    size: opts.size || "1024x1024",
    n: 1,
  }) as any;
  const url = data.data?.[0]?.url || data.data?.[0]?.b64_json || "";
  return { url, raw: data };
}

// --- CLI ---

if ((() => { try { return import.meta.url === `file://${realpathSync(process.argv[1])}`; } catch { return false; } })()) {
  (async () => {
    const [, , cmd, ...args] = process.argv;

    switch (cmd) {
      case "chat": {
        const prompt = args.filter(a => !a.startsWith("--")).join(" ");
        const modelIdx = args.indexOf("--model");
        const model = modelIdx >= 0 ? args[modelIdx + 1] : undefined;
        if (!prompt) { console.error("Usage: api.ts chat <prompt> [--model <model>]"); process.exit(1); }
        const { text } = await chatCompletion(prompt, { model });
        console.log(text);
        break;
      }
      case "embed": {
        const text = args.join(" ");
        if (!text) { console.error("Usage: api.ts embed <text>"); process.exit(1); }
        const { vector } = await embed(text);
        console.log(`Dimensions: ${vector.length}`);
        console.log(JSON.stringify(vector.slice(0, 5)) + "...");
        break;
      }
      case "image": {
        const prompt = args.filter(a => !a.startsWith("--")).join(" ");
        const sizeIdx = args.indexOf("--size");
        const size = sizeIdx >= 0 ? args[sizeIdx + 1] : undefined;
        if (!prompt) { console.error("Usage: api.ts image <prompt> [--size <WxH>]"); process.exit(1); }
        const { url } = await generateOpenAIImage(prompt, { size });
        console.log(url);
        break;
      }
      default:
        console.log("Usage: npx tsx api.ts [chat|embed|image] ...");
    }
  })();
}

scripts- helper scripts it can run

prose-only skill - 1 inline code block live in SKILL.md above (no state/bin/ sidecar yet).

how we check it- the checks, plus the last 10 runs

rubric shape schema-shape check (no inline rubric)

recent mean 1.00 · 10 runs actor/auditor: unverifiable

deps settings

timestamp	verb	score	primary_issue	artifact
2026-04-25 04:11Z	-	1.00	-	-
2026-04-21 15:58Z	-	1.00	-	-
2026-04-21 15:56Z	-	1.00	-	-
2026-04-21 03:53Z	-	1.00	-	-
2026-04-25 04:11Z	-	1.00	-	-
2026-04-21 15:58Z	-	1.00	-	-
2026-04-21 15:56Z	-	1.00	-	-
2026-04-21 03:53Z	-	1.00	-	-
2026-04-25 04:11Z	-	1.00	-	-
2026-04-21 15:58Z	-	1.00	-	-