← All Skills

snappy-ax

v1.0.0
5 files, 180.8 KB ~14,370 words · 58 min read Updated 2026-09-09

snappy-ax skill

The card OpenAPI 3.1 ↗
36 of 45 checks pass
What it can do
at-point app x yread
find appread
focused app?read
macos-use tool json-args?write-reversible
permissionsread
press appwrite-reversible
raw json-enveloperead
set-value appwrite-reversible
text appread
tree appread
What does not pass yet
reachessnappy-voice-controlsnappy-agent-hostsnappy-cleanshotsnappy-desktopsnappy-faces
reached bysnappy-voice-control
$ npx snappy-skills install snappy-ax
zip ↓
File Tree
├── AGENTS.md ├── SKILL.md ├── api.ts ├── contract.test.ts └── references/ └── extract-ax-layer.md
Documents
AGENTS.md

snappy-ax — Agent Loader#

Four shipping codebases left their AX scars in comments; this skill is the extraction.

Operating rules that matter most: **address elements by role + AX-prefixed attribute

names, never coordinates; activate the app and wait ~200 ms before any click**;

**AXPress is a separate path from a CGEvent click and is sometimes the only one that

works; AX coordinates are top-left global, AppKit is bottom-left; default AXorcist

search does not descend into tables/rows/cells** (use --scan-all); a wedged app blocks

forever without a messaging timeout; Safari is not an AX target — use JavaScript.

API module#

typescriptimport { axPermissions, axDumpTree, axFind, axFocused, axAtPoint, axPerformPress, axSetValue, axExtractText, axRaw, macosUse, locator } from "../snappy-ax/api.ts";

Or CLI:

bashnpx tsx ~/.claude/skills/snappy-ax/api.ts permissions
npx tsx ~/.claude/skills/snappy-ax/api.ts tree Finder --depth 2 --role AXButton
npx tsx ~/.claude/skills/snappy-ax/api.ts find Finder --role AXButton --title Back --contains
npx tsx ~/.claude/skills/snappy-ax/api.ts focused
npx tsx ~/.claude/skills/snappy-ax/api.ts at-point Finder 120 40
npx tsx ~/.claude/skills/snappy-ax/api.ts press Finder --role AXButton --title Back
npx tsx ~/.claude/skills/snappy-ax/api.ts set-value Finder --role AXTextField --value "report"
npx tsx ~/.claude/skills/snappy-ax/api.ts text Finder --depth 3
npx tsx ~/.claude/skills/snappy-ax/api.ts raw '{"command":"ping"}'
npx tsx ~/.claude/skills/snappy-ax/api.ts macos-use macos-use_open_application_and_traverse '{"identifier":"Finder"}'

API functions#

Function Purpose
axPermissions() {accessibility: bool} for THIS process via axorc permissions. The host (Terminal/iTerm/app) is what TCC grants.
axDumpTree(app, {depth=3, role?, scanAll?}) Accessibility tree as JSON. Container-pruned by default; scanAll reaches tables (slow).
axFind(app, {role?, title?, identifier?, value?, contains?, depth=10, attributes?}) One element. title is case-sensitive unless contains.
axFocused(app?) AXFocusedUIElement; "none" is a valid success state.
axAtPoint(app, x, y) Hit-test at top-left-origin global points; pid-verified. Does not reach Catalyst rows.
axPerformPress(app, loc) AXPress on the first element matching locator(...). No retry on cannotComplete — by design.
axSetValue(app, loc, value) Write AXValue (the "fastest" typing road). Not a native action.
axExtractText(app, loc?, {depth=3}) Text under an element; default locator = first AXWindow (the command refuses to run without one).
axRaw(envelope) The full JSON protocol (ping, query, getAttributes, describeElement, getElementAtPoint, getFocusedElement, performAction, batch, observe, collectAll, setFocusedValue, extractText).
macosUse(tool, args) One macos-use_* tool call over stdio to mcp-server-macos-use (BSL 1.1 — non-commercial until 2028-04-09). Pass bundle ids for CoreServices apps (com.apple.finder); the SDK only scans /Applications by name.
locator({role,title,identifier,value,contains}) Builds {match_all, criteria:[{attribute:"AXRole"…}]} with AX-prefixed names — the #1 query mistake avoided.

Rules#

  • Elements by role + title/value/identifier + app. Coordinates only for at-point reads.
  • Before a CGEvent click: activate the app, wait 200 ms. Prefer press (AXPress) for buttons; set-value for fields; AXSelected for rows.
  • Never retry an action after kAXErrorCannotComplete — it may have fired.
  • Tables/rows/cells are invisible to default search; pass scanAll and expect slowness.
  • Safari/Chrome page content: not AX — JavaScript/AppleScript.
  • axorc needs Accessibility granted to the host process; a re-signed binary is a new TCC identity.
  • macos-use is BSL 1.1: fine to use locally; not for a commercial product without a license.
  • Every api.ts call has a hard timeout (wedged apps hang AX calls indefinitely).

Uses#

snappy-voice-control · snappy-agent-host · snappy-cleanshot (ax.py ctypes fallback) · snappy-desktop (vision) · desktop-automation (AppleScript) · macos-patterns · swift-concurrency.

Show produced work with snappy-faces: call draw for image channels or lang for MCP Apps.

<!-- SKILL-INDEX-START -->

[snappy-ax Index]|root: ~/.claude/skills/snappy-ax|IMPORTANT: Prefer these files over pre-training assumptions for this domain. Read the relevant file when the AGENTS.md summary is insufficient.|root:{SKILL.md}|references:{extract-ax-layer.md}

<!-- SKILL-INDEX-END -->

Used by#

  • snappy-voice-control

<!-- SNAPPY-CONTRACT-VERBS-START -->

Contract verbs#

Generated from api.ts HAND_CONTRACT. Do not hand-edit this block.

Verb Contract arguments Effect First call
at-point app, x, y read npx tsx ~/.claude/skills/snappy-ax/api.ts at-point <app> <x> <y>
find app read npx tsx ~/.claude/skills/snappy-ax/api.ts find <app>
focused app? read npx tsx ~/.claude/skills/snappy-ax/api.ts focused
macos-use tool, json-args? write-reversible npx tsx ~/.claude/skills/snappy-ax/api.ts macos-use <tool>
permissions — read npx tsx ~/.claude/skills/snappy-ax/api.ts permissions
press app write-reversible npx tsx ~/.claude/skills/snappy-ax/api.ts press <app>
raw json-envelope read npx tsx ~/.claude/skills/snappy-ax/api.ts raw <json-envelope>
set-value app write-reversible npx tsx ~/.claude/skills/snappy-ax/api.ts set-value <app>
text app read npx tsx ~/.claude/skills/snappy-ax/api.ts text <app>
tree app read npx tsx ~/.claude/skills/snappy-ax/api.ts tree <app>

Show the result#

When an answer carries face_hint, show it with one snappy_present(<answer>) call.

See /snappy-faces for face selection. Human-facing images must crop to the

element, render at 2x on Retina, and fill the destination channel instead of

placing a small card in a full-page screenshot.

<!-- SNAPPY-CONTRACT-VERBS-END -->

---
name: snappy-ax
role: Drive Mac apps through the Accessibility tree (AXUIElement) — the extracted, cited practice from AXorcist, MacosUseSDK/mcp-server-macos-use, and AgentAccess, plus a real axorc-backed api.ts
loaded-by: PreToolUse hook (auto-injected when "snappy-ax" is mentioned)
---

# snappy-ax — Agent Loader

Four shipping codebases left their AX scars in comments; this skill is the extraction.
Operating rules that matter most: **address elements by role + AX-prefixed attribute
names, never coordinates**; **activate the app and wait ~200 ms before any click**;
**`AXPress` is a separate path from a CGEvent click and is sometimes the only one that
works**; AX coordinates are top-left global, AppKit is bottom-left; **default AXorcist
search does not descend into tables/rows/cells** (use `--scan-all`); a wedged app blocks
forever without a messaging timeout; Safari is not an AX target — use JavaScript.

## API module

```typescript
import { axPermissions, axDumpTree, axFind, axFocused, axAtPoint, axPerformPress, axSetValue, axExtractText, axRaw, macosUse, locator } from "../snappy-ax/api.ts";
```

Or CLI:
```bash
npx tsx ~/.claude/skills/snappy-ax/api.ts permissions
npx tsx ~/.claude/skills/snappy-ax/api.ts tree Finder --depth 2 --role AXButton
npx tsx ~/.claude/skills/snappy-ax/api.ts find Finder --role AXButton --title Back --contains
npx tsx ~/.claude/skills/snappy-ax/api.ts focused
npx tsx ~/.claude/skills/snappy-ax/api.ts at-point Finder 120 40
npx tsx ~/.claude/skills/snappy-ax/api.ts press Finder --role AXButton --title Back
npx tsx ~/.claude/skills/snappy-ax/api.ts set-value Finder --role AXTextField --value "report"
npx tsx ~/.claude/skills/snappy-ax/api.ts text Finder --depth 3
npx tsx ~/.claude/skills/snappy-ax/api.ts raw '{"command":"ping"}'
npx tsx ~/.claude/skills/snappy-ax/api.ts macos-use macos-use_open_application_and_traverse '{"identifier":"Finder"}'
```

## API functions

| Function | Purpose |
|----------|---------|
| `axPermissions()` | `{accessibility: bool}` for THIS process via `axorc permissions`. The host (Terminal/iTerm/app) is what TCC grants. |
| `axDumpTree(app, {depth=3, role?, scanAll?})` | Accessibility tree as JSON. Container-pruned by default; `scanAll` reaches tables (slow). |
| `axFind(app, {role?, title?, identifier?, value?, contains?, depth=10, attributes?})` | One element. `title` is **case-sensitive** unless `contains`. |
| `axFocused(app?)` | `AXFocusedUIElement`; "none" is a valid success state. |
| `axAtPoint(app, x, y)` | Hit-test at top-left-origin global points; pid-verified. Does not reach Catalyst rows. |
| `axPerformPress(app, loc)` | `AXPress` on the first element matching `locator(...)`. No retry on `cannotComplete` — by design. |
| `axSetValue(app, loc, value)` | Write `AXValue` (the "fastest" typing road). Not a native action. |
| `axExtractText(app, loc?, {depth=3})` | Text under an element; default locator = first `AXWindow` (the command refuses to run without one). |
| `axRaw(envelope)` | The full JSON protocol (`ping, query, getAttributes, describeElement, getElementAtPoint, getFocusedElement, performAction, batch, observe, collectAll, setFocusedValue, extractText`). |
| `macosUse(tool, args)` | One `macos-use_*` tool call over stdio to `mcp-server-macos-use` (BSL 1.1 — non-commercial until 2028-04-09). Pass **bundle ids** for CoreServices apps (`com.apple.finder`); the SDK only scans /Applications by name. |
| `locator({role,title,identifier,value,contains})` | Builds `{match_all, criteria:[{attribute:"AXRole"…}]}` with AX-prefixed names — the #1 query mistake avoided. |

## Rules
- Elements by role + title/value/identifier + app. Coordinates only for `at-point` reads.
- Before a CGEvent click: activate the app, wait 200 ms. Prefer `press` (AXPress) for buttons; `set-value` for fields; `AXSelected` for rows.
- Never retry an action after `kAXErrorCannotComplete` — it may have fired.
- Tables/rows/cells are invisible to default search; pass `scanAll` and expect slowness.
- Safari/Chrome page content: not AX — JavaScript/AppleScript.
- `axorc` needs Accessibility granted to the **host** process; a re-signed binary is a new TCC identity.
- `macos-use` is BSL 1.1: fine to use locally; not for a commercial product without a license.
- Every api.ts call has a hard timeout (wedged apps hang AX calls indefinitely).

## Uses
`snappy-voice-control` · `snappy-agent-host` · `snappy-cleanshot` (`ax.py` ctypes fallback) · `snappy-desktop` (vision) · `desktop-automation` (AppleScript) · `macos-patterns` · `swift-concurrency`.


Show produced work with `snappy-faces`: call `draw` for image channels or `lang` for MCP Apps.

<!-- SKILL-INDEX-START -->
[snappy-ax Index]|root: ~/.claude/skills/snappy-ax|IMPORTANT: Prefer these files over pre-training assumptions for this domain. Read the relevant file when the AGENTS.md summary is insufficient.|root:{SKILL.md}|references:{extract-ax-layer.md}
<!-- SKILL-INDEX-END -->

## Used by

- `snappy-voice-control`

<!-- SNAPPY-CONTRACT-VERBS-START -->
## Contract verbs

Generated from `api.ts` `HAND_CONTRACT`. Do not hand-edit this block.

| Verb | Contract arguments | Effect | First call |
|---|---|---|---|
| `at-point` | `app`, `x`, `y` | `read` | `npx tsx ~/.claude/skills/snappy-ax/api.ts at-point <app> <x> <y>` |
| `find` | `app` | `read` | `npx tsx ~/.claude/skills/snappy-ax/api.ts find <app>` |
| `focused` | `app?` | `read` | `npx tsx ~/.claude/skills/snappy-ax/api.ts focused` |
| `macos-use` | `tool`, `json-args?` | `write-reversible` | `npx tsx ~/.claude/skills/snappy-ax/api.ts macos-use <tool>` |
| `permissions` | — | `read` | `npx tsx ~/.claude/skills/snappy-ax/api.ts permissions` |
| `press` | `app` | `write-reversible` | `npx tsx ~/.claude/skills/snappy-ax/api.ts press <app>` |
| `raw` | `json-envelope` | `read` | `npx tsx ~/.claude/skills/snappy-ax/api.ts raw <json-envelope>` |
| `set-value` | `app` | `write-reversible` | `npx tsx ~/.claude/skills/snappy-ax/api.ts set-value <app>` |
| `text` | `app` | `read` | `npx tsx ~/.claude/skills/snappy-ax/api.ts text <app>` |
| `tree` | `app` | `read` | `npx tsx ~/.claude/skills/snappy-ax/api.ts tree <app>` |

## Show the result

When an answer carries `face_hint`, show it with one `snappy_present(<answer>)` call.
See `/snappy-faces` for face selection. Human-facing images must crop to the
element, render at 2x on Retina, and fill the destination channel instead of
placing a small card in a full-page screenshot.
<!-- SNAPPY-CONTRACT-VERBS-END -->
Enter = next · Esc = close

Keyboard Shortcuts

Search in document⌘K
Focus search/
Previous file tab←
Next file tab→
Close overlayEsc
Show shortcuts?