document-summarize

Handles uploaded document attachments in snappy-os. Extracts text server-side and lets the LLM summarize, analyze key points, answer questions, render tables, and compare documents.

Rubric

criteria:
  - name: text_extraction_complete
    kind: deterministic
    check: "parseDocumentFromBase64(name, mimeType, base64) completes and returns {success: true, text, pages?, rows?, error: null} with text.length > 0 (non-empty extracted content)."
  - name: text_not_truncated_alone
    kind: deterministic
    check: "When text exceeds 12000 chars, truncation includes a note: '... [truncated at 12000 chars]'. Not truncated ≠ passing  -  must have actual extracted content."
  - name: context_injected
    kind: deterministic
    check: "<document name=\"...\" [N page(s)]>extracted text</document> block is prepended to promptForLLM before LLM dispatch."
  - name: followup_context_preserved
    kind: judge
    check: "Follow-up questions include the prior assistant summary in prior_conversation block so the LLM can reference it."

Supported formats

Install requirements

All parsers are installed in ~/projects/snappy-os:

cd ~/projects/snappy-os
npm install pdf-parse mammoth csv-parse
# optional:
npm install xlsx

Architecture

1. file-attach-button.tsx - user drops/selects PDF/DOCX/XLSX/CSV:

• Binary files read as ArrayBuffer → btoa() → base64 string
• Encoded as [Document: name.pdf]\ndata:application/pdf;base64,<data>\n\n
• Added to pendingAttachment.content (same as text/image path)

1. server.ts (POST /dispatch/chat):

• Extracts [Document: ...] blocks via regex (parallel to image extraction)
• Calls state/lib/document-parse.ts → parseDocumentFromBase64()
• Builds <document name="..." [N page(s)]>extracted text</document> blocks
• Injects into promptForLLM as: "The user has attached the following document(s)...\n\n<document>...\n\nUser request: <intent>"

1. state/lib/document-parse.ts - document parsing library:

• parseDocumentFromBase64(name, mimeType, base64) → DocumentParseResult
• Truncates to 12000 chars to stay within LLM context budget
• CLI self-test: npx tsx state/lib/document-parse.ts <path-to-file>

Steps

1. User attaches a document via the paperclip button or drag-drop in snappy-os.
2. file-attach-button.tsx detects binary doc (isBinaryDoc()) → reads as ArrayBuffer → encodes as [Document: ...] marker.
3. User types intent ("summarize this", "what are the key points", etc.) and sends.
4. App.tsx prepends pendingAttachment.content to the intent string before dispatch.
5. server.ts extracts the [Document: ...] block, calls parseDocumentFromBase64().
6. Extracted text is wrapped in <document> tags and prepended to promptForLLM.
7. LLM receives clean prompt with document context; responds with summary/analysis.
8. For follow-up questions (step 5-7 in the flow), prior conversation history includes the assistant's previous summary so the LLM can reference it.

Key points / limitations

• PDF text extraction requires selectable text (scanned/image PDFs will yield empty or garbled text).
• Max 12000 chars extracted per document; large documents are truncated with a note.
• XLSX requires npm install xlsx separately (not installed by default).
• For .docx files, only raw text is extracted (no table structure, no images).
• CSV files render as a markdown table (max 200 rows; excess noted).
• "Compare this PDF to the previous one" works via conversation history - the previous summary is in the prior_conversation block.
• "Extract the table on page 3 as a Lang Table" works for PDFs with selectable table text; the LLM will structure it using OpenUI Table primitive.

Testing

# Verify PDF parsing works:
npx tsx state/lib/document-parse.ts /path/to/test.pdf

# Verify CSV:
npx tsx state/lib/document-parse.ts /path/to/test.csv

# Verify DOCX:
npx tsx state/lib/document-parse.ts /path/to/test.docx

# Verify the server extracts documents (direct API):
curl -X POST 127.0.0.1:3147/dispatch/chat \
  -H "Content-Type: application/json" \
  -d "{\"intent\":\"[Document: test.pdf]\\ndata:application/pdf;base64,$(base64 -i /path/to/test.pdf | tr -d '\\n')\\n\\nSummarize this.\"}" \
  | grep -o '"delta":"[^"]*' | head -5

document-summarize - loader

Per-turn rules for document attachment + summarization in snappy-chat. Full skill: state/skills/document-summarize/SKILL.md. Integration skill, eval=auto-shape. No executable verbs; parser at state/lib/document-parse.ts.

Critical Rules

1. Binary docs use ArrayBuffer, not readAsText. readAsText() on PDF/DOCX/XLSX = garbage. Route via isBinaryDoc() → readAsArrayBuffer() → base64.
2. Encoding: [Document: name]\ndata:<mime>;base64,<data>\n\n. Server regex extracts via [Document:([^\]]+)]\ndata:([^;]+);base64,([^\n]+) - exact format or silent fail.
3. Parser at state/lib/document-parse.ts. Import: await import("../../lib/document-parse.ts"). Test: npx tsx state/lib/document-parse.ts <file>.
4. PDF: PDFParse class, not default import. Correct: import { PDFParse } from 'pdf-parse'. Wrong: import pdfParse from 'pdf-parse'.
5. mammoth: extractRawText({ buffer: buf }) → { value: string }. Import: import { extractRawText } from 'mammoth'.
6. csv-parse: sync path, { columns: true, skip_empty_lines: true, relax_column_count: true }. Renders as markdown table.
7. LLM context: <document name="..." [N page(s)]>...text...</document>. Prepended with "The user has attached the following document(s)...".
8. Empty intent allowed. Defaults to "Please summarize this document."
9. 12000 char truncation intentional. Max per-document for LLM context budget. Suffix: [...document truncated...].
10. Follow-ups via history. No re-attach needed; prior conversation includes prior summary for context.
11. XLSX requires separate install. npm install xlsx.
12. Drag-drop same as click. useFileDrop hook routes via readAsArrayBuffer().

Commands

Test commands: npx tsx ~/projects/snappy-os/state/lib/document-parse.ts <file.pdf|csv|docx> | Verify server: curl -s -X POST 127.0.0.1:3147/dispatch/chat -d '{"intent":"[Document: test.pdf]\ndata:application/pdf;base64,...\n\nSummarize."}' | grep delta

Self-Test

• [ ] Binary docs use readAsArrayBuffer(), not readAsText()
• [ ] Encoding: [Document: name]\ndata:<mime>;base64,...
• [ ] Parser at state/lib/document-parse.ts
• [ ] PDF uses PDFParse class import
• [ ] mammoth uses extractRawText({ buffer })
• [ ] csv-parse uses sync path with correct options
• [ ] 12000 char truncation is context budget (intentional)
• [ ] Follow-ups work via conversation history (no re-attach)
• [ ] XLSX needs separate npm install
• [ ] Drag-drop routes via readAsArrayBuffer

Found a gap? Edit this file. <!-- footer-injection-point -->