Format specification

A response is Markdown with a small set of components written like JSX. Models already write both, so the format needs little teaching, and a half-written response is always a valid prefix of a finished one.

The catalog is the only definition of the components. The system prompt, the validator and the renderers all read it, so they always agree.

#Catalog: packs and custom components

A catalog is built from packs (spec/packs/<name>.json) and an app's own components, in that order:

• core is always included: layout, text blocks, numbers, charts, forms and buttons. The API and the SDKs add every other pack unless an app names the ones it wants (all names every pack; [] is core alone, for a prompt about half as long: 1,400 tokens against 2,900, about 22 a component).
• An app's own components use the same JSON as a pack, or come from a JSON Schema of their props (Zod 4: z.toJSONSchema). Strings, numbers, booleans, string enums, URLs and arrays of strings map to attributes; other shapes are rejected, since a model can't write them in an attribute.
• A name defined twice is an error, never an override, so an app can't replace a built-in component by accident. The one exception: a pack that comes in only through the default (all) gives way to the app's own, so an app's own Product or Map replaces the pack's; a pack the app names outright still clashes. Definitions are checked when the catalog is built: names, content kinds, attribute types, enum defaults, parents, alias targets, and that tone uses the shared values.
• Each renderer draws the components it knows. An app registers renderers for its own; a component with none shows its attributes as a list and then its content, so nothing the model wrote is lost.

#Document

• The document is the root. There is no root statement to forget and no names to wire up: a component's place in the tree is where it's written.
• Text between components is Markdown.
• Anything the model writes before or after the UI (a sentence of intro, for example) is just Markdown and renders as text.

#Markdown subset

CommonMark plus GFM tables, limited to: ATX headings (# to ######), paragraphs, emphasis and strong, inline code, links, images, unordered and ordered lists, block quotes, fenced code, thematic breaks and tables.

Images:

• ![what it shows](find) asks for a photo, when the app gave the API its own Pexels, Unsplash or Openverse key. The API searches that provider for what the alt text names and rewrites the image to the provider's address before the answer reaches the app. People and the answer's own made-up things (an app's dashboard, a listing's rooms) get none. A request nothing answers keeps find, and renderers show its description in a tile.
• ![what it shows](img:ID) shows one of the app's own pictures, given with the request; the API puts its address in.
• An image's title is its credit line, as inline Markdown with links: ![Belém Tower](https://images.pexels.com/... "[Author](page), [Pexels License](license), via Pexels"). Renderers show it under the image.
• Renderers load images only from the hosts the app allows. Any other address, model-written URLs included, shows the description (an avatar its initials): loading it would tell that host who is reading, and a URL can carry what the page holds.
• Link and image addresses may hold balanced parentheses (https://en.wikipedia.org/wiki/Mercury_(planet)), as in CommonMark.

Raw HTML is never rendered as HTML, but models write some anyway (<strong> and <br> in about one answer in ten), so it is read as formatting:

• <br> is a line break, <p> and <div> are paragraph breaks (line breaks inside a table cell), <h1> to <h6> are headings.
• <strong>/<b>, <em>/<i>, <u>, <s>/<del>, <code>, <small>, <sub>, <sup>, <mark> and <kbd> keep their meaning. Their attributes are dropped and unclosed ones are closed where the text ends.
• <span>, <font>, <a> and other layout tags are dropped; their text stays.
• Anything else, and anything inside code, is text.

#Components

<Name attr="value" other=42 flag>content</Name>
<Name attr="value" />

• A component name starts with an uppercase letter. <div> or <br> is text.
• Attribute values are quoted strings. Numbers and true/false may be unquoted; JSX-style {value} is accepted and unwrapped. A bare attribute (required) means true.
• List attributes separate values with |: options="S | M | L".
• Components whose content is none are void, like HTML's <br>: they never need closing, and a stray </Input> is ignored.
• A line that opens with a known component but isn't a tag the core can read (<Stat label! value!>, the prompt's signature copied in; subtitle "Eno" without its =; a value with no name) shows as text, and is an unreadable-tag error on its own line, not only the stray closing tags it leaves behind.
• Only the attributes listed in the catalog exist. ! in the prompt marks a required attribute; an enum lists every allowed value, the first being the default.
• An attribute name means the same thing on every component, with the same values: tone is always neutral, info, success, warning or danger. (In v0.1 Stat's tones lacked info and Callout's lacked neutral; models wrote the missing one in 8 of 10 answers.)

#Aliases

Some names models write on their own for something that exists: <Table> (a section holding a table), <Column> (one cell of Columns) and <Buttons> (a row of buttons). The catalog lists them under aliases; they are read as the component they mean and reported as a note, never taught in the prompt.

#Content kinds

A component with a parent (like Tab) is only valid inside that parent; a Result may sit anywhere inside its WhatIf (in a Columns, say). A text component marked "actions": true (Product) also takes Buttons inside, drawn under it as what the reader can do with it ("Add to cart"); nothing else goes inside it.

#Inline components

Any component may sit inside a Markdown table cell (models put badges, avatars and buttons in cells), and Badge and Button may also appear inside a sentence. They render in place without breaking the table.

#Emojis

Renderers strip emojis from everything a model writes (©, ® and ™ stay), and the prompt asks for none.

#Device pack

spec/packs/device.json reaches the device's own hardware and apps. Each component is native on every platform, and a web page without JavaScript gets a fallback that still works:

Rules that hold on every platform:

• The model only asks. Nothing is captured, read or sent until the user acts: a tap opens the camera or shares the location, and values leave the device only when the user submits the form they belong to.
• The operating system's own permission prompts apply; a renderer never works around a refused permission, it falls back to the typed field.
• Location defaults to approximate. Precise location needs precision="precise" and the platform's precise-location permission.
• Device components are form fields: their values are submitted with the form, photos and files as multipart uploads, a location as coordinates.

#Streaming

The format is designed so the core only ever reads the new text and never re-reads what it has already processed.

A value is committed only once it is complete. This one rule replaces the per-component patches OpenUI needed (step arriving last, cut-short URLs, truncated $state defaults, max arriving as "800" before "8000"):

• A component appears when its opening tag is complete (> received), with every attribute final.
• Text streams word by word.
• A table row counts at its newline. A chart draws when its closing tag arrives and shows a placeholder of the same size until then.
• Numbers, enum values and URLs are only ever read in full.

Finished nodes never change. A closed component is final, so renderers never redraw it. Nodes are identified by their position in the document, never by an array index that can shift.

One parser. Parsing a whole response is streaming it in one chunk; the test suite checks that every split of every recording gives the same tree.

#Validation and repair

Lenient where the intent is clear, strict where it isn't:

• Enum values match without case, and a few common synonyms are accepted (error means danger, column means bar). These are reported as notes.
• An invalid value is resolved at its edge (OpenUI's #729 rule): use the default if there is one; otherwise drop an optional attribute, or invalidate the component if the attribute is required. A bad table row drops only that row.
• An unknown component is dropped and its content kept as text.
• Every problem is reported as a diagnostic tied to its line, so a repair pass can rewrite just that line. Truncation is reported, never silent.

#Output: the op stream (v1)

The core turns chunks of model text into a stream of operations, one JSON object per line (served as SSE data: lines by the API). Every renderer, on every platform, only applies ops; none of them parses the format.

#Nodes

The document is a tree. A node's id is its position: the indexes from the root joined with dots ("3", "3.0", "3.0.2"). Ids never shift, because nodes only ever arrive after the ones before them.

#Ops

data by content kind:

• table (Chart, Map): {"columns": [...], "rows": [[...], ...]}, cells as the model wrote them (renderers parse numbers for charts).
• list (FollowUps): {"items": [spans, ...]}.
• images (Gallery): {"images": [{"src": "...", "alt": "...", "credit": [spans]}]} (credit only when the image has one).
• A Question (interactive pack): {"items": [spans, ...], "answer": index or null}; the core matches answer to a choice by its letter, number or words.
• A Checklist: {"checks": [{"done": true or false, "c": [spans]}]} ([x] marks done).
• A What-if Result: {"formula": [tokens], "value": number or null}: the formula in postfix tokens (numbers, the Sliders' names, + - * / ^ %, neg, min, max, round, floor, ceil, abs, sqrt) and its value at the Sliders' starting values. A Slider's name is its name, or its label as one word ("Unit price ($)" is unit_price). Renderers run the tokens on a stack as the Sliders move; nothing else runs.

#Spans

Inline content is an array of spans:

The core strips emojis (keeping ©, ® and ™), reads the HTML models slip in as formatting, and keeps a dash between words from starting a line, so every renderer shows the same text.

#Example

{"op":"open","id":"0","type":"h","attrs":{"level":"1"}}
{"op":"spans","id":"0","spans":[{"t":"text","v":"Revenue overview"}]}
{"op":"close","id":"0"}
{"op":"open","id":"1","type":"Card","attrs":{"title":"Weekly revenue"}}
{"op":"open","id":"1.0","type":"Chart","attrs":{"type":"line","x":"Week","y":"USD"}}
{"op":"close","id":"1.0","data":{"columns":["Week","Revenue"],"rows":[["W1","24100"],["W2","30800"]]}}
{"op":"close","id":"1"}
{"op":"end","complete":true,"truncated":false}

#Actions: what the app sends back (v1)

Components never run code. When the user acts, the renderer sends an action, and the API turns it into the model's next turn.

fields is keyed by each field's name (or its label as a slug when it has none):

Photos and files come to the app in the action as the platform's own file objects (a File on the web), for the app to upload where it likes; with AG-UI they go to the agent as content parts. The renderer checks required before sending. Device values exist only after the user acted: a tap took the photo, a tap shared the location.

#Component state

What the reader has done to the interactive components and forms is also kept as plain JSON per node, the same shape on every platform, so it can be saved, restored and shared with an agent. With AG-UI it is the shared state's morphscreen key, by message id and node id, sent with every run; an agent that sets it sets the components, and that sends no action.

The words of an action's user message come from the core (morphscreen_core::action::message), so every SDK and the API send the same turn. docs/agui.md has the rest.

#Versioning

Every pack carries a version, and a built catalog has a hash of everything in it. The prompt's hash is recorded with every benchmark run, so results are tied to the exact prompt the model saw.