All demosApp Pilot

App Pilot

Give an agent a goal. Watch it work through a simulated app.

Examples are free. Bring a key for your own inputs. Get a free key

Performance & cost

This session

Ready to run
Delivery time · median
—
Live model compute
—
Live input tokens
0
Est. cost · USD
$0.00

Bypass examples to measure live inference. Uses your quota.

Measurement details · 0 successful requests
Successful requests
0

Run an example for free to see its measurements.

Model compute is the average per measured request. Delivery time is the median browser time for successful requests, including queue, network, and retries. Cached delivery speed is not inference speed. Model compute can exceed browser time when one request processes many inputs.

Tokens come from response headers. Inference costs use $0.04 per million input tokens, before plan credits. The estimate includes live and recorded tokens at the same rate; cached examples are not charged again. Missing usage is unavailable or a partial total (≥). Demo resets keep these totals; reload to start a new session.

Try it

Pick a goal and run the example.

Build with the SDKTypeScript · Python · dm1

One request from this demo. Copy it into your app.

Uses your MS_API_KEY and quota. Keep the key on your server.

Install · set your key
npm install @cloudraker/milliseconds
export MS_API_KEY="your-api-key"
example.ts
import { DecisionMachine } from "@cloudraker/milliseconds";

// Run on your server; reads MS_API_KEY from the environment.
const dm = new DecisionMachine();

const { result, usage } = await dm.classify(
  "Goal: In Calculator compute 48*12.\nApp: Calculator.\nWindow: Calculator.\nFocused: nothing.\nModal sheet: no.\nVisible text: 0.\nText candidates: 48*12.\nAlready typed: none.\nFacts: 48*12 equals 576.\nPrevious actions: none.\nSelected right now: nothing.\nElements: e1 button 'All Clear'; e2 button 'Change Sign'; e3 button 'Percent'; e4 button 'Divide'; e5 button '7'; e6 button '8'; e7 button '9'; e8 button 'Multiply'; e9 button '4'; e10 button '5'; e11 button '6'; e12 button 'Subtract'; e13 button '1'; e14 button '2'; e15 button '3'; e16 button 'Add'; e17 button '0'; e18 button 'Point'; e19 button 'Equals'; e20 button 'Delete'.\nWhich single action advances the goal next?",
  {
    "e1 button 'All Clear'": "Press the button labelled 'All Clear'.",
    "e2 button 'Change Sign'": "Press the button labelled 'Change Sign'.",
    "e3 button 'Percent'": "Press the button labelled 'Percent'.",
    "e4 button 'Divide'": "Press the button labelled 'Divide'.",
    "e5 button '7'": "Press the button labelled '7'.",
    "e6 button '8'": "Press the button labelled '8'.",
    "e7 button '9'": "Press the button labelled '9'.",
    "e8 button 'Multiply'": "Press the button labelled 'Multiply'.",
    "e9 button '4'": "Press the button labelled '4'.",
    "e10 button '5'": "Press the button labelled '5'.",
    "e11 button '6'": "Press the button labelled '6'.",
    "e12 button 'Subtract'": "Press the button labelled 'Subtract'.",
    "e13 button '1'": "Press the button labelled '1'.",
    "e14 button '2'": "Press the button labelled '2'.",
    "e15 button '3'": "Press the button labelled '3'.",
    "e16 button 'Add'": "Press the button labelled 'Add'.",
    "e17 button '0'": "Press the button labelled '0'.",
    "e18 button 'Point'": "Press the button labelled 'Point'.",
    "e19 button 'Equals'": "Press the button labelled 'Equals'.",
    "e20 button 'Delete'": "Press the button labelled 'Delete'.",
    "type_text": "Type '48*12' into the focused control. The text is not there yet.",
    "press_key": "Press a keyboard shortcut instead of an element, for example Return to submit text already typed.",
    "done": "The goal is already visible in the state. Nothing more to do."
  }
).withUsage();

console.log(result);
console.log({ inputTokens: usage.inputTokens, modelMs: usage.inferenceMs });

Under the hood How it works & limitations

API calls/classify/yes-no

How it works

The model reads a text representation of the app’s accessibility tree and selects the next action. Stock goals replay recorded model responses for free. With your own key, custom goals use live inference. In both modes, the demo applies each action and checks the updated app state.

Demo scope

The app runs inside this page and does not control your computer. The recorded examples contain accessibility trees captured from macOS apps. Completion judgments can be wrong; inspect the final app state.

Inspired by the work of dabit3.