Computer use

The routes below only work while the desktop is awake -- a 409 means the desktop isn't awake. A 501 means this desktop's host predates the computer-use API. See Concepts: Computer use for the look → act loop these compose into, and why input is never auto-retried.

Take a screenshot

POST /v1/desktops/{id}/screenshot

Request body (ScreenshotBody, all fields optional -- an empty/absent body captures the full display as binary PNG):

{ "encoding": "base64", "region": null, "scale_percent": 50 }

curl -sS -X POST "$BASE/desktops/$DESKTOP_ID/screenshot" \
  -H "$AUTH" -H 'Content-Type: application/json' \
  -d '{}' -o screenshot.png

curl -sS -X POST "$BASE/desktops/$DESKTOP_ID/screenshot" \
  -H "$AUTH" -H 'Content-Type: application/json' \
  -d '{"encoding": "base64", "scale_percent": 50}'

Response (encoding: "base64"):

{ "data_base64": "...", "width": 640, "height": 400 }

Send input

POST /v1/desktops/{id}/input

Runs a batch of input actions in order, as one guest round trip regardless of batch size.

Request body (InputBody):

{
  "actions": [
    { "type": "mouse_move", "x": 100, "y": 200 },
    { "type": "click", "button": "left" },
    { "type": "type", "text": "hello world" },
    { "type": "key", "combo": "ctrl+shift+t" },
    { "type": "wait", "ms": 250 }
  ]
}

actions (required, InputActionJson[]) is a type-discriminated list, capped at 50 actions, 32 KiB total text, 10 s total waits per batch:

An empty actions list is valid and performs the cursor-position read alone.

curl -sS -X POST "$BASE/desktops/$DESKTOP_ID/input" \
  -H "$AUTH" -H 'Content-Type: application/json' \
  -d '{"actions": [{"type": "click", "x": 400, "y": 300}]}'

Response (InputResponse) -- always the final cursor position after the whole batch ran:

{ "cursor": { "x": 400, "y": 300 } }

Get display geometry

GET /v1/desktops/{id}/display

One guest round trip -- there is deliberately no cached copy, so this only works while awake.

curl -sS "$BASE/desktops/$DESKTOP_ID/display" -H "$AUTH"

{ "width": 1024, "height": 768 }

Set display geometry

PUT /v1/desktops/{id}/display

Request body (SetDisplayBody):

{ "width": 1280, "height": 800 }

curl -sS -X PUT "$BASE/desktops/$DESKTOP_ID/display" \
  -H "$AUTH" -H 'Content-Type: application/json' \
  -d '{"width": 1280, "height": 800}'

Response is the geometry actually applied, re-read from the X server after the resize -- not necessarily identical to what was requested. It survives hibernate/wake (VM-memory snapshots); a cold boot reverts to the image default 1024x768.