Dynamo is the headless AI Visual & Video Harness for coding agents, browser automation and computer-use sandboxes. It runs the virtual display, records every session, and compiles it into VLM-ready keyframes and PR-ready MP4/GIF artifacts — in one context manager.
Drops into the stack your agents already run on
The problem
Sample a real session on the wall clock, the way the hand-rolled workflow does, and you pay for a thousand-plus frames. In our run, 19 of every 50 consecutive frames were pixel-identical — the model paid full price for an unchanged screen.
Race conditions on Xvfb startup, and corrupted MP4s from SIGKILL. Our DIY workflow lost its file to a missing moov atom on every run; the harness kept the file and all four logged actions. The 40 MB GIF claim we started with was fiction, by the way — both GIFs measured about 2 MB.
A single PNG can’t tell you which click caused the regression. Without timestamped telemetry, visual evidence is just a guess.
Architecture
Each layer is independently usable, fully typed, and battle-tested against headless Linux edge cases.
display.py
Owns the full Xvfb lifecycle. Configurable display number, resolution and DPI emulation, plus a readiness probe that blocks until X accepts connections.
recorder.py
Non-blocking ffmpeg -f x11grab in the background. Graceful SIGINT shutdown finalizes the moov atom — no corrupted containers, ever.
processor.py
CRF-tuned H.264 / WebM for crisp code legibility. Two-pass palettegen → paletteuse GIFs small enough for GitHub. Optional click-ring overlays from telemetry.
token_reducer.py
Keeps only frames that matter: scene-change detection plus exact frames at every logged action. Downsampled, deduplicated, labeled and timed for your LLM payload.
harness.py
One ergonomic VisualHarness context manager wires everything together. Same capabilities from the CLI for shell-driven agents and CI jobs.
vision.py
Routes each frame to an open-weight model, or to no model at all: skip unchanged frames, let a cheap text decision model score the survivors, and crop what is left to what the diff says moved. Records what the model was actually charged beside what we estimated.
Developer experience
No daemons to babysit, no display variables to export, no ffmpeg flags to memorize. Enter the context, act, exit — artifacts are finalized automatically.
take_screenshot() returns bytes mid-run for in-loop agent reasoning.log_action() timestamps every click, key and scroll for overlays and keyframe anchors.Artifacts result.from agent_visual_harness import VisualHarness
# Usage in agent script:
with VisualHarness(display=":99", resolution=(1920, 1080), record=True) as harness:
# 1. Agent runs GUI actions or Playwright tests here...
harness.log_action("click", x=500, y=300)
# 2. Grab instant viewport screenshot during execution
screenshot_bytes = harness.take_screenshot()
# Upon exit, files are finalized automatically:
artifacts = harness.export(
video_path="output/repro.mp4",
gif_path="output/demo.gif",
vlm_keyframes_dir="output/vlm_frames/"
)
print(f"Generated {len(artifacts.keyframes)} frames for VLM context.")# Start a display + recorder, run any command inside it
$ dynamo record --display :99 --resolution 1920x1080x24 --fps 24 \
-- python run_agent.py
# Compile artifacts from a finished session
$ dynamo export ./session \
--mp4 output/repro.mp4 --crf 23 \
--gif output/demo.gif --max-colors 128 --dither bayer \
--overlay-clicks
# Produce a token-budgeted keyframe set for your VLM
$ dynamo keyframes ./session --scene 0.05 --max-size 1280 \
--format webp --out output/vlm_frames/
✓ 29 keyframes · 1,042 frames scanned · manifest.json writtenFROM python:3.12-slim
RUN apt-get update && apt-get install -y --no-install-recommends \
xvfb ffmpeg x11-utils && rm -rf /var/lib/apt/lists/*
RUN pip install --no-cache-dir agent-visual-harness playwright \
&& playwright install --with-deps chromium
WORKDIR /app
COPY . .
# Dynamo manages Xvfb itself — no DISPLAY export needed
ENTRYPOINT ["dynamo", "record", "--", "python", "agent.py"]name: agent-e2e
on: [pull_request]
jobs:
visual-run:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: sudo apt-get install -y xvfb ffmpeg
- run: pip install agent-visual-harness
- run: dynamo record -- pytest tests/e2e
- run: dynamo export ./session --gif pr/demo.gif --overlay-clicks
- uses: actions/upload-artifact@v4
with: { name: agent-recording, path: pr/ }Token Reducer
Estimate how much visual context Dynamo removes before it ever reaches your multimodal model.
fewer visual tokens per session — with every action-anchored frame preserved.
The measurements
A real Chromium drives a real checkout — card declined twice, retried, captured on the third attempt — and the same 29 keyframes reach a local Qwen3-VL-4B three different ways. Two token columns, because they disagree.
| How the frames reach the model | Frames sent | Estimated tokens | Counted by the model |
|---|---|---|---|
| A · every keyframe, whole | 29 | 22,707 | 32,132 |
| B · cropped to what the diff says moved | 29 | 10,947 | 32,040 |
| C · skip unchanged → filter → crop | 24 | 8,575 | 26,528 |
We estimated a 52% cut and were wrong. The server resizes every image to a fixed budget before the vision encoder sees it: 1280×800 and 128×80 both cost 1,108 tokens, and a 25×16 crop cost more. The real bill moved 0.3%.
A cheap text decision model (Laya) scored each frame and dropped 5 of 29 — p = 0.05 to 0.10. Not sending the frame is the lever that actually reduces what you are charged.
An 8-byte stub frame that walked past a zero-byte guard, a single-threaded storefront that stalled the browser, and a gallery gated on a file written four lines later. All fixed, all tested.
Our own two-pass palette came out 4% larger than the naive GIF on dense browser chrome. We changed the caption rather than keep the claim.
Outputs
repro.mp4CRF-tuned H.264 keeps monospace text crisp at a fraction of raw size. Attach to bug reports and incident reviews.
demo.gifTwo-pass palette generation with bayer dithering. Small enough to inline in a GitHub PR description.
vlm_frames/Deduplicated JPEG/WebP frames with a timing manifest — drop straight into a multimodal LLM context payload.
telemetry.jsonlEvery agent action with a precise timestamp. Correlate visual state to cause for debugging and evals.
{"timestamp": 1.45, "action": "click", "x": 420, "y": 680}Why Dynamo
| Capability | Screenshots only | DIY ffmpeg scripts | Dynamo |
|---|---|---|---|
| Managed Xvfb lifecycle + readiness probe | |||
| Corruption-safe MP4 finalization | |||
| Timestamped action telemetry | |||
| Click overlays rendered on video | |||
| Scene + action-anchored keyframes for VLMs | |||
| PR-ready GIF with click overlays baked in | |||
| Python SDK + CLI parity |
Enterprise
Dynamo is a library, not a SaaS. Recordings, frames and telemetry are written to storage you control — nothing phones home.
Read what we actually measuredRuns entirely in your VPC, air-gapped clusters, or ephemeral sandboxes.
Isolated display numbers per worker. Run hundreds of parallel agent sessions per node pool.
Immutable video + action timelines for agent governance, compliance and incident review.
No GPU dependency. Tunable framerate and CRF to fit tight sandbox budgets.
Keyframes work with any multimodal model or in-house VLM — no vendor lock-in.
Standard library only. ffmpeg and tesseract are optional extras, not requirements.
FAQ
A Linux environment with Xvfb and ffmpeg installed, and Python 3.10+. It's designed for Docker, Kubernetes pods, CI runners, and computer-use sandboxes — no physical display or GPU required.
Yes. Anything that renders to an X11 display can be recorded. Launch your browser or GUI inside the VisualHarness context and Dynamo captures it; call log_action() from your agent loop to anchor telemetry.
It combines FFmpeg scene detection (e.g. select='gt(scene,0.05)') or image-difference scoring with exact frames at each logged action timestamp, then deduplicates and downsamples to your max bounding box.
The context manager sends SIGINT to the recorder on exit — including on exceptions — so FFmpeg finalizes the moov atom and the MP4 stays playable.
No. Dynamo is a local Python package. All recordings, frames and telemetry are written to paths you specify.
Get started
Tell us about your agent stack. An engineer — not a sales rep — will reply within one business day. Inquiries and demo requests are delivered directly to smasherthedevansh@gmail.com.