munim-computer-use

Introduction

Computer Use is an open-source MCP server — a computer-use agent (CUA) backend — that lets any coding agent use your computer the way a person does. It reads the screen through accessibility trees, clicks and types in the background so your mouse stays yours, shows an agent pointer where it is working, zooms in on small text, and drives tabs in your signed-in Chrome — on macOS, Windows and Linux.

Works with Claude Code, Codex, Cursor and MT Code, or any other MCP client, with any model — no vision model is required for interaction.

Built by Munim Technologies as the Computer Use engine of MT Code, and published here on its own.

Table of contents

• Quick start
• Capability matrix
• Why it works well
• Tools
• Repository layout
• Environment flags
• Prompting your agent
• Contributing
• Credits and license

Quick start

1. Download the latest binary for your platform from Releases (munim-computer-use-macos-universal.zip, munim-computer-use-windows-x64.zip) or build from source.
2. Put it somewhere on your PATH (/usr/local/bin/munim-computer-use, or %LOCALAPPDATA%\Programs\munim-computer-use\munim-computer-use.exe).
3. macOS only: run munim-computer-use request-permissions once to be prompted for Accessibility and Screen Recording.
4. Register it with your agent:

# Claude Code — fastest: the npm launcher fetches the signed binary on first run
claude mcp add munim-computer-use -- npx -y munim-computer-use
# or point at a downloaded binary
claude mcp add munim-computer-use -- /usr/local/bin/munim-computer-use

# Codex — ~/.codex/config.toml
[mcp_servers.munim-computer-use]
command = "npx"
args = ["-y", "munim-computer-use"]

// Cursor — .cursor/mcp.json
{ "mcpServers": { "munim-computer-use": { "command": "npx", "args": ["-y", "munim-computer-use"] } } }

Then ask: "Open Safari, find the cheapest flight to Denver on Tuesday and put it in a note." The agent reads the UI with get_app_state, acts by element id, and verifies with screenshot.

Capability matrix

Munim Computer Use is the highlighted first column; the others are the computer-use servers people reach for. each cell comes from that project's own README in September 2026 (sources under Credits and license). ✅ present · ❌ absent or not documented · ⚠️ partial.

Capability	Munim Computer Use	Codex Computer Use	Anthropic reference demo	Windows-MCP	MacOS-MCP	open-computer-use	computer-use-mcp (zavora)	Notes
macOS	✅	✅	❌	❌	✅	✅	✅	The Anthropic demo drives a Linux desktop inside Docker, not your machine.
Windows	✅	✅	❌	✅	❌	✅	✅	Windows-MCP is Windows only; MacOS-MCP is macOS only.
Linux	✅	❌	✅ (sandbox)	❌	❌	✅	✅	Munim Computer Use uses AT-SPI + X11; native Wayland apps get element actions but not coordinate clicks.
Accessibility tree with element ids	✅	❌	❌	✅	✅	✅	✅	Codex and the Anthropic demo are screenshot-driven. Ids let the agent press the button instead of a pixel.
Background input (your mouse never moves)	✅	✅	n/a	❌	❌	❌	❌	Munim Computer Use addresses events to the target window (SkyLight on macOS, posted window messages on Windows, XTEST on Linux). Codex does this too, with a second cursor of its own. Every other server in this table drives the real cursor.
Agent pointer overlay	✅	✅	❌	⚠️	❌	❌	❌	Windows-MCP flashes a border around captures; Codex draws its own cursor on your screen.
Zoom into a region at full resolution	✅	❌	✅	❌	❌	❌	❌	Anthropic's toolset has zoom; here it is a tool on every platform.
Screenshots carry screen-coordinate mapping	✅	n/a	n/a	❌	❌	❌	❌	Origin and pixels-per-point in every capture, so clicks from Retina or downscaled images land.
Hover, wait, label query	✅	⚠️	⚠️	✅	⚠️	❌	⚠️	Windows-MCP has Wait/WaitFor; MacOS-MCP has Wait; Anthropic has wait/mouse_move.
Your signed-in Chrome, own tab group or yours	✅	⚠️	❌	⚠️	❌	❌	❌	Codex uses its in-app browser; Windows-MCP reads the DOM of open browsers. Munim Computer Use opens its own labelled tab group in your real Chrome, and can also take over a tab you already have open when you ask it to.
Works with any MCP client	✅	❌	❌	✅	✅	✅	✅	Codex Computer Use is Codex only; the Anthropic demo is Claude only.
Identical tool surface on every platform	✅	n/a	n/a	n/a	n/a	⚠️	✅	28 tools with byte-identical schemas across the Swift and Rust servers.
Prebuilt signed binaries + npm launcher	✅	✅	❌	❌	❌	✅ (npm)	✅ (npm)	macOS universal (Developer ID signed) and Windows x64 on Releases.
Open source	✅ Apache-2.0	❌	✅	✅ MIT	✅ MIT	✅ MIT	✅ MIT

Also looked at: mediar-ai/mcp-server-macos-use (macOS, accessibility, real input), deploymenttheory/windows-mcp-server (Windows, UIA Invoke patterns, WaitFor), nuphus-mcp (OCR + bring-your-own vision model, CDP Chrome), computer-control-mcp (PyAutoGUI + OCR), and microsoft/playwright-mcp (browser only). Corrections welcome — open an issue with a link.

Why it works well

• Accessibility first, pixels second. get_app_state returns the app's accessibility tree with stable element ids, so the agent presses the button instead of guessing at a coordinate. It costs a fraction of the tokens of a screenshot and it is what scores highest on OSWorld-style tasks. Screenshots are for verifying and for content the tree cannot describe.
• Background control. Events are addressed to the target window (SkyLight on macOS, posted window messages on Windows, XTEST on Linux). No focus stealing, no hijacked mouse.
• Pointer overlay, not your pointer. A soft lavender agent pointer shows where the agent is acting. Your cursor is untouched.
• Coordinates that land. Every screenshot and zoom carries its screen origin and pixels-per-point. zoom captures any region at full physical resolution.
• Your browser, your logins. The Chrome extension gives the agent its own labelled tab group in your signed-in Chrome, and leaves your tabs alone unless you point it at one.
• Or the tab you already have open. browser_list_tabs all=true shows every tab in the browser and browser_use_tab takes one over in place — useful when the page is already signed in or mid-flow and re-opening the URL would throw that away. An adopted tab is not moved into the agent's group, not activated and not reloaded; cleanup releases it rather than closing it, and browser_release_tab hands it back early.
• Model-agnostic. No vision model is required for interaction; local models work too.
• Look → act → verify. hover for mouse-over menus, wait for loads, query to find a control by label without reading a whole tree.

Tools (28)

Area	Tools
See	list_apps, get_app_state (with query), screenshot, zoom, list_displays
Act	click, right_click, hover, drag, scroll, type_text, set_value, select_text, press_key, activate_app, wait
Browser	browser_open_tab, browser_list_tabs, browser_use_tab, browser_release_tab, browser_select_tab, browser_navigate, browser_snapshot, browser_click, browser_type, browser_press_key, browser_close_tab, browser_close_all_tabs

Names, argument shapes and descriptions are identical on every platform; a model that learned them on a Mac needs nothing new on Windows.

Repository layout

Directory	What	Build
macos/	Swift server on the Accessibility API and ScreenCaptureKit (macOS 14+)	swift build -c release → .build/release/munim-computer-use
windows-linux/	Rust server: UI Automation on Windows, AT-SPI + X11 on Linux	cargo build --release → target/release/munim-computer-use
chrome-extension/	Chrome extension + native messaging host for the browser_* tools	Load unpacked; sh install.sh / install.ps1 registers the host; node background.test.mjs

Build from source

# macOS
cd macos && swift build -c release
# Windows / Linux
cd windows-linux && cargo build --release

Linux notes: element actions work everywhere; coordinate clicks need an X11 or XWayland client, since native Wayland apps do not expose absolute geometry.

Chrome extension (optional)

1. chrome://extensions → Developer mode → Load unpacked → select chrome-extension/.
2. Register the native messaging host: sh chrome-extension/install.sh (macOS/Linux) or powershell -File chrome-extension/install.ps1 (Windows). Point COMPUTER_USE_PATH at the binary if it is not in the default build location.

Environment flags

Variable	Effect
COMPUTER_USE_BROWSER=0	Hide the browser_* tools
COMPUTER_USE_AGENT_CURSOR=0	Do not draw the agent pointer
COMPUTER_USE_AGENT_CURSOR_TASK_FADE_SECS	How long the pointer stays after the last tool call (default 8)
COMPUTER_USE_ALLOW_SECURE_FIELD_INPUT=1	Allow typing into password fields (refused by default)
COMPUTER_USE_COMPUTER_USE_YIELD_SECS	Pause the agent while the user is actively using the machine

Prompting your agent

Look → act → verify. get_app_state for ids, act by id, then get_app_state or screenshot again before the next step. Use zoom for small text, hover for menus that appear on mouse-over, wait after loads, keyboard shortcuts for stubborn widgets. The system-prompt text MT Code gives its agents lives in CodexDeveloperInstructions.ts and is a good starting point.

Contributing

This repository mirrors the native/ tree of munimtechnologies/mtcode, where the server is developed and shipped inside MT Code. Issues and discussions are welcome here; code changes land in mtcode first and are synced.

Credits and license

Designed and built by Munim Technologies (Munim, Inc.) for MT Code. Copyright 2026 Munim, Inc. Licensed under the Apache License 2.0; see LICENSE.

Comparison sources: Codex Computer Use · Anthropic computer-use demo · CursorTouch/Windows-MCP · CursorTouch/MacOS-MCP · QwenLM/open-computer-use · zavora-ai/computer-use-mcp · mediar-ai/mcp-server-macos-use · deploymenttheory/windows-mcp-server · nuphus-mcp · computer-control-mcp · microsoft/playwright-mcp