ZRui-C/dsh-computer-use
Text-first computer use for DSH: background Chromium control via Playwright/CDP plus accessibility-first macOS control; actions stay pinned to the right process and window without taking the user's pointer, ships a Developer ID signed, notarized Universal 2 DMG.
Text-first browser and background macOS control for DSH that targets the right process and window without taking over the user's pointer. For Chromium it perceives via Playwright, CDP accessibility/DOM, frames, tabs and optional OCR, and acts with ref-pinned navigation, pointer, keyboard, forms, scroll, drag and upload; for macOS it reads the Accessibility tree first with Vision OCR for semantic gaps and independent window capture, acting through AX actions and targeted SkyLight/CoreGraphics, falling back to global HID only when there is no target. Every action returns a fresh bounded text observation, so the model works with roles, names, values, state, geometry and stable snapshot refs instead of assuming it can inspect screenshots — and UI/OCR strings are explicitly treated as untrusted data. Background macOS control carries PID, WindowServer window ID, AX window frame and element identity through every action, prefers semantic AX actions before coordinate input, uses a click-through software cursor so targeted actions do not move the physical pointer, and fails closed when a private capability is unavailable. It also detects the macOS 26 Stage Manager thumbnail case and returns an explicit warning instead of stretching a thumbnail into a fake full-window screenshot. The embedded package declares a DSH bundle; its cordis.patch.yml installs the Host runtime and registers computer_observe / computer_action in DSH's global tool layer, inherited by every agent preset, with no manual edits to user profile YAML.
Install
brew tap zrui-c/tap && brew trust zrui-c/tap && brew install --cask dsh-computer-useREADME Homebrew block quoted verbatim (the lines after it are open -a "DSH Computer Use"). Verified live 2026-09-13: the latest GitHub release is v0.3.0, publishing DSH-Computer-Use-0.3.0-universal.dmg plus its .sha256. The README also documents the DMG path: download the latest DSH-Computer-Use-*-universal.dmg from Releases, drag the app into Applications and open it. With either method, authorize Accessibility and Screen Recording, select Install under DSH Plugin, then restart the running DSH Host. The README states the Homebrew Cask and official DMG install the same Universal 2, Developer ID signed, Apple-notarized app, so users do not need Xcode, Swift or the source checkout — but DSH and Google Chrome must already be installed. For the embedded package, the setup center runs the official equivalent of dsh plugin --profile web add --save-exact file:/path/to/DSH\ Computer\ Use.app/Contents/Resources/Plugin.
Compatibility
macOS 14+ only, Universal 2 (arm64 | x86_64), Apache-2.0. Source builds require macOS 14+, Xcode/Swift 5.9+, Node.js 22+, pnpm 11+, DSH and Google Chrome. The README is explicit about the platform boundary: ScreenCaptureKit is public API, while the optional background input path dynamically loads private SkyLight symbols and is intended for Developer ID distribution, not the Mac App Store.
Details
- Repo: ZRui-C/dsh-computer-use
- Category: Tools & Capabilities
- Stars: 24
- Version: GitHub release v0.3.0 (release asset DSH-Computer-Use-0.3.0-universal.dmg verified 2026-09-13); macOS 14+
- Last push: 2026-08-14
- First seen: 2026-08-14
Recent updates
The README points at documentation/distribution.md and documentation/architecture.md rather than an in-README changelog; the latest release verified on 2026-09-13 is v0.3.0.
FAQ
- How do I install it?
- The README's Homebrew block is brew tap zrui-c/tap, brew trust zrui-c/tap, brew install --cask dsh-computer-use; the DMG from Releases is the documented alternative and installs the same app.
- Do I need to move my mouse?
- No — the README says targeted actions use a click-through software cursor and do not move the physical pointer, with post-action observation pinned to the previous target even while another app is active.
- What permissions are required?
- With either install method, authorize Accessibility and Screen Recording, select Install under DSH Plugin, then restart the running DSH Host.
Alternatives
Anionex/dsh-computer-use · JohnXu22786/computer-control · FrankZhangIronly/dsh-system-control