kunjinkao-os/dsh-mobile-gui-agent
Android Mobile GUI Agent plugin for DeepSeek Harness with ADB control, iterative verification, approvals, and
dsh-mobile-gui-agent is a DeepSeek Harness plugin for controlling Android devices through ADB: it adds a mobile_gui_agent entry to the Harness Web UI and drives every task through an observe → decide → act → verify loop. It provides ADB discovery, wireless connection, screenshots, UIAutomator hierarchy capture, tap, long press, swipe, verified text input/replacement, keys, back, home, and package launch; strict phone_observe and phone_act tools with observation-local element IDs; one meaningful action per model turn with fresh observation and deterministic verification; stale-element protection, adaptive screen stabilization, stuck detection, step and time limits, and recoverable ADB errors. Consequential semantic controls (send, publish, delete, purchase, pay, transfer, call, install, account security) require Harness approval. Recommended workflow: send a short initialization message in the ordinary Harness conversation, select the mobile_gui_agent tab, pick the connected device, and enter the phone command only in its Task field. No Android accessibility service is installed — it combines ADB screenshots with UIAutomator hierarchy.
Install
dsh plugin --profile web add github:kunjinkao-os/dsh-mobile-gui-agent#v0.2.1GitHub source install (no npm package published as of 2026-08-24): dsh plugin --profile web add github:kunjinkao-os/dsh-mobile-gui-agent#v0.2.1. Built lib/ artifacts are tracked, so a pinned Git checkout installs without allowing a dependency build script. Verify with dsh --profile web --dump-config (must contain a '# == dsh-mobile-gui-agent' layer and one dsh-mobile-gui-agent row), then dsh --profile web. Prerequisites: Android Platform Tools (adb), Developer options + USB debugging on the device, RSA prompt authorized, device row reporting 'device'. The mobile_gui_agent entry is a browser client contribution, so a web profile is required (not headless-only). No root or APK required for observation, navigation, or printable ASCII input; Unicode input uses the external ADB Keyboard helper when installed (or adb.unicodeImeApkPath pointing to a reviewed APK — the plugin never downloads one, and installation requires explicit Harness approval).
Compatibility
DeepSeek Harness ^0.1.0-rc.5 (verified against upstream commit 47f943859bef60e4160492346772ded9b24f765a; also built/tested against published 0.1.0-rc.6 packages). Node.js ^22.19.0 || >=24.0.0. Android: physical device or emulator visible to adb devices. Follows upstream dsh.bundle.patch and dsh.client manifests; does not patch the Harness Agent loop or require unpublished Phone packages. Strict observe → decide → act → verify loop: one meaningful action per model turn followed by a fresh observation and deterministic verification; stale-element protection, adaptive screen stabilization, stuck detection, step/time limits, recoverable ADB errors; Harness approval for semantic controls that can send, publish, delete, purchase, pay, transfer, call, install, or change account security.
Details
- Repo: kunjinkao-os/dsh-mobile-gui-agent
- Category: Web UI Enhancements
- Stars: 7
- Version: GitHub source install github:kunjinkao-os/dsh-mobile-gui-agent#v0.2.1 (no npm package as of 2026-08-24)
- Last push: 2026-08-20
- First seen: 2026-08-14
Recent updates
The current bilingual README (English primary) documents: quick start (adb devices -l, pinned GitHub install, dump-config verification), recommended workflow, demos, compatibility, full feature list, requirements, installation options (local checkout, pnpm dsh, pinned GitHub, release tarball), wireless ADB, usage steps, complete configuration reference (adb. and agent. blocks), architecture, model experience, security model, known limitations, development, and troubleshooting.
FAQ
- Do I need root access or an APK on the phone?
- No — observation, navigation, and printable ASCII input work without root or an installed APK. Unicode input uses the external ADB Keyboard helper when installed; if absent, you can configure adb.unicodeImeApkPath to a reviewed APK and the plugin never downloads one.
- Can the model run arbitrary ADB shell commands?
- No — the model cannot issue arbitrary ADB shell text. Provider-owned diagnostic commands use a closed PhoneShellRequest classification, and consequential semantic controls require the existing Harness approval service when approvalEnabled is true.
- Why does the mobile_gui_agent entry not appear?
- Confirm the plugin was added to the same Web profile you started (headless profiles have no browser conversation views), inspect --dump-config, and hard-refresh the browser after installation. The entry is a browser client contribution.
Alternatives
sorsama/deepseek-harness-mobile · lehhair/dsh-mobile · Hotsteel2901/dsh-client-ui-mobile-adapt