ciky20171114/dsh-plugin-midscene

Lets the model see a screen and act on it — a real Android device over ADB or the active page of a running Chrome — through one tool with an action parameter.

One tool per target with an action parameter — tap, act, input, query, assert, boolean, back — instead of a wide tool surface, so the model names an action and the plugin dispatches internally. The model locates elements from a natural-language description and the action lands on a real target: an ADB-connected phone or the live page in your own Chrome.

Workflow & Automation ★ 3
View on GitHub ↗

Install

dsh plugin --profile mysetup add dsh-plugin-midscene

The README's install line, kept verbatim; mysetup is the example profile name it uses, so substitute your own profile. npm dsh-plugin-midscene, latest 0.1.0, registry repository resolves to this repo. The bundle registers the tools but ships no provider: exactly one provider row (android or web) must be added to the profile's cordis.patch.yml, and the README shows both.

Compatibility

Android needs a device visible in adb devices. Web needs Chrome started with --remote-debugging-port=9222 — the provider connects and never launches a browser. A Midscene-compatible vision model is configured through environment variables. Both providers cannot own ctx.midscene in the same context.

Details

Alternatives

alib8b8/dsh-plugin-aflare · vitaliy-bobrov/dsh-plugin-bmad-sprint-board · AK-blank/dsh-plugin-auto-answer

More plugins in Workflow & Automation

Browse more in Workflow & Automation

Guides for Workflow & Automation plugins