bowang-lab/dsh-medomni

Routes a request to a modality-specific imaging tool — X-ray, CT, MRI, ultrasound, retinal — backed by models running in one shared environment.

The plugin exposes a composable set of imaging tools behind one DSH entry point and lets the agent route by modality — X-ray, CT, MRI, ultrasound and retinal images — to models such as MAIRA-2, MedGemma, TotalSegmentator, BiomedParse and BiomedCLIP running in a shared Python environment. Because the models live in one environment instead of one container per task, the agent can chain steps across modalities inside a single conversation: read a study, segment structures, then describe findings. The README documents prefetch commands for teams that want models warm before the first request, and a section on enabling image input so pasted images reach the tools.

Vision & Multimodal ★ 7
View on GitHub ↗

Install

dsh plugin --profile web add github:bowang-lab/dsh-medomni

The Quick start line from the README. Node.js >= 22 is declared on the package badge; the Python side is provisioned with uv. The README's quick start also tells you to confirm python3, uv and git are on PATH before installing.

Compatibility

An NVIDIA GPU with CUDA is expected: the README says CPU mode exists for some tools but the multi-GB vision-language and segmentation models are generally impractical without one. Gated Hugging Face checkpoints (MAIRA-2, BiomedParse and others) must have their terms accepted, and a Hugging Face read token must be available through hf auth login or HF_TOKEN. No checkpoint is downloaded by hand: each tool fetches its model on first use.

Details

Alternatives

huashenglian/dsh-omni-workstation · PiyotaHu/muxiva-dsh-voice · AtropinolTT/dsh-guide-dog

More plugins in Vision & Multimodal

Browse more in Vision & Multimodal

Guides for Vision & Multimodal plugins