dongsheng123132/dsh-benchmark
Deterministic revision-pinned benchmarks and regression evidence for DeepSeek Harness
Reproducible, deterministic benchmark evidence for DeepSeek Harness tools and plugins. Deliberately does not duplicate dsh-batch-regression (which runs one shell command repeatedly for median/distribution statistics): dsh-benchmark defines an evidence protocol around fixed cases — explicit target and suite revisions, file-derived target fingerprints, bounded argv-only subprocesses, raw measurements, versioned deterministic scoring, content-addressed reports, and baseline regression comparison. The first release evaluates commands against fixed cases and writes content-addressed evidence files.
Install
dsh plugin --profile benchmark add github:dongsheng123132/dsh-benchmarkGitHub install per README (English): dsh plugin --profile benchmark add github:dongsheng123132/dsh-benchmark (README uses a benchmark profile; use your own profile name). Requires Node.js 22+; no runtime dependency or install lifecycle script beyond the optional DSH tools SDK peer.
Compatibility
DeepSeek Harness (dsh). Node.js 22+. Evidence protocol for reproducible benchmarks.
Details
- Repo: dongsheng123132/dsh-benchmark
- Category: Coding & Development
- Stars: 3
- Version: GitHub source (no npm package)
- Last push: 2026-09-07
- First seen: 2026-08-13
Recent updates
deterministic benchmark evidence; fixed cases + file-derived fingerprints; versioned scoring; content-addressed reports; baseline comparison.
FAQ
- How is it different from dsh-batch-regression?
- dsh-batch-regression runs one command repeatedly for medians/distributions; dsh-benchmark scores fixed cases with an evidence protocol (fingerprints, versioned scoring, content-addressed reports).
- What does it require?
- Node.js 22+ and the dsh plugin install; no runtime dependency or install lifecycle script beyond the optional DSH tools SDK peer.
- Are reports reproducible?
- Yes — explicit target/suite revisions, file-derived fingerprints and bounded argv-only subprocesses make results deterministic and comparable.
Alternatives
dongsheng123132/dsh-cost · dongsheng123132/dsh-action-parity · golitter/dsh-deepseek-billing