kittimzhe/dsh-session-eval

Retrospective evaluation for DSH sessions: deterministic grade cards and cross-session regression diffs, with no benchmarks to author and no LLM judges.

Answers the question a plugin or prompt change raises and nobody measures: did the agent get better or worse? Every grade is a fixed threshold over sessions that already happened — no benchmark suite to write, no new runs, no LLM judge — so the same logs always produce the same card. /eval grades a session, /eval-diff compares across sessions to expose regressions, and /eval-history shows the trend. Grade bands and dimension thresholds are configurable, which matters because a threshold that fits one workflow misgrades another. It is the measurement layer under a session toolchain rather than another dashboard.

Sessions & Messages ★ 0
View on GitHub ↗

Install

dsh plugin --profile web add github:kittimzhe/dsh-session-eval

The README's GitHub install line; an npm-style line is also shown. The bundle overlay mounts three commands — /eval, /eval-diff and /eval-history — and the README gives the patch row to keep if you maintain your own cordis.patch.yml.

Compatibility

The README requires Node.js 20 or 22 and a DSH profile that mounts the commands and sessionQuery services, which the shipped web and agent profiles do. Grades are fixed documented thresholds over persisted session logs, so results are reproducible rather than sampled.

Details

Alternatives

hy-sde/dsh-session-url · reinocheong/dsh-session-move · PwnKY/dsh-session-link

More plugins in Sessions & Messages

Browse more in Sessions & Messages

Guides for Sessions & Messages plugins