J.A TESTED · INDEPENDENT BENCH

Hands-on.
Reproducible.

Fixed environment, fixed inputs, median of three runs; commands, failure cases and corrections included. One question only: does it actually work for this task?

Reproducible · Authors can respond · No rankings

Latest

First report: we ended up testing the host

MemoryPilot2026-10-20

Memory plugins: all four passed. The no-plugin baseline passed too.

With a random codeword, the baseline still recalled it across sessions — the token was persisted in workspace AGENTS.md and the host's session cache. Cross-session recall is a DSH capability, not a plugin differentiator.

Read the full report →
Categories

Only tests that can fail the baseline

MemoryPilot done

Memory / context

v0.1 did not discriminate; v0.2 moves to post-compaction consistency, unit-level store/recall, and cross-project conflicts.

Read the finding →
ComfyUIQueued

ComfyUI packs

Environment diagnosis, broken-image triage, batch runs; hit rate, success rate, reproducibility.

Method first →
BlenderQueued

Blender packs

Headless render, glTF integrity, error recovery.

Method first →
Method

Four fixed steps

01 Pick

Publicly installable, version-pinned; validate that the baseline fails before testing candidates.

02 Fix

Record DSH version, model, machine, inputs and time budget.

03 Run

Three independent runs with fresh HOME and workspace; keep logs and artifacts.

04 Publish

Task-level verdicts, failure cases, reproducible commands; no rankings.

Corrections

Verdicts can change. Records are not deleted.

Authors are notified by email or GitHub issue with a 72-hour right of reply. Corrections are published with a revision log. Our own plugins are listed separately as “author-tested” and excluded from rankings.