Issues

Roadmap: the order we are working in, and why
#24 · leggetter opened on Aug 19, 2026
3

Operator events list endpoints advertise filters and paging they silently ignore
hookdeck/evals#88 · opened on Oct 1, 2026 · 5. Sustain
Outpost answers 502 for an unknown operator event destination id
hookdeck/evals#87 · opened on Oct 1, 2026 · 5. Sustain
Docs-call reporting counts unobserved search results as evidence, in both directions
hookdeck/evals#83 · opened on Sep 22, 2026 · 4. Trust the numbers
1
One errored cell stops a whole run from publishing
hookdeck/evals#80 · opened on Sep 21, 2026 · 4. Trust the numbers
Agents wire a placeholder verification secret into a live source, and nothing surfaces it
hookdeck/evals#75 · opened on Sep 2, 2026 · 5. Sustain
Re-running publish-results on a held run fails to push
hookdeck/evals#74 · opened on Sep 2, 2026 · 4. Trust the numbers
1
Documentation access differs by agent and by arm, and the search path mostly returns nothing
hookdeck/evals#61 · opened on Aug 26, 2026 · 2. Prove the skills earn their place
4
A merged snapshot does not say it is merged, and the page renders one date for it
hookdeck/evals#60 · opened on Aug 26, 2026 · 4. Trust the numbers
2
Five benchmark scenarios have never been failed by any agent
hookdeck/evals#47 · opened on Aug 24, 2026 · 1. Cover the product
5
Hookdeck's webhook notification config cannot be read back, and the natural way to set it returns 200 while doing nothing
hookdeck/evals#43 · opened on Aug 21, 2026 · 5. Sustain
The provider-key preflight only runs in CI, not for a local run
hookdeck/evals#42 · opened on Aug 21, 2026 · 4. Trust the numbers
The Outpost skill never says you need a key for an Outpost project
hookdeck/evals#40 · opened on Aug 21, 2026 · 5. Sustain
1
The Outpost API answers "Not Found" when your key belongs to a non-Outpost project
hookdeck/evals#39 · opened on Aug 21, 2026 · 5. Sustain
1
Nothing detects an agent reading this repository during a run
hookdeck/evals#38 · opened on Aug 21, 2026 · 4. Trust the numbers
The operator events API is undocumented: absent from both OpenAPI definitions and from the docs
hookdeck/evals#34 · opened on Aug 20, 2026 · 5. Sustain
2
Outpost docs give the wrong env var name for every operator events setting
hookdeck/evals#32 · opened on Aug 20, 2026 · 5. Sustain
1
A leftover from a crashed run becomes permanent project state
hookdeck/evals#28 · opened on Aug 19, 2026 · 4. Trust the numbers
2
Does documenting hookdeck ci stop agents giving up on CLI authentication?
hookdeck/evals#27 · opened on Aug 19, 2026 · 2. Prove the skills earn their place
The skills submodule pin drifts silently, and moving it will change results with no record
hookdeck/evals#26 · opened on Aug 19, 2026 · 4. Trust the numbers
No signal for when a configuration change is in force
hookdeck/evals#25 · opened on Aug 19, 2026 · 5. Sustain
1
Roadmap: the order we are working in, and why
hookdeck/evals#24 · opened on Aug 19, 2026
3
Publish transcripts with releases so a published cell can be checked
hookdeck/evals#21 · opened on Aug 18, 2026 · 5. Sustain
1
Product coverage: Console still has no scenarios
hookdeck/evals#19 · opened on Aug 17, 2026 · 1. Cover the product
3
Add Claude Opus: decide the cadence and the budget it needs
hookdeck/evals#16 · opened on Aug 17, 2026 · 5. Sustain
1
Release process: package a run, its closed issues and its measured delta
hookdeck/evals#13 · opened on Aug 14, 2026 · 3. Finish publishing
2