Skip to content

fix(admin-jobs): count a job's own run in its attempts - #70

Merged
plainprogrammer merged 8 commits into
mainfrom
061-retried-job-attempts
Oct 13, 2026
Merged

plainprogrammer merged 8 commits into
mainfrom
061-retried-job-attempts

Conversation

@plainprogrammer

Copy link
Copy Markdown
Owner

A job's page at /admin/jobs/:id now counts the job's own run, so a job that failed reads 1 rather than "Attempts 0". The row is labelled "Attempts since queued".

Closes #61

Cause

BackgroundJobs::Entry#attempts returned executions from the job's stored Active Job envelope. Two things make that the wrong number:

  • Active Job raises executions only in memory in perform_now (activejob 8.1.4 execution.rb:47). It is stored only on the new row a retry_on retry inserts, so a row's stored value is the runs before that row was queued and never its own run.
  • An admin's retry calls FailedExecution#retry → Job#reset_execution_counters (solid_queue 1.7.0 job/retryable.rb:33), which sets it back to 0 on the same row. After that the row is identical to a freshly queued one.

So the queue's tables can't give a total across an admin retry.

What changed

  • Entry#attempts is the stored executions plus one when the job is running, failed or finished. No new storage, and nothing is written to the queue's rows (ADR 0014).
  • The page's label is "Attempts since queued".
  • AC-6.5, contracts/api.md and the code comment say how the count behaves (spec 015 version 1.1.3). The research note that recorded "Attempts 0" points here.
State Shows
Queued, waiting, scheduled (never run) 0
Running, failed or finished on its first run 1
Failed, then retried by an admin 0 while queued, 1 once it runs again
retry_on retries 1, 2, 3 across the rows; a waiting retry row shows the runs before it
Put back by a worker's graceful shutdown, then run again 1 → 0 → 1

Decision for you: the label

I chose the no-storage option and the label "Attempts since queued" during the work. The reviewer found that no single reading of "queued" is true in every row beside the page's own "Queued" time:

  • After an admin retry the page can read "Attempts since queued 1 / Queued 03:15 / Failed 04:02" for a job that has run twice since 03:15, because a retry keeps the row's original created_at.
  • A waiting retry_on row reads 1 although nothing has run since that row was queued.
  • A job released by a graceful shutdown and started again reads 1.

Alternatives: keep "Attempts" and let AC-6.5 carry the caveats; remove the row; or add storage to count admin retries (a migration, and the primary and queue databases are separate files). Changing the label is one line in app/views/admin/jobs/show.html.erb plus the spec wording.

Verification

  • bin/ci: passed. RSpec 1226 examples, 0 failures, 1 pending (the art agreement spec, unrelated).
  • The new examples failed before the fix (4 model, 2 request). The retry_on examples run jobs through Solid Queue's real ClaimedExecution#perform. The graceful-shutdown example is a characterization that passed on first run; it fails under a mutation of attempts.
  • An independent read-only review (Opus) confirmed the arithmetic against Solid Queue and Active Job's source and asked for wording fixes only, which the last four commits make.
  • Not verified: a real worker in development (retry, shutdown), and the light theme. Dark-theme screenshots at 1280 px and in a 390 px frame showed the label on one line.

Notes

🤖 Generated with Claude Code

plainprogrammer and others added 7 commits October 12, 2026 18:48
Issue #61. A job that ran and failed, is running, or finished reads
"Attempts 0", and so does a failed job an admin retried, once it has run
again. The specs cover every state the page shows: queued, waiting,
scheduled, running, failed, finished, retried by an admin, and retried
by the job's own retry_on (run through the queue's claimed execution).

They fail until the fix that follows.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Issue #61. The job page read the "executions" value in the job's stored
Active Job envelope. Active Job raises it in memory when the job runs
(ActiveJob::Execution#perform_now) and it reaches the queue only when
retry_on queues the job again, as a new row; so a row's stored value is
the runs before it was queued, and a job that ran once and failed read
"Attempts 0". Solid Queue also sets it back to 0 when a failed job is
retried (FailedExecution#retry -> Job#reset_execution_counters), which
leaves no record of the earlier runs in the queue's tables.

The entry now adds the row's own run when it is running, failed or
finished, and the page labels the fact "Attempts since queued", which
is what the queue's records can say: the count starts again when an
admin's retry queues the job again. No storage, and nothing written to
the queue's rows.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Issue #61. The coordinator ruled on option 1 of four (count the row's
own run and relabel; remove the row; write a running total into the
queue's row on retry; add storage for admin retries): no storage and no
writes into the queue's rows, because Solid Queue resets executions on
FailedExecution#retry, so its tables cannot give a total across an
admin's retry. The maintainer is to confirm the label "Attempts since
queued" at PR review.

AC-6.5's "attempts (executions)" is reworded to match, the spec goes to
1.1.3, and the page contract follows.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…d again

A worker that shuts down gracefully releases its claimed jobs
(SolidQueue::ClaimedExecution#release, solid_queue 1.7.0): the job gets a
ready execution again and its stored executions stay as they were. The
page therefore reads 1, 0, 1 for a job started twice. This is a limit of
the queue's records; the example states what the code does (issue #61).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
"Since it was queued" was false for a row a retry_on retry queues: that
row is queued after the runs it counts. The comment now covers the job's
own retries, an admin's retry and a graceful shutdown. Comment only; the
computation is unchanged (issue #61).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The count carries across the job's own (retry_on) retries, includes a run
in progress, starts again when an admin retries a failed job, and leaves
out a run cut short by a graceful shutdown once the job is back in the
queue. The contract says the same. The changelog row no longer says the
label awaits confirmation: the label "Attempts since queued" stays as
ruled for issue #61 and the pull request asks the question.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The dated observation and its screenshots stay as they are.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@plainprogrammer plainprogrammer self-assigned this Oct 12, 2026
Spec 015's changelog conflicted with issue #60's rows (1.2.0, 1.2.1).
Issue #61's row follows them as 1.2.2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@plainprogrammer
plainprogrammer marked this pull request as ready for review October 13, 2026 20:31
@plainprogrammer
plainprogrammer merged commit 5f2a099 into main Oct 13, 2026
4 checks passed
@plainprogrammer
plainprogrammer deleted the 061-retried-job-attempts branch October 13, 2026 20:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A retried job shows "Attempts 0"

1 participant