Repository navigation
Conversation
…; debug logs on chosen nodes Fable's analysis of the bootstrap-livelock stall with #77 (run 39886642406): the cause is not the stage-1 hash race but timing. The starter re-sends its bootstrap table every round_elapse_time and every same-round table restarts the consensus on each node; a round with one late stage message needs the fallbacks (12.5-16.5 s), so with 10 s it never completes. Official behaviour (master showed the same stall, run 39866725719). - PAUSE_AT / PAUSE_SECONDS: a trusted node (not n1) is stopped with SIGSTOP and continued, so a bootstrap with a silent member happens every run - LOG_DEBUG_NODES: those nodes log at debug level (stage and timer messages) - robustness: bootstrap-pause (10 s, 15 s pause, debug on n1-n3), run by name, fails until the node stops restarting the consensus on a re-sent table; comments name the real cause Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…a reproduction Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ery node Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Fable analysed the
bootstrap-livelockstall that still happened with #77 (run 39886642406). The stage-1 hash race was not the cause. The real cause is timing, and it is official behaviour:round_elapse_time;Master shows the same stall (run 39866725719).
Changes
PAUSE_AT/PAUSE_SECONDS: a node that is trusted in the current round (not n1) is stopped with SIGSTOP and then continued. Every run gets a bootstrap with a silent member.LOG_DEBUG_NODES: those nodes log at debug level, so stage and timer messages are visible.bootstrap-pause: 10 s timer, a 15 s pause, debug logs on n1 to n3. It runs only when named, and it is expected to fail until the node stops restarting the consensus on a re-sent table.bootstrap-livelockandROUND_ELAPSE_MSnow name the real cause.Result: the pause does not reproduce the livelock
bootstrap-pausewas run twice on master (runs 39896667779 and 39896678331). Both passed: 1,500 blocks with matching digests.So a silent trusted node with a 10 s timer is not enough on its own. The livelock seen under loss needs more, probably continuous message delays on two confidants. The debug logs of these runs (
logs-bootstrap-pause, n1 to n3) are the starting point.Keep this PR for the tools (
PAUSE_AT,LOG_DEBUG_NODES) and the corrected comments.bootstrap-pausestays a manual scenario, and its comment should not claim that it fails.🤖 Generated with Claude Code