Repository navigation
Fix a race condition in dmypy - #5956
Merged
Merged
Conversation
While thinking about what might cause flakes in #5859, I found a race condition between `dmypy stop` and other `dmypy` commands. `dmypy stop` will send a response and close the socket before the status file has been deleted. This means that there is a window in which the daemon has reported to a client that it has exited but while a status file still exists. This can result in a number of issues, including the daemon appearing to be stuck (instead of stopped) to subsequent commands and also the exiting server deleting the status file of a subsequently started server. The fix is to remove the status file in `cmd_stop` before replying to the request. This ensures that, as far as clients are concerned, the daemon is exited after a stop command completes. I tested the bug and the fix by inserting a `time.sleep(1)` immediately before the `sys.exit(0)` in `serve`: this caused several tests to fail, and the changes fixed them. I believe that the observed flakes in the windows version in #5859 were caused by this issue, but in a way that the unix version was not susceptible to.
emmatyping
approved these changes
Nov 26, 2018
emmatyping
left a comment
emmatyping
left a comment
Member
There was a problem hiding this comment.
This looks good to me! I will pull this into my Windows PR once merged. Hopefully it will end the flaking.
gvanrossum
approved these changes
Nov 27, 2018
gvanrossum
left a comment
gvanrossum
left a comment
Member
There was a problem hiding this comment.
LG. I wonder if the race condition is being exacerbated by the call to reset_global_state()? That seems to do a bunch of cleanup that looks pointless given that we're about to exit the process.
Two other things bug me about this code: the use of sys.exit(0) to break out of the loop (why not break?), and the two nested try/finally blocks, presumably needed to record whether the status file has been created. But I propose to clean that up later to avoid merge races for Ethan.
| sys.exit(0) | ||
| finally: | ||
| os.unlink(STATUS_FILE) | ||
| # If this is something other than a clean stop, remove |
Member
There was a problem hiding this comment.
Maybe s/this/the final command/?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
While thinking about what might cause flakes in #5859, I found a race
condition between
dmypy stopand otherdmypycommands.dmypy stopwill send a response and close the socket before thestatus file has been deleted. This means that there is a window in
which the daemon has reported to a client that it has exited but while
a status file still exists. This can result in a number of issues,
including the daemon appearing to be stuck (instead of stopped) to
subsequent commands and also the exiting server deleting the status
file of a subsequently started server.
The fix is to remove the status file in
cmd_stopbefore replying tothe request. This ensures that, as far as clients are concerned, the
daemon is exited after a stop command completes.
I tested the bug and the fix by inserting a
time.sleep(1)immediately before the
sys.exit(0)inserve: this caused severaltests to fail, and the changes fixed them.
I believe that the observed flakes in the windows version in #5859
were caused by this issue, but in a way that the unix version was not
susceptible to.