Repository navigation
Document queue polling behavior better #359
Description
It may be surprising to users that when a job running on a LRM completes, PSI/J may not immediately pick up the status update because the queue status is only queried at certain intervals (by default 30 seconds). This should probably be made more clear where users are likely to read.
See, e.g., #358
Activity
j-woz commentedon Mar 10, 2023
I think the enum JobState could capture this. Maybe split NEW into "not yet submitted" and "not yet polled"?
hategan commentedon Mar 10, 2023
Not sure. There's an initial polling delay of two seconds. So, after 5 seconds, it has been polled once.
andre-merzky commentedon Mar 11, 2023
For this to be useful and consistent, wouldn't one need to do that for every state? Like, the system may have the job
FAILEDwhile psij thinks it's stillRUNNINGbecause the update has not been pulled just yet. In other words: the locally exposed state is a cached value, and we know that cache invalidation is indeed one of the hard problems. I would agree that relying on clear documentation is more viable.j-woz commentedon Mar 11, 2023
Yes, the user should know that the state is old, but that is different from the state of a job that has never been submitted or polled. Right now, "not yet polled" returns None, but I think that documented enum values for these cases would be more clear.
hategan commentedon Mar 11, 2023
If PSI/J knows that there is a new state, there is no point in reporting an old state and marking it as out of date. You can simply report the new state instead.
That's not quite the case.
wait()specifically returns None when you have a timeout parameter and it times out. And that's independent of whether the job was polled or not.The polling state (whether a job was polled or and when) is not meant to be conveyed to the user. There is only an upper bound on how old the state can be from PSI/J's perspective. But there is no absolute upper bound. The queuing system may have arbitrary delays, and the queue querying may introduce arbitrary delays.