Skip to content

Pubchem Retrieval Script Update #473

Closed

Description

@jjacobson95

Pubchem may have made some rate limit updates and seems to be more burdened than before. Increase the retry time. And add a secondary (a tertiary) sweep to capture missing drugs at the end (just wrap current script in for loop).

This will ensure that when when Errno 101 (Network is unreachable) occurs for several minutes, these drugs are not missing in final result.

This will also ensure that Errno 503 (Rate Limit) occurs, we back off and resume more effectively to avoid cascading 503 errors.

Activity

  1. self-assigned this
    on May 15, 2026
  2. added this to the 2.4 new build milestone on Sep 22, 2026
  3. jjacobson95 commented on Nov 2, 2026

    @jjacobson95
    CollaboratorAuthor

    Done on main: PubChem requests now back off on 503s (honouring Retry-After) and missing drugs get up to 3 retry sweeps. Used for the 2.4 rebuild.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions