The GraphQL request to GitHub API is failing python.cz build

[hroncok]

I see:

ValueError: Unexpected status '500 INTERNAL SERVER ERROR' on URL /zapojse/

Every now and then. I often need to restart the build for the website to deploy.

Maybe the GitHub API call should have some retrying instead?

[honzajavorek]

We should first put there some logging to figure out why it happens to be 500 in the first place.

…

On Mon, 17 Sep 2018 at 11:18, Miro Hrončok ***@***.***> wrote: I see: ValueError: Unexpected status '500 INTERNAL SERVER ERROR' on URL /zapojse/ Every now and then. I often need to restart the build for the website to deploy. Maybe the GitHub API call should have some retrying instead? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#287>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AARTMftRbQJzSEZpTcxWC1srm2zVQwwFks5ub2adgaJpZM4WrhGa> .

[honzajavorek]

Official reply from GitHub:

Thanks for reaching out about this. Our logs indicate that you've hit a timeout with that query, as the error message suggested as a possible cause. All API requests have a 10 second limit on execution time -- if that limit is reached, the request is terminated and you get back that error.

Because execution time is not deterministic and known in advance (e.g. may be impacted by current load on database or filesystem, or changes in the underlying data), this execution time varies a bit with each request you make, even though the query is the same. Our logs don't show us the exact query you are using, but I'm guessing it involves fetching more that a few objects. And it seems that the query takes around 10s to execute -- sometimes more (and then causing the error) and sometimes less (and then completing successfully and returning the results).

What you might do here is change the query so that fewer objects are needed to process it -- that should reduce the execution time so that it is consistently under 10s and you get back the results.

[honzajavorek]

The steps to solve this would be:

1. Decide whether the "Get Involved" page is useful in its current form
2. If it's useful (which I personally doubt), try to optimize the GraphQL query
3. If it's not useful, rethink the page and then implement it from scratch with the new ideas in mind

[honzajavorek]

It's not flaky anymore. It is just failing. The last time it passed was a month ago.

[honzajavorek]

Interestingly, the page already was trying to be resilient, expecting "Zapoj se" to fail, catching and logging the exception while still rendering the page. The problem was, all the time, that the page returned 500 status code and that failed the Elsa build 🤦‍♂ I hotfixed it here: 93fc670

[frenzymadness]

I think that the idea is still useful but the list should contain only easily fixable issues which are the best starting point for a new community member.

[honzajavorek]

We could implement that with dedicated labels. Then its perhaps just a matter of rewiring the GraphQL query so it only takes issues with that label. Theres no real standard (as can be seen in https://github.com/MunGell/awesome-for-beginners#readme ), we can follow https://www.firsttimersonly.com/ or invent our own.

…

On Thu, 29 Aug 2019 at 10:30, frenzymadness ***@***.***> wrote: I think that the idea is still useful but the list should contain only easily fixable issues which are the best starting point for a new community member. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#287?email_source=notifications&email_token=AACFGMIWUCBGRZFBXG3F6GLQG6CL5A5CNFSM4FVOCGNKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5NWNJA#issuecomment-526083748>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AACFGMKQWIVLUSZMF25YK63QG6CL5ANCNFSM4FVOCGNA> .

[frenzymadness]

@honzajavorek Could you please explain how it works now? I think that use a dedicated label (zapojse or more common easyfix) and list all issues with that label from all our organizations is what we want.

[honzajavorek]

From the top of my head: there is a GraphQL query for GitHub v4 GraphQL API, which is ran for each GitHub organization (currently two, Pyladiescz and pyvec, but afaik they created sth for pydata on the sprint a few days ago). It collects issues and PRs from all repos and sorts them by a number of votes / participants.

…

On Thu, 29 Aug 2019 at 11:34, frenzymadness ***@***.***> wrote: @honzajavorek <https://github.com/honzajavorek> Could you please explain how it works now? I think that use a dedicated label (zapojse or more common easyfix) and list all issues with that label from all our organizations is what we want. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#287?email_source=notifications&email_token=AACFGMKLOASNK25GNBB66ZDQG6JZTA5CNFSM4FVOCGNKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5N4C3Y#issuecomment-526106991>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AACFGMOHY4MRVRRQ5XQNFMTQG6JZTANCNFSM4FVOCGNA> .

[frenzymadness]

With a query like this: {"query_string": "org:PyLadiesCZ is:issue state:open label:easyfix"} and GraphQL query:

query($query_string:String!) {
    search(first: 100, type: ISSUE, query: $query_string) {
      issueCount
      edges {
        node {
          ... on Issue {
            createdAt
            title
            url,
            repository {
              name
              url
            }
            author {
              login
              url
            }
          }
        }
      }
    }
}

I can imagine that it might work because the result will be smaller and will contain only issues we marked as good entry points for newcomers on purpose.

Then reason I am using $query_string as an input variable is that I cannot find a way how to put $org_name into query string in GraphQL.

IIRC @messa was working with GraphQL so he might be able to help us here.

[encukou]

A hint for pagination: https://til.simonwillison.net/github/graphql-pagination-python