You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
BigQuery: Implement progress_bar_type support when bqstorage_client is used #7654
We now support displaying a progress bar in to_dataframe when the tabledata.list API is used, but not when the bqstorage_client is used. The BigQuery Storage API is faster, but not instantaneous, so it would still be useful to display a progress bar.
There are a couple of complications with this.
When the BigQuery Storage API is used, a threadpool is used to download query results. The progress bar should only be updated from a single thread, so the threadsafe queue should probably be used from the worker threads to send updates to the main thread that updates the progress bar.
to_dataframe in the BigQuery Storage API client only supports downloading a whole stream at once. The reader needs to be updated to provide a pages property to allow converting individual blocks to a pandas DataFrame. That way incremental updates can be sent.
Activity