Error when running BiqQuery query with APPROX_QUANTILES

OS type and version - Ubuntu 16.04.1 LTS
Python version and virtual environment information python --version - 2.7.12
google-cloud-python version pip show google-cloud, pip show google-<service> or pip freeze - 0.21.0
Stacktrace if available

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-23-1528bd7c380f> in <module>()
----> 1 job.rows()

/home/mtebeka/python-docs-samples/appengine/flexible/hello_world/venv/local/lib/python2.7/site-packages/google/cloud/bigquery/query.pyc in rows(self)
    225         :returns: fields describing the schema (None until set by the server).
    226         """
--> 227         return _rows_from_json(self._properties.get('rows', ()), self.schema)
    228 
    229     @property

/home/mtebeka/python-docs-samples/appengine/flexible/hello_world/venv/local/lib/python2.7/site-packages/google/cloud/bigquery/_helpers.pyc in _rows_from_json(rows, schema)
    114 def _rows_from_json(rows, schema):
    115     """Convert JSON row data to rows with appropriate types."""
--> 116     return [_row_from_json(row, schema) for row in rows]
    117 
    118 

/home/mtebeka/python-docs-samples/appengine/flexible/hello_world/venv/local/lib/python2.7/site-packages/google/cloud/bigquery/_helpers.pyc in _row_from_json(row, schema)
    105         if field.mode == 'REPEATED':
    106             row_data.append([converter(item, field)
--> 107                              for item in cell['v']])
    108         else:
    109             row_data.append(converter(cell['v'], field))

/home/mtebeka/python-docs-samples/appengine/flexible/hello_world/venv/local/lib/python2.7/site-packages/google/cloud/bigquery/_helpers.pyc in _float_from_json(value, field)
     33     """Coerce 'value' to a float, if set or not nullable."""
     34     if _not_null(value, field):
---> 35         return float(value)
     36 
     37 

TypeError: float() argument must be a string or a number

Steps to reproduce - Run a query with APPROX_QUANTILES.
Code example

from google.cloud import bigquery

client = bigquery.Client(project='PROJECT-ID')

sql = '''
SELECT
APPROX_QUANTILES(
  CASE
     WHEN ctr_before = 0 THEN NULL
     ELSE (ctr_after - ctr_before) / ctr_before
  END,
  100) quantile
FROM
 `ltv.user_ctr`
 '''
job = client.run_sync_query(sql)
job.use_legacy_sql = False
job.run()
job.rows

@tebeka

Thanks for reporting @tebeka!

@tseaver @dhermes I think this needs to support array types in the response right?
See: https://cloud.google.com/bigquery/docs/reference/standard-sql/functions-and-operators#approx_quantiles

@tebeka

@tebeka Can you print the value of job._properties['rows'] in the failing case?

In [8]: job._properties['rows']
Out[8]: 
[{u'f': [{u'v': [{u'v': u'-1.0'},
     {u'v': u'-1.0'},
     {u'v': u'-1.0'},
     {u'v': u'-1.0'},
     {u'v': u'-1.0'},
     {u'v': u'-1.0'},
     {u'v': u'-1.0'},
     {u'v': u'-1.0'},
     {u'v': u'-1.0'},
     {u'v': u'-1.0'},
     {u'v': u'-1.0'},
     {u'v': u'-1.0'},
     {u'v': u'-1.0'},
     {u'v': u'-1.0'},
     {u'v': u'-1.0'},
     {u'v': u'-1.0'},
     {u'v': u'-1.0'},
     {u'v': u'-1.0'},
     {u'v': u'-1.0'},
     {u'v': u'-1.0'},
     {u'v': u'-1.0'},
     {u'v': u'-1.0'},
     {u'v': u'-1.0'},
     {u'v': u'-1.0'},
     {u'v': u'-1.0'},
     {u'v': u'-1.0'},
     {u'v': u'-1.0'},
     {u'v': u'-1.0'},
     {u'v': u'-1.0'},
     {u'v': u'-1.0'},
     {u'v': u'-1.0'},
     {u'v': u'-1.0'},
     {u'v': u'-1.0'},
     {u'v': u'-0.8864864864864864'},
     {u'v': u'-0.8303571428571428'},
     {u'v': u'-0.7902330743618202'},
     {u'v': u'-0.7543859649122807'},
     {u'v': u'-0.7235023041474654'},
     {u'v': u'-0.693069306930693'},
     {u'v': u'-0.6666666666666666'},
     {u'v': u'-0.6375'},
     {u'v': u'-0.6117216117216117'},
     {u'v': u'-0.5855263157894737'},
     {u'v': u'-0.5608856088560885'},
     {u'v': u'-0.5353535353535354'},
     {u'v': u'-0.5114155251141552'},
     {u'v': u'-0.4864864864864864'},
     {u'v': u'-0.4625'},
     {u'v': u'-0.43750000000000006'},
     {u'v': u'-0.41442715700141447'},
     {u'v': u'-0.3902308105206656'},
     {u'v': u'-0.3653846153846154'},
     {u'v': u'-0.34146341463414626'},
     {u'v': u'-0.31677018633540377'},
     {u'v': u'-0.2923766816143497'},
     {u'v': u'-0.2666666666666667'},
     {u'v': u'-0.24242424242424238'},
     {u'v': u'-0.21590383726306067'},
     {u'v': u'-0.19047619047619052'},
     {u'v': u'-0.16396903589021816'},
     {u'v': u'-0.13746630727762796'},
     {u'v': u'-0.10954235637779944'},
     {u'v': u'-0.08196721311475408'},
     {u'v': u'-0.05263157894736842'},
     {u'v': u'-0.02325581395348833'},
     {u'v': u'0.0038560411311053186'},
     {u'v': u'0.036920659858601754'},
     {u'v': u'0.06928066037735833'},
     {u'v': u'0.10179640718562884'},
     {u'v': u'0.1366666666666667'},
     {u'v': u'0.17073170731707327'},
     {u'v': u'0.20923076923076936'},
     {u'v': u'0.2484848484848484'},
     {u'v': u'0.28813559322033905'},
     {u'v': u'0.33333333333333326'},
     {u'v': u'0.3749999999999999'},
     {u'v': u'0.42051282051282063'},
     {u'v': u'0.46875000000000006'},
     {u'v': u'0.5217391304347826'},
     {u'v': u'0.5766423357664235'},
     {u'v': u'0.6363636363636365'},
     {u'v': u'0.7'},
     {u'v': u'0.7692307692307692'},
     {u'v': u'0.8400000000000001'},
     {u'v': u'0.9230769230769231'},
     {u'v': u'1.0'},
     {u'v': u'1.103723404255319'},
     {u'v': u'1.206896551724138'},
     {u'v': u'1.333333333333333'},
     {u'v': u'1.456140350877193'},
     {u'v': u'1.6085043988269796'},
     {u'v': u'1.7777777777777775'},
     {u'v': u'1.9889502762430937'},
     {u'v': u'2.210526315789473'},
     {u'v': u'2.5000000000000004'},
     {u'v': u'2.857142857142857'},
     {u'v': u'3.352941176470588'},
     {u'v': u'4.0'},
     {u'v': u'5.083333333333333'},
     {u'v': u'7.2'},
     {u'v': u'1987.9999999999998'}]}]}]

@tebeka

I believe #2787 addressed this issue: it fixed marshalling arrays.

The imminent 0.22.0 release of google-cloud-bigquery will contain the fix. @tebeka are you set up to test with the current git master branch?

Since we believe this has already been fixed, I am going to close this. Please re-open if it is still an issue on the current version of the library.

Navigation Menu

Repository navigation

Error when running BiqQuery query with APPROX_QUANTILES #2804

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Error when running BiqQuery query with APPROX_QUANTILES #2804

Description

Activity

daspecster commented on Dec 5, 2016

tseaver commented on Dec 5, 2016

tebeka commented on Dec 6, 2016

tseaver commented on Dec 6, 2016

lukesneeringer commented on Feb 28, 2017

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions