Repository navigation
BigTable: row.commit throwing _Rendezvous exception instead of ServiceUnavailable #4656
Description
We are using retry.py and creating a retry exception predicate as follows:
predicate=retry.if_exception_type(exceptions.ServiceUnavailable)However, since we are only getting _Rendezvous exceptions with StatusCode.UNAVAILABLE (see below stack trace), we had to change this to:
predicate=retry.if_exception_type(exceptions.GrpcRendezvous)I see a different issue in cloud datastore for mapping gRPC codes to service level exceptions:
Will this also be done for cloud bigtable?
Stack Trace:
File "single_key_read_write.py", line 135, in write
row.commit()
File "/home/jenkins/.local/lib/python3.5/site-packages/google/cloud/bigtable/row.py", line 445, in commit
retry_(retry_commit)()
File "/home/jenkins/.local/lib/python3.5/site-packages/google/api_core/retry.py", line 261, in retry_wrapped_func
on_error=on_error,
File "/home/jenkins/.local/lib/python3.5/site-packages/google/api_core/retry.py", line 178, in retry_target
return target()
File "/home/jenkins/.local/lib/python3.5/site-packages/google/cloud/bigtable/row.py", line 256, in __call__
client._data_stub.MutateRow(self.request_pb)
File "/home/jenkins/.local/lib/python3.5/site-packages/grpc/_channel.py", line 492, in __call__
return _end_unary_response_blocking(state, call, False, deadline)
File "/home/jenkins/.local/lib/python3.5/site-packages/grpc/_channel.py", line 440, in _end_unary_response_blocking
raise _Rendezvous(state, None, None, deadline)
grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with (StatusCode.UNAVAILABLE, Transport closed)>
Activity
dhermes commentedon Jan 2, 2018
@zakons In order for the "new" exceptions to be in place, the Bigtable gRPC / GAPIC surface will need to be regenerated.
@lukesneeringer Can we make this happen?
sduskis commentedon Jan 2, 2018
@dhermes: IIUC, the Bigtable client doesn't actually use GAPIC.
FWIW, I'm all for regenerating the protobuf and gRPC artifacts. I'd also be happy if we can migrate to GAPIC.
dhermes commentedon Jan 2, 2018
That's correct. Bigtable was the very first package to use
protocwith thegrpcextension and we still have aMakefileto build thegoogle.cloud.bigtable._generatedpackage.This very much should be retired in favor of the "modern" autogen infrastructure.
zakons commentedon Jan 3, 2018
Thanks. Just to clarify, after the regeneration we would get a mapping of the Rendezvous exception to a ServiceUnavailable exception? Curious to know how this code generation works so that the code in api_core/google/api_core/exceptions.py for ServiceUnavailable is used.
theacodes commentedon Jan 3, 2018
Yes
The generator generates wrappers for the underlying grpc client (stub in grpc terminology). One of the wrappers applied to all RPC methods is
google.api_core.grpc_helpers.wrap_errorswhich handles re-mappinggrpc.RpcErrorsto appropriate exception classes.wrap_errorsis applied to RPC methods bygoogle.api_core.gapic_v1.method.wrap_methodwhich is called by the generated code (example).tseaver commentedon Mar 16, 2018
@jonparrott I'm not sure if it is you or @lukesneeringer who needs to do the non-re-generation of the GAPIC source from the protos.
theacodes commentedon May 1, 2018
This should be all covered by #5178