python3 system tests fail to use gRPC

@tseaver

@tseaver Can you confirm? I'm seeing basic imports failing.

======================================================================
ERROR: test_log_struct (logging_.TestLogging)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/GoogleCloudPlatform/gcloud-python/system_tests/logging_.py", line 149, in test_log_struct
    entries, _ = _retry_backoff(_has_entries, logger.list_entries)
  File "/home/travis/build/GoogleCloudPlatform/gcloud-python/system_tests/logging_.py", line 37, in _retry_backoff
    from grpc.beta.interfaces import StatusCode
ImportError: cannot import name 'StatusCode'

@dhermes

@dhermes Looks like they need to dump the PYTHONPATH=_testing bit.

But grpcio successfully installs in the Python 3 environment otherwise?

UPDATE: Or you're saying that the GAX version wouldn't be used if the import failed?

I've run the Py3k system tests with grpcio installed (and the gax-* libraries), but suppressing having _testing on the PYTHONPATH.

On my machine pip3 install grpcio fails with

could not find grpc_python_plugin (protoc plugin for GRPC Python)

I could put grpc_python_plugin on my path but that won't help Travis.

I'm beginning to think we should back out all the "just pretend grpc works" stuff we've done (make it a pure extra), disabled by default. I also want to shoot the _testing directory in the head.

The _testing directory was just there for unit tests (so imports didn't break), never intended to be used in system tests. The real culprit is grpcio.

I think unit tests don't really need it, either: the bigtable tests could just use `unittest2.skipUnless' like the GAX-related tests in logging and pubsub.

At the time it was added, there was no Travis environment where grpcio installed, so that was the only way. I'm fine with your suggestion given the current state of things.

W00t! https://pypi.python.org/pypi/grpcio/1.0.0rc1/#downloads has wheels for Python 3.4 on Linux/Mac/Windows!

I'm surprised they're at / near a 1.0

I wonder if pinning the RC would unbreak Travis system-tests3 and Appveyor.

Apologies for the gap in 3.4-on-Windows.

@tseaver

@tseaver https://ci.appveyor.com/project/daspecster/gcloud-python-utjkc/build/job/ibd6g2k1a4gkumjo

Installed c:\users\appveyor\appdata\local\temp\1\easy_install-idvc8i\grpcio-1.0.0rc1\.eggs\markupsafe-0.23-py2.7-win32.egg
warning: no files found matching '*.h' under directory 'src\python\grpcio\grpc'
warning: no files found matching '*.python' under directory 'src\python\grpcio\grpc'
warning: no previously-included files matching '*.so' found under directory 'src\python\grpcio\grpc\_cython'
warning: no previously-included files matching '*.pyd' found under directory 'src\python\grpcio\grpc\_cython'
warning: no files found matching 'src\python\grpcio\precompiled.py'
cygrpc.c
include\grpc/impl/codegen/compression_types.h(38) : fatal error C1083: Cannot open include file: 'stdbool.h': No such file or directory
a.c

@daspecster

@daspecster Looks like Appveyor is trying to build from source on Windows for Python 2.7.

https://pypi.python.org/packages/3f/81/b14dc4e143d1eb0efabb0c8a4d88fafc6798d6392e26867f4e45a44d617a/grpcio-1.0.0rc1.tar.gz#md5=ce2462a64aa86670696a50c6d947d5e6

It should be downloading the wheels...

I think the issue is that pip install grpcio won't grab he RC version so it's trying a pre-1.0 release. Either that or your wheel doesn't support Windows?

Note that here you'll need to use the --pre flag to get 1.0.0rc1 unless you want to be grabbing it during nosetests as part of the command's potentially problematic gathering of (constrained to egg or .tar.gz?) dependencies (AFAICT).

@soltanmm

Thanks @soltanmm, I actually just forced the version with pip install grpcio==1.0.0rc1 for now to try it in my branch.

https://ci.appveyor.com/project/daspecster/gcloud-python-pjqjn

Some success and some more failures :(
https://ci.appveyor.com/project/daspecster/gcloud-python-pjqjn/build/job/1gpl7vqkrafyvjcu

64-bit 3.4 fails with

    include\grpc/impl/codegen/compression_types.h(38) : fatal error C1083: Cannot open include file: 'stdbool.h': No such file or directory
    creating Users
    creating Users\appveyor

while both versions of 3.5 fail with an OpenSSL issue

IIRC grpcio can be built within a MinGW 64-bit Windows environment with vanilla Python 3.4 (NOT e.g. MSYS2's) if configured to use the GCC toolchain via e.g. -c mingw32. grpcio-tools will fail with linker errors that might be alleviated by compiling with TDM-GCC (haven't tried that yet) or finishing up C++98-ifying the plug-in code enough for MSVC 2010 to handle it.

It looks like y'all just need grpcio, not grpcio-tools, though. Is that the case?

EDIT: Turns out I recalled incorrectly and that grpcio also fails with linker errors.

@soltanmm

@soltanmm Correct: users of gcloud-python only need grpcio: developers may need grpcio-tools for regenerating proto-based files, but we don't have any Windows-based developers, and we don't do that task via Windows CI.

Via #2065, #2085.

refactor: Remove duplicated comments in _function_session.py (#2001)

Navigation Menu

Repository navigation

python3 system tests fail to use gRPC #2001

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

python3 system tests fail to use gRPC #2001

Description

Activity

tseaver commented on Jul 20, 2016

dhermes commented on Jul 20, 2016

tseaver commented on Jul 20, 2016

dhermes commented on Jul 20, 2016

tseaver commented on Jul 20, 2016

dhermes commented on Jul 20, 2016

tseaver commented on Jul 20, 2016

dhermes commented on Jul 20, 2016

tseaver commented on Jul 21, 2016

dhermes commented on Jul 21, 2016

tseaver commented on Jul 21, 2016

nathanielmanistaatgoogle commented on Jul 22, 2016

daspecster commented on Jul 22, 2016

soltanmm commented on Jul 22, 2016

dhermes commented on Jul 22, 2016

soltanmm commented on Jul 22, 2016

daspecster commented on Jul 22, 2016

daspecster commented on Jul 22, 2016

dhermes commented on Jul 25, 2016

soltanmm commented on Jul 26, 2016

tseaver commented on Jul 26, 2016

tseaver commented on Aug 25, 2016

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions