Skip to content

How to test gcloud applications #1822

Closed
Closed
How to test gcloud applications#1822
Assignees
lukesneeringer
Labels
testing

Description

@txomon
txomon
opened on May 26, 2016

Hello,

I have seen there are some efforts to create a local environment for testing, such as the datastore emulator, and the pubsub emulator, but I haven't seen more or documentation on how to use them.

One of the main blocks for the development with gcloud AFAI see is that there is not an standard to setup a integration testing environment. Certainly the ability to transparently connect and authenticate to upstream services is nice, but in order to develop, in the teams I work in, we really need to have a local testing environment.

The library is easy to mock, but I would highly appreciate having an standard way for developers have their testing environment, and documentation on how to do it.

I understand this effort is under gcloud beta emulators, but I have failed to find a way to integrate them with this library.

Activity

  1. daspecster commented on May 26, 2016

    @daspecster
    daspecster
    on May 26, 2016
    Contributor

    Hey @txomon!
    Are datastore and pubsub the main services that you need to emulate for your testing environments or are there other services that you rely on as well?

  2. txomon commented on May 26, 2016

    @txomon
    txomon
    on May 26, 2016
    Author

    @daspecster

    The documentation I am looking now is for Datastore and Pubsub, but these are the services we are already using:

    • Appengine with credentials/service account
    • Datastore
    • Pub sub
    • Metadata service from compute
    • Cloud Logging - logs are already being taken through stdin/stdout but we would love to have this implemented in the library. I will open a ticket for this one.
    • Cloud SQL - Is already testable because it's Mysql

    If ongoing reported issues are resolved, we will be migrating also to:

    • Storage
    • Bigquery

    Storage and bigquery are still waiting for approval and to have a proper environment, but the others are already in use.

  3. dhermes commented on May 26, 2016

    @dhermes
    dhermes
    on May 26, 2016
    Contributor
    • The only emulators that exist (as of now) are only for Datastore and Pub/Sub
    • Usage of the emulators is included in our CONTRIBUTING doc. However, the code is written to support the emulators as-is. That is, if you use the docs for the emulators (which say to run gcloud beta emulators FOO start and then $(gcloud beta emulators FOO env-init)$), your code will just talk to the emulator (we pick up the env. vars. that get set).
    • More emulators is outside the scope of this library and something the service owners would need to provide
  4. dhermes
    added
    testing
    on May 26, 2016

  5. theacodes commented on May 26, 2016

    @theacodes
    theacodes
    on May 26, 2016
    Contributor

    @txomon if you were developing using another cloud provider's services, how would you do system testing?

    I ask this question because the emulators are currently pretty limited, so the best recommendation for testing is to either do integration testing (with mocks) or do system testing against a separate project.

  6. txomon commented on May 26, 2016

    @txomon
    txomon
    on May 26, 2016
    Author

    @dhermes

    1. Ok, I will be checking for new modules if created
    2. It looks nice, if that info could be documented in the official docs that would be awesome (for visibility)
    3. Totally understand that, I was just giving a list on what we are using.

    @jonparrott

    We basically run with rabbitmq instead of pubsub, dynamodb has an emulator as datastore does, and the rest of the databases are opensource (Postgres/Mysql). Redis and memcached are both opensource, so no problems there, and for metadata we used environment variables.

    I agree however that Google Cloud Store may not make much sense, but for BigQuery it does definitely make sense to have something (a provided mock or something).

  7. majuscule commented on Nov 30, 2016

    @majuscule
    majuscule
    on Nov 30, 2016

    Hi all,

    I wanted to second @txomon's request/suggestion. I am looking for a compute backend mock and was disappointed to find that no support is provided. I was unaware of gcloud beta emulators before reading this issue, but even if compute was supported (which it is not), I am wary of requiring a separate process for testing (see #2 (comment)).

    To answer your question @jonparrott, AWS cloud services have relatively comprehensive mock coverage provided by https://github.com/spulec/moto.

    google.appengine.ext.testbed seems to be the closest thing provided by Google to this, but is again lacking in service coverage.

  8. theacodes commented on Nov 30, 2016

    @theacodes
    theacodes
    on Nov 30, 2016
    Contributor

    Thanks for the feedback.

    To answer your question @jonparrott, AWS cloud services have relatively comprehensive mock coverage provided by https://github.com/spulec/moto.

    This is mocking the library, whereas we prefer to provide an emulator that will work across languages. Once this library reaches 1.0 it might be worthwhile to look into providing some sort of mocking for the library, but I'm unsure how useful that'll be in light of the emulators.

    I am wary of requiring a separate process for testing (see #2 (comment)).

    We are as well, but that applies to unit tests which explicitly don't need to hit the live services. Our system tests do indeed hit both lives services and the emulators (if available).

  9. txomon commented on Nov 30, 2016

    @txomon
    txomon
    on Nov 30, 2016 via email
    Author

  10. dhermes commented on Dec 2, 2016

    @dhermes
    dhermes
    on Dec 2, 2016
    Contributor

    @txomon We do have support in the library for Datastore, Pub / Sub and Bigtable emulators. You can run our system tests against the emulators via:

    $ cd google-cloud-python/
$ tox -e bigtable-emulator
$ tox -e datastore-emulator
$ tox -e pubsub-emulator

    See: https://github.com/GoogleCloudPlatform/google-cloud-python/blob/d775489da48d594ce5173a4078af4a0d3d2863f4/CONTRIBUTING.rst#system-test-emulators

  11. lukesneeringer
    closed this as completedon Apr 19, 2017
  12. JustinBeckwith
    assigned
    lukesneeringer
    on Feb 1, 2021
  13. parthea
    added 2 commits that reference this issue on Mar 2, 2026

    feat: support Python 3.14 (#1822)

    78dc85d

    chore: librarian release pull request: 20251212T161150Z (#1888)

    54ec638
  14. parthea
    added a commit that references this issue on Apr 1, 2026

    chore: refactor IsNullOp and NotNullOp logic to make scalar ops gener…

    03e350b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

testing

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions