Create client from service account stored in string

Hi,

This was raised in python-storage: googleapis/python-storage#93

python-cloud-core library has support for from_service_account_json, which accepts a file path to a service account.

Feature request is to add support to accept a json string of the service account. User stores the service account file contents in an environment variable instead of in a file.

Open question: I'm not sure if this is a path that should be supported so I'm leaving that as an open question in this issue. If this feature is not supported then the workaround is to construct a credentials object and explicitly set a project id:

# Storage example but is a general issue AFAIU

import os
import json

from google.cloud import storage
from google.oauth2 import service_account

credentials_json_string = "{...service account....}"

credentials_json = json.loads(credentials_json_string)
credentials = service_account.Credentials.from_service_account_info(credentials_json)

# Credentials has project_id already so project should be optional but it's not.
client = storage.Client(project=credentials['project_id'], credentials=credentials)

print(list(client.list_buckets()))

Proposal:

    @classmethod
    def from_service_account_json_string(cls, json_credentials, *args, **kwargs):
        """Factory to create credentials with provided json_credentials.
        :type json_credentials: str
        :param json_credentials: The contents of a private key file. This string must contain
                                      a JSON object with a private key and
                                      other credentials information (downloaded
                                      from the Google APIs console).
        :type args: tuple
        :param args: Remaining positional arguments to pass to constructor.
        :param kwargs: Remaining keyword arguments to pass to constructor.
        :rtype: :class:`_ClientFactoryMixin`
        :returns: The client created with the provided JSON credentials.
        :raises TypeError: if there is a conflict with the kwargs
                 and the credentials created by the factory.
        """
        if "credentials" in kwargs:
            raise TypeError("credentials must not be in keyword arguments")
        credentials_info = json.load(json_credentials)
        credentials = service_account.Credentials.from_service_account_info(
            credentials_info
        )
        if cls._SET_PROJECT:
            if "project" not in kwargs:
                kwargs["project"] = credentials_info.get("project_id")

        kwargs["credentials"] = credentials
        return cls(*args, **kwargs)

@busunkim96

@busunkim96 are you an owner of Python-cloud-core? Not sure who I should ask if this is something that we'd like to support.

@frankyn

@frankyn Yes!

If this is added to google-cloud-core it should also be added to the generated clients.

https://github.com/googleapis/python-asset/blob/15b60a349c93c928fe121dc47d44d812a0c14439/google/cloud/asset_v1p1beta1/gapic/asset_service_client.py#L53

Since this will impact all the clients, I'll bring it up tomorrow at the Python client weekly.

I just want to point out, this issue contains a secondary feature request. The original request provides this example:

# Credentials has project_id already so project should be optional but it's not.
client = storage.Client(project=credentials['project_id'], credentials=credentials)

In fact project_id is optional today, but bad things happen if you don't specify it (see googleapis/python-storage#177 for an example issue)

So the secondary feature request here is: if credentials are explicitly provided, then the client should skip autoloading. This behavior would be good whether the credentials are loaded from a file or string.

@rvandegrift

@rvandegrift The secondary request is now tracked in #27, and should be resolved in PR #51.

@frankyn The spelling of the @classmethod factory in google-auth taking a dictionary, rather than a filename, is from_service_account_info. ISTM that we could add that factory, and have users responsible for decoding the serialized values from whatever envvar they like, stored in whatever format:

class _ClientFactoryMixin(object):
    ...
    @classmethod
    def from_service_account_info(cls, info, *args, **kwargs):
        """Factory to load JSON credentials while creating client.

        :type info: dict
        :param info:
            A JSON object with a private key and other credentials information
            (downloaded from the Google APIs console).

        :type args: tuple
        :param args: Remaining positional arguments to pass to constructor.

        :param kwargs: Remaining keyword arguments to pass to constructor.

        :rtype: :class:`_ClientFactoryMixin`
        :returns: The client created with the retrieved JSON credentials.
        :raises TypeError: if there is a conflict with the kwargs
                 and the credentials created by the factory.
        """
        if "credentials" in kwargs:
            raise TypeError("credentials must not be in keyword arguments")

        credentials = service_account.Credentials.from_service_account_info(info)

        if cls._SET_PROJECT:
            if "project" not in kwargs:
                kwargs["project"] = credentials_info.get("project_id")

        kwargs["credentials"] = credentials
        return cls(*args, **kwargs)

The existing from_service_account_json would then be just a wrapper around the new method, loading the JSON credentials file.

@busunkim96 I created googleapis/gapic-generator-python#705 to track adding this method for the microgenerator. Do we need to add support for it in the macrogenerator too?

@tseaver

@tseaver Nope, just the microgenerator is fine. The monolith is almost gone (we're down to 2 libraries). :)

feat: add 'Client.from_service_account_info' factory

feat: add 'Client.from_service_account_info' factory (#54)

Navigation Menu

Repository navigation

Create client from service account stored in string #8

Proposal:

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Create client from service account stored in string #8

Description

Proposal:

Activity

frankyn commented on Apr 20, 2020

busunkim96 commented on Apr 20, 2020

rvandegrift commented on Jun 16, 2020

tseaver commented on Dec 10, 2020

busunkim96 commented on Dec 10, 2020

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions