Repository navigation
Add support for JSON API headers and query string parameters #30
Description
Description
The XML-Api and the Json-API are supporting a large set of parameters as described at https://cloud.google.com/storage/docs/xml-api/reference-headers.
Into those parameters you can find very useful features like x-goog-metageneration which let you control how to deal with versions and also blocking rewriting of blobs in a bucket.
These API used to be available in the AppEngine library for google cloud storage as you can see at https://cloud.google.com/appengine/docs/standard/python/googlecloudstorageclient/functions#open
Proposition
Adds support for extended options on different operations like upload_from_string and delete, something like:
def upload_from_string(self, data, content_type='text/plain', client=None,
predefined_acl=None, options=None):Which would like you pick and sets of options from https://cloud.google.com/storage/docs/xml-api/reference-headers
Activity
tseaver commentedon Jun 26, 2018
Docs for the JSON API headers / query string parameters. Where they differ, we would be following the latter, because we never directly invoke the XML API (
Blob.public_urlandBlob.generate_signed_urlcreate URLs against the XML API endpoint, but don't invoke them).tseaver commentedon Aug 7, 2018
Of the "standard" HTTP headers, I can see that there might be a usecase for supplying:
Cache-Control(override default 3600 value; only supported for fetching blobs accessible to all anonymous users).Range(download partial content)If-Match/If-None-Match(check an ETag)AFAICT, the JSON API only supports one "extension" header which is not covered / superseded by existing
google-cloud-storageusage:PATCHrequests through obtuse firewalls)tseaver commentedon Aug 7, 2018
Of the the query string parameters supported by the JSON API, we might want to support:
prettyPrint(remove whitespace from JSON responses)quotaUser(enforce per-user quotas from a server-side application even in cases when the user's IP address is unknown)userIp(enforce per-user quotas when calling the API from a server-side application.)The others are ether superseded by the library, or are tied to deprecated authentication schemes / JSON-P.
tseaver commentedon May 24, 2019
With the release of
google-cloud-core 1.0.0, theConnectionclass now exposes anextra_headersproperty, which can be used to pass whatever additional headers are needed (but not on a per-request basis:mlhamel commentedon May 24, 2019
Thanks for the updates I'm guessing it means we can now close this issue.
tseaver commentedon May 24, 2019
@mlhamel I don't think the new
Connection.extra_headerssupport is exactly what this issue wanted: it is kind of an "always on" configuration, rather than one which takes effect per-request.[-]Storage: Missing support for HTTP Headers and Query String Parameters[/-][+]Add support for JSON API geaders and query string parameters[/+]daniellehanks commentedon Mar 16, 2021
Any update on this? I stumbled upon this needing to specify an
If-None-Matchheader. Basically my use case is my API is a thin layer over GCS (for storing profile images), and I only want to actually read the image data and serve it to the client if the actual content is different from their cache. They will specify anIf-None-Matchheader to my API (as they won't know anything about GCS or generation numbers, kind of the point), so I just need to essentially pass this along to GCS. I don't want to either do the extra round tripping of reading the current file to check the etag and then use the generation number from that in a subsequent request, or not use the client library.daniellehanks commentedon Mar 23, 2021
@tseaver Are you still a good resource for this ticket? Is this ticket still a good place to request etag conditional headers support or should I open a new ticket?
tseaver commentedon May 24, 2021
@daniellehanks Adding conditional support for
If-Match/If-None-Matchshould be a separate feature request from this issue -- implementing it would touch tons of places (pretty much every method which already hasif_generation_match/if_metageneration_matchwould ), and It would need design (how does the user ask for the test? Should it use theetagof the resource by default? etc.)This feature would need to be commissioned by the GCS DPE team.
/cc @frankyn, @tritone, @andrewsg .
[-]Add support for JSON API geaders and query string parameters[/-][+]Add support for JSON API headers and query string parameters[/+]daniellehanks commentedon May 25, 2021
Thanks @tseaver. Opened #451.
cojenco commentedon Apr 8, 2025
Done via #489 and #1121