Repository navigation
Documentation on how to access Azure stores #92
Description
I am trying to use zappend to append to an Azure data store. However, I cannot figure out what the configuration should be for that case.
What I tried so far:
config = {
"target_dir": "https://mystore.blob.core.windows.net/mycontainer/myfile.zarr",
"target_storage_options": {
"account_name": "name",
"account_key": "key",
}
}
zappend(ds_code, append_dim="time", config=config)This fails with TypeError: ClientSession._request() got an unexpected keyword argument 'account_name'
I also tried:
from azure.storage.blob import ContainerClient
container_client = ContainerClient.from_connection_string(conn_str="my_string", container_name="mycontainer")
# store = zarr.ABSStore(client=container_client, prefix = "myfile.zarr") this works
config = {
"target_dir": "https://mystore.blob.core.windows.net/mycontainer/myfile.zarr",
"target_storage_options": {
"client": container_client,
}
}
zappend(my_dataset, append_dim="time", config=config)This fails with TypeError: ClientSession._request() got an unexpected keyword argument 'client'
It would be nice to have it in documentation how you access datastores other local data and S3.
Thank you
Activity
forman commentedon May 28, 2024
@caiostringari, sorry I missed your report! I agree, it would be good to give examples for other common datastores.
As you can see from the code,
zappendpassestarget_storage_optionsdirectly to the underlying data store. Therefore the valid keyword arguments are prescribed by the specificfsspecfilesystem implementation. Look here for Azure:https://github.com/fsspec/adlfs?tab=readme-ov-file#details
okz commentedon Jun 11, 2024
Hi, to add @forman's note, I don't think a
https://would correctly get decoded as an azure path.the following configuration patterns worked for me