Architecture

@Ogaday

Let's discuss (feel free to stop me if there was too much discussion and no code!) the client architecture here

As mentioned in #3 I'd suggest structuring out the client in the following spirit (this is just a mock-up):

class Endpoint:
    """Endpoint implements all HTTP methods
    serves as a base class for the real endpoints.
    """
    def __init__(self, root, path, token):
        self.path = path

    def _get(**params):
        return requests.get(
            urllib.parse.urljoin(root, path),
            headers={'x-storage-token':self.token},
            params=params)


# either make a subclass
class BucketEndpoint(Endpoint):
    def listBuckets(self, *args, **kwargs):
        return self._get(*args, **kwargs)

class StorageClient:
    def __init__(self, token):
        self.token = token
        self.root = 'http://something.com'
        self.buckets = BucketsEndpoint(self.root, 'buckets'):
    
# and use like this:

client = StorageClient('apitoken')
client.buckets.listBuckets(id='1234')

but @Ogaday mentioned he was developing his own client, so he might already have an alternative. How do you feel about it?

Didn't mean to hit close!

I like this. I was only focused on managing workspaces, so I wasn't thinking high level! If I'm not mistaken, this architecture is most similar to the mailchimp client, like mailchimp entity ~ kbcstorage endpoint with some key differences, such as the parent class handling the requests rather than the child class?

The only thing I'd like to think about is how to handle the asynchronous requests, eg. loading tables to workspaces. I think we'd want to return a reference to a job object that could be polled, and I'm not sure how to handle that. I previously created a StorageJob with a poll method, but I'm not sure if that's the best way to approach it.

AD async: I'd keep it simple and return the async response the api returns and let the user handle the processing (with a possible blocking argument which would dumbly wait for completion)

class Endpoint:
    def _get(*args, **kwargs):
        pass
    def wait_for_async_request_to_complete(self, async_resp):
        """ Poll the url in async_response every X seconds and return when done
        """
        pass
class TablesEndpoint(Endpoint):
    def get_table_async(self, id, blocking=False):
       # this is the json response containing the polling url
        async_resp = self._get('kbcapi.com/tables', {'resource_id': id}) 
        if blocking:
            resp = self.wait_for_async_request_to_complete(async_resp)
            return resp # the final response
        else:
            return async_resp # let the user handle it

I'll probably take a closer look at the mailchimp client (and possibly others) if we can mimick a proven architecture. I haven't stuided it yet in great detail.

Yeah I was thinking that would be the solution - keeps it clean and consistent.

So functionality would be like this:

>>> # imports, constants etc
>>> client = kbstorage.Client(token)
>>> resp = client.tables.create_table_async(data)
>>> job_id = resp['id']
>>> while client.jobs.detail(job_id)['status'] not in ('error', 'success'):
        time.sleep(x)
>>> # job is complete

How does that seem? If that's the case I'd be happy to not handle async requests specially.

@pocin

I had a look at mailchimp and it seems that the client handles async requests / resources in exactly the way you propose @pocin 👍 (without the blocking arg and I agree it's probably not necessary to have an option to block)

rather than having a blocking argument, i'd suggest two methods, sth along the line import_table (blocking) and create_import_table_job (non-blocking returns resource), the first one uses the latter, but someone for some reason needs the non-blocking one, he can call the low level method. Typical usecase is blocking though (many people struggle with implementing the polling correctly).

as for the structuring, i agree with structuring along the endpoints, though the class could be called only Buckets and not BucketsEndpoint.

current version: endpoint classes, encapsulating Client class, async requests supported -> complete

Navigation Menu

Repository navigation

Architecture #6

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Architecture #6

Description

Activity

Ogaday commented on Aug 16, 2017

pocin commented on Aug 16, 2017

Ogaday commented on Aug 16, 2017

Ogaday commented on Aug 16, 2017

odinuv commented on Aug 17, 2017

odinuv commented on Aug 17, 2017

odinuv commented on Sep 5, 2017

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions