Repository navigation
Infinite request loop in listObjects() if last item returned is directory #217
Description
Hello,
After testing, the issue might be unrelated to delimiters.
When making requests to list objects in containers, there are at least two requests :
The first one is a simple get (with additional parameters if present)
The second one take the last returned item's name and set it as a marker, to make sure all items were returned. The requests stop if the response is either a 204 response code or the body is empty.
If for some reason, the last item does not have the name attribute (I.E, it's a directory)
(Might happen only with directories created with the Swift web interface), the marker is empty, and thus the same request is sent over and over again (since the response is never empty).
Seems related to #214
Example of a response causing issue (via HTTP requests) :
[{u'bytes': 0, u'last_modified': u'2018-02-20T10:12:34.698650', u'hash': u'd41d8cd98f00b204e9800998ecf8427e', u'name': u'test/', u'content_type': u'application/octet-stream'}, {u'subdir': u'test/folder2/'}, {u'subdir': u'test/folder3/'}, {u'subdir': u'test/folder4/'}, {u'bytes': 7424, u'last_modified': u'2018-02-20T14:03:44.385580', u'hash': u'949493fe0b3520e7937d87e0690ba01b', u'name': u'test/virtual-machine-scale-sets.png', u'content_type': u'image/png'}, {u'subdir': u'test/zolder5/'}]
I'm guessing the script tries to set marker to the last item "name", and thus is empty
Activity
[-]Infinite request loop in listObjects() with delimiter =' /' and no prefix (or empty)[/-][+]Infinite request loop in listObjects() if last item returned is directory[/+]haphan commentedon Feb 25, 2018
@mboudet Did you encounter the issue before #214 merged?
Appreciate if you can provide minimum re-producible code.
mboudet commentedon Feb 25, 2018
The issue was actually fixed by #214 , I still had 3.0.3 locally. It works perfectly now.
Issue actually stems from Horizon with the Swift dashboard : when asked to create a folder, instead of creating a normal object, it create a "folder" object, without the "name" key.
Folder item : {'subdir': 'test/'} instead of {'name': 'test/'}
I'm guessing that the marker sent to the API is the "name" of the last item, thus the issue occurring when a folder is the last one listed.
While the issue is already fixed, it might be interesting to check if a new marker was actually set before re-sending the request, to avoid infinite loops.
haphan commentedon Feb 26, 2018
Great to hear that. I am going to release 3.0.4 now.