Listing large numbers of objects

[bhagemeier]

Hi,

it seems that the Swift API, which is ultimately called for requests to the CDMI API, has an upper page size limit of 10000 elements. We have a container that has more than 10000 elements under a single (pseudo) path.

Containers (pseudo-directories) with more than 10000 elements must be listed through pagination (last element of a page given as marker to start the next page).

The implementation of cdmiutils.check_resource() does not do this at the moment. It merely lists the first 10000 elements and returns them.

Cheers,
Björn

[osaddon]

Yes, need more patch work to support pagination.