Support directory-like "browsing" of SBAsset6 objects?

[apocalyptech]

(Very excellent library you've created, btw - thanks a bunch!)

While writing a little utility with this library, I came across the need to get "directory listings" of paths within the pakfile. Something for a possible enhancement to the SBAsset6 class, perhaps?

In my own app I added it with an ugly little class like so:

class PakTree(object):
    """
    Tree-based dict so we can "browse" pak contents by directory.
    Makes no distinction between directories and files.
    """

    def __init__(self):
        self.top = {}

    def add_path(self, pathname):
        """
        Adds a path to our tree
        """
        parts = pathname.split('/')[1:]
        cur = self.top
        for part in parts:
            if part not in cur:
                cur[part] = {}
            cur = cur[part]

    def get_all_in_path(self, path):
        """
        Gets all "files" within the given path.
        """
        parts = path.split('/')[1:]
        cur = self.top
        for part in parts:
            if part not in cur:
                return []
            cur = cur[part]
        return sorted(cur.keys())

    def get_all_matching_ext(self, path, ext):
        """
        Gets all "files" within the given path which match the given
        extension.  Note that "extension" is being used here a bit
        generously - be sure to pass in a leading dot if you want it
        to *actually* be an extension.
        """
        to_ret = []
        for item in self.get_all_in_path(path):
            if item.endswith(ext):
                to_ret.append(item)
        return to_ret

Which gets populated via:

            paktree = PakTree()
            pakdata = starbound.SBAsset6(pakdf)
            pakdata.read_index()
            for path in pakdata.index.keys():
                paktree.add_path(path)

... but that's pretty ugly, and the get_all_matching_ext method may not be something you'd want in the base package anyway. If this is something you'd be interested in having, I could try my hand at a more-suitable-for-public-consumption version.

[blixt]

This makes sense, I'd be open to a glob-like method on SBAsset6, so you can do something like pak.glob('/items/**/*.json')

Since glob is not super trivial to implement maybe there's something to do around regular expressions? Here's a standalone example to do something similar to what you have above:

import re

def generate(matcher, paths):
    for path in paths:
        match = matcher.match(path)
        if not match:
            continue
        yield match.group(1)

matcher = re.compile('^/a/([^.]+\\.gif)$')  # group 1 will be used
paths = ['/b', '/c/', '/a/test.gif']
list(generate(matcher, paths))

[apocalyptech]

Yeah, that could certainly be a way of doing it... An alternative that I'd been thinking about, as opposed to actually doing globbing/regexing/whatever is to just implement something which mimics os.walk() and leave it up to the app itself to match on filenames, etc. Anyway, I'll get back to this in a bit after it's had a chance to mull around in my head a bit.

[apocalyptech]

FYI, I suspect I'll probably never actually get around to implementing this "properly" - I'm more or less through Starbounding in general, and the hacky little class I'd put together above, in my own project, does the trick well enough for me. Do with this issue what you will! :)