stor CLI wish list

[kristjaneerik]

Here are some things I'm thinking of implementing for the CLI.

I'm opening the issue to solicit feedback / comments & keep track of progress.

stor ls , stor list, stor walkfiles

• with no path should just use ./
• default to a long listing format that has fixed-width columns: file size ("human readable" / KiB/MiB/GiB/TiB as appropriate), modification time (%Y-%m-%d %H:%M:%S, except if --relative-time set; then e.g. 3 d), mimetype, filename
• -l / --simple-list: only show the filename column
• -b / --use-bytes: just show # of bytes with no unit
• default sorting should be alphabetical?
• -S / --sort-by-file-size: sort by file size, smallest first
• -t / --sort-by-time: sort by modification time, newest first
• -U / --sort-by-directory-order: I believe this is the current default
• -r / --reverse: reverse the sort order
• -u / --url: rather than filename (or in addition to?), print the URL, e.g. for swift/s3 it would be https://..., for Unix it would be file://...
• -T / --tabs: rather than fixed width use tabs as separators for long format columns
• --tree: for list and walkfiles indicate shared prefixes (directories) with a visual tree

stor cp

• multiple inputs
• -p / --progress: some sort of progress indicator, e.g. % of files and/or % of bytes done
• -s / --skip-existing: if destination file exists, skip it. Could check the modification time and size and if they don't match, replace the target file with the source one (or add a flag to control this behavior, e.g. like rsync -u / --update which skips files that are newer on the receiver). I don't know if an rsync-like checksum check is feasible. Probably the trickiest thing on this list.

stor cat

• multiple inputs

stor rm

• multiple inputs

stor mv

• this doesn't exist, but would be nice to have, even if not possible to do cleanly on the server side for e.g. swift. Sounds tricky in any case.

stor touch

• this doesn't exist, but would be nice to have. Would only apply to files (if exists, update modification time; if doesn't, create empty file and relevant directories). Might be tricky?

stor stat

• this doesn't exist, but would be nice to have. Would mimic swift stat

stor verify

• this doesn't exist, but would be nice to have. Would check checksum of SRC and DEST; printing out missing / extra files (see comments).

stor swift

• would expose get-tenant, get-container, get-object/get-resource (see comments)
• also exposes get-url to get an HTTPS url for a swift:// path

Most of the work is being done in #34.

@jtratner -- thoughts/comments/suggestions? Does anything seem too tricky to implement or too resource-heavy (e.g. the stuff for stor ls --long)?

[jtratner]

my big wish would be stor stat and stor list with similar output to swift list --lh

[jtratner]

There are a few helpers I wanted to expose:

$ stor swift get-object swift://AUTH_rnd/C/whatever
whatever
$ stor swift get-container swift://AUTH_rnd/C/whatever
C
$ stor swift get-tenant swift://AUTH_rnd
AUTH_rnd

stor mv would have to be COPY then DELETE.

BUT I think it would be nice to have something like a:

stor verify $SRC $DEST

that checks all the md5sums between src and dest and confirms that $DEST is a superset of $SRC, and prints out missing / extra files. Would make it easier when doing lots of copies.

Additionally, I was thinking regexes on list / ls would be pretty nice too.