RFE: server-side option

[joshmoore]

In talking to the iRODS imaging WG, a new use-case occurred to me. A script (or other remote method) could run on the server and wouldn't need to download files locally since the execution is server-side. Either a flag such as --server or a new command could "pack" just the metadata and then on unpacking the files could be referenced in their original location.

A regular version of such a script could be used to transfer files to a new OMERO more efficiently, while an iRODS specific version of the script could be used to archive a packed version of the data to more permanent storage. (In either case, the script might need extra guarantees to prevent unintended access to files.)

[erickmartins]

So essentially just generating a serialization of the relevant database bits in a XML/ome-types compliant file? I can see the benefit way down the road, but is the use case right now restoring stuff "selectively" from storage + metadata files instead of storage + database dump?

[joshmoore]

The overarching use case is generally, "what's the best mechanism for exchanging between a multi-headed system like OMERO and iRODS?" I proposed omero-cli-transfer 😄. One of the main drawbacks of trying to store OMERO stuff on iRODS is that iRODS doesn't have a concept of image, so putting a bunch of files in requires extra layers of metadata to know what's what. Instead, iRODS can track packed collections. So in that sense, yes "selecting" or "subsetting" data out of OMERO and migrating/archiving it to iRODS (and potentially restoring) is the starting point.

[erickmartins]

Got it. Have a bunch of stuff on my plate right now (all good things!), but I will have a look at this - possibly in a month or two. Otherwise, PRs always welcome :)

[joshmoore]

👍. Maybe ping if you do start to hack anything. I'll bring it up at the next couple of hackathons I go to and maybe something will get underway...

[glyg]

Is it more complicated than stopping the pack method there?

Would then adding transfer.xml as a file annotation to the args.object suffice?

[joshmoore]

Is it more complicated than stopping the pack method there?

Except there's another rocrate block below, so it might need something a bit more from it getting too complicated.

Would then adding transfer.xml as a file annotation to the args.object suffice?

I was looking for having it locally with the intent of running an immediate unpack --server, so --attach might be "Yet-Another-Option".

[glyg]

Alright, I think I can give it a try.

I guess attaching can be left to the user. On second thought, I am not even sure there is a point to it as the transfer.xml is a snapshot, so you can't trust it for long.

with the intent of running an immediate unpack --server

For the unpack, wouldn't unpack --folder be enough? I don't get what the --server option would do in unpack

[glyg]

huhu --server is already used by omero, confusion ensues. Would --metadata-only be ok, although it's a bit verbose?