Revise pinning strategy

[lkluft]

The current approach to pinning the ORCESTRA data is based around creating and pinning the "HEAD CID", which is a meta-object pointing at the entirety of the data.

This was an intentional choice in the beginning, because it allowed to browser the campaign data before the data browser existed. However, this approach also comes with several downsides, e.g.:

1. It is hard to pin subsets of the dataset
2. Updates can take a while because the full tree needs to be traversed
3. Some services (e.g. Piñata) have an implicit upper limit for the size of CIDs

A possible alternative would be to actually list individual datasets in the pinlist.yaml directly. This would solve most of the technical problems right away. However, we would need to think about metadata a bit more. For example, it would be nice to handle different versions of a dataset. We already have a similar convention with the prev metadata which we use to enable incremental pinning:

cid: Qmas...
name: Dataset title
meta:
   prev: QmRpRyRX75cauJKMPSTHMrwJQpNCKLayQJScm29VMVZjkb

Maybe this is simple enough and already sufficiently robust. One could then create the history of a dataset by simply following the chain of previous entries (and add an optional version metadata).

[lkluft]

This raises the question of whether we should provide more metadata for pinlist entries, such as the file type. Currently, we mostly expect them to be HEAD CIDs, or people to just open them as Zarr anyway (most aux data). Encoding this information in the name feels like going back to file names, which we probably want to avoid. Therefore, adding information to help downstream applications open the files (as in Intake) might be helpful.

Edit: We could add MIME types. See zarr-developers/zarr-specs#123