Index v2 (ocaml-kc / kv-hash)

[tomjridge]

A possible replacement for Index is here: https://github.com/tomjridge/kv-hash

It is also informally known as: ocaml-kc (for Kyoto Cabinet - an inspiration for the implementation) and kv-hash (i.e., a key-value store based on a persistent hashtable approach) and possibly index-v2.

If we replace the index in this repo with the kv-hash alternative, then a lot of the current issues disappear. In particular, the "blocking merge" problem is not observed with kv-hash during irmin tree.exe bench replay of Tezos traces.

I'm opening this issue mainly so that there is a record that a lot of the other issues may be redundant. As an example, here are some issues that might be redundant if we move to index-v2:

• io_array's get_entry_from_io shouldn't be called #368
• Investigate using mmap instead of preads and pwrites #350
• Test I/O failures #349
• Index.v is racy #340
• Close fd when aborting a merge #325
• Semaphore find_instance blocks for up to 2 min #323
• Index.try_merge doesn't check for an ongoing merge #311
• etc etc