GC-ing epoch files #1
Description
Hey @kellabyte, so cool to see this project!
I saw a todo in the code on removing stale epoch files eventually. This reminded me of an interesting discussion I had with someone after publishing the article. Essentially, if there are multiple nodes, the GC process may not remove any file which still may be attempted to be created by any of the nodes. This is what the person wrote to me about this, and I agree with this assessment:
The background process only keeps the last file . So if node 1 starts the algorithm step 1 & before it completes step 4 , say 2 other nodes have increased the lock epoch by +2 , and background process clears the last minus 1 file , there's a chance node 1 would still end up decreasing the epoch value.
I think a safe way would be to only remove those epoch files which have been created before the currently running nodes were started, if that is possible.