Fix false concurrent conflict limit threshold detection between non-concurrent metafile revisions

[pdames]

Summary

These changes resolve #585 by ensuring that only metafile revisions with the same revision number are counted against DeltaCAT's max concurrent write conflict limit (vs. prior behavior of treating all revisions in a revision directory partition as potential conflicts). All changes required to resolve #585 are in metafile.py, with corresponding tests in test_metafile_io.py.

Other Changes (not directly related to #585):

1. Tests to ensure that manifest entry order is preserved during delta download and merge operations.
2. Fix concurrent log file rotation and log file sizing bugs in the new rolling log file appender (Updated rolling log file appenders to have 64MiB cutoff and compress … #579).
3. Flaky test hardening (test_file_object_store.py, test_default_catalog_impl.py: use test-level mocks over conflicting class-level mocks, resolve signal timing bugs, thread naming race conditions, and add fallback to handle unexpected failures of concurrent test case writers).

FYI @IvanPartsunev.

Testing

Wrote failing unit tests and ensured that they passed post-update.

Checklist

• Unit tests covering the changes have been added

  • If this is a bugfix, regression tests have been added

• E2E testing has been performed

[Zyiqin-Miranda]

Is it correct that the current log will always be log.1? Seem like the for i in range(self.backupCount - 1, 0, -1): will have log.1 as well during the for loop?

[pdames]

Yeah, this is intentional since the previously loop should have vacated the log.1 slot (due to excluding the ending range of 0 from inclusion in the loop) which is populated here by the current log.

For example, if backupCount = 10, then we get: [9, 8, 7, 6, 5, 4, 3, 2, 1] then the above loop should:
Move log.9 -> log.10 (i.e., push the oldest log out)
Move log.8 -> log.9
... and so on down to log.1 ...
Move log.1 -> log.2 (i.e., make room for a new log.1)

So finally here on line 234-235 we move current -> log.1 (now empty)