Potential improvements to the repo benchmarks

[erezrokah]

Problem or use case

This is not a feature request for the CLI, it's a suggestion for improving the current (already great) benchmarking suite (tried to start a discussion for it, but couldn't #752).

I tried using existing benchmarks to see if there are low hanging fruits to improve CPU and memory allocation, but the profiles generated using pprof are dominated by test setup code due to in memory go-git operations, preventing from seeing any application code in the profiles.

You can see below pprof memory profile for benchmarks of the cli package

Most of the allocations are test related work.

Desired behavior

To identify bottlenecks it would be great if the benchmarks CPU and Memory profiles would consist of mostly application code and not test code.

Proposed solution

The proposed solution would be to reduce usage of go-git when setting up tests, and offload that work to the git CLI. Please note the purpose here is not to make the tests run faster (they won't) or consume less memory (they won't), but to offload work outside the main test Go process so it's easier to identity bottlenecks in application code.
There are other potential ways to deal with test setup code impacting profiling of application code, but this seems to be the most aligned with the current approach in the codebase (I've seen other places where go-git usage was replaced by git CLI invocation calls).

See below memory profile with relevant changes

Where it's easier to identify allocations for application code.
This fix will help with CPU profiles as well

Alternatives or workarounds

Haven't looked into it but maybe it would be possible to pre-create variations of NewBenchRepo then use those instead of recreating them each time.
Not sure it's worth the effort, as the goal is not to make the benchmarks faster/more efficient (at least for now), but to be able to distinguish between test code to application code when using them for profiling.

Another option is to separate the concerns, and create new dedicated set of tests only for profiling.