Benchmark tests

[InnovativeInventor]

Here are some simple benchmarks to diagnose performance issues and quantify overhead. Looks like the printing_delay argument doesn't help performance overhead. https://github.com/InnovativeInventor/ProgressBars.jl/runs/3172796733?check_suite_focus=true#step:4:47

If I get some free time, I'll dig into this deeper. I've noticed that sometimes writing to stdout/stderr is faster than printing (I think). Any thoughts on how to reduce the overhead?

[khdlr]

That's odd, I definitely remember it making a difference on my machine. Have you run these tests on your local machine as well to check the performance there?

If so, what's your setup (i.e. OS, Terminal, etc.)?

[InnovativeInventor]

I ran these tests on my local machine and got similar results. I'm running a flavor of Ubuntu and using Alacritty (side note: alacritty is awesome). Do you get the same results on your machine?

[khdlr]

Alacritty is probably one of the fastest terminals available, and might be doing a lot of internal caching before display - would be interesting to test this on a slower terminal that is widely used (e.g. the js-based ones in VSCode/Atom).

Never forgave Alacritty that it doesn't support font ligatures, so I'm a Kitty guy myself 😉

[khdlr]

Upon running the tests myself, I'm wondering where the BenchmarkTools pipes its outputs to, does it just silently discard the output?

If so, this might be the reason we're not seeing any difference from the printing_delay function, since the printing/flushing of the output stream can never be the bottleneck.

[khdlr]

Ah, I just saw that you're manually setting the output to /dev/null for the benchmark. I would suggest to leave the output_stream at the default and instead specify leave=false. This will delete each one of the benchmark progress bars, but still provide a more "realistic" setting.

Also, on my machine, I have to go to very small values of printing_delay to notice a performance impact (e.g. printing_delay=0.00001, which corresponds to 10 µs).

[khdlr]

Here are a number of benchmarks I ran, it seems I judged VSCode too soon! 😂

XTerm

URXVT

Konsole

Alacritty

Kitty

VSCode Terminal

[InnovativeInventor]

Ah, ok that makes sense. Just pushed some changes with a smaller printing delay. I see what you're seeing on my local machine now (when the printing delay is set small).

Have you profiled ProgressBars.jl? If not, I may profile it this weekend to see if I can find any easy optimizations.

[khdlr]

Good to see that you can reproduce the benchmark results that I'm getting as well.

No I haven't gotten around to profiling the library. If you can find the time, that would be great!