Benchmarks for bindings

[countvajhula]

Summary of Changes

We don't have any benchmarks making use of bindings.

Public Domain Dedication

• In contributing, I relinquish any copyright claims on my contribution and freely release it into the public domain in the simple hope that it will provide value.

[countvajhula]

Any other patterns that might be useful to benchmark @benknoble ? Do we feel that bindings would have any nonlocal interactions worth benchmarking, e.g. would checking (~> sqr add1 (as v) (gen v)) tell us anything more than just (~> (as v))?

I tried this with your change to boxes and here are the results before and after:

set!

$ ./report.rkt -s as

Running local (forms) benchmarks...
as: 70 ms
...

boxes:

$ ./report.rkt -s as

Running local (forms) benchmarks...
as: 89 ms
...

Average over 5 runs:

flow.rkt> (~> (70 82 97 88 85) (-< + count) / round)
84
flow.rkt> (~> (89 94 92 85 84) (-< + count) / round)
89

They're about the same. Honestly, I'm not sure our local benchmarks might be dominated by the cost of constructing lambdas and such, so that we may not actually be seeing anything specific here about bindings.

[benknoble]

When you only construct the binding, you don't do anything useful... that is, the code is basically (let ([v <binding>]) (set! v <input>) (values)). So I would expect increased code size from boxes and the box-set! to dominate perf.

It might also be that the compiler output is complicated enough to defeat the Chez optimizer anyway, in which case this won't matter (I never did prove that using boxes re-enables the optimizations disabled by set!). But it might still matter at the module level.

I would be interested in a module that Chez can optimize noticeably, but where using set! creates a large slow-down by disabling optimizations, and then seeing if changing to boxes "fixes" it.

Fortunately I think if we can find a small pure optimizable function, the set! just has to be elsewhere in the module to turn off optimizations? Unfortunately, I don't know of such an example.

[countvajhula]

Fwiw I asked Bard which optimizations are turned off by use of set! and it came up with:

• Common Subexpression Elimination (CSE)
• Constant Folding
• Dead Code Elimination
• Loop Optimizations
• Inlining

We could try coming up with an example exhibiting as many of these as possible...