Increasing `--max-block-depth` results in incredibly long generation times

[john-h-k]

I am trying to use csmith to generate stress-testing files that are large (~1m lines).

Version:

csmith 2.3.0
Git version: 30dccd73b

Running

csmith --max-block-depth 5

Is fast.

Then attempting

csmith --max-block-depth 10

either takes <1s or takes 10+ minutes (has not yet finished on my computer when it hits the "slow" path).
Is this expected behaviour? (and if so, is there a better way to generate very large files with csmith?)

[eeide]

Hi! A maximum block depth of 10 is license to generate some really enormous code, and we did not tune Csmith for that case.

The wide variance in run time is expected, I would say, since --max-block-depth is a maximum, not a minimum. I expect that when Csmith finishes quickly, it probably made random choices that resulted in a relatively small program. When it makes random choices that lead to a lot of code being generated, it takes a lot longer, as you have found!

Beyond just generating a lot of code, I'm not really sure what the specific cause of "extreme slowness" would be. There are a few places in Csmith where the code-generation strategy is to try something, and if it doesn't work out, backtrack. It could be that you are hitting these situations often when the block depth is so high. (But it would take profiling to know if this is the real cause.)

Here are a couple of things you might try, singly or in combination, to generate large-ish programs quickly:

• Increase the maximum block size and/or the maximum number of functions, not the maximum block depth.
• Disable the use of pointers in generated code; pointers cause Csmith to do more generation-time work.
• Put a time limit on Csmith, and if it runs too long, abort and try again.
• Put a filter after Csmith, and discard programs that don't meet the minimum size you want.

Please let us know if some combination of the above works well for you!