strmxor -u -l wrongly reports differences on identical layers

[stefanottili]

Testing hierarchical xor on https://github.com/RTimothyEdwards/gf180mcu_ocd_sram_test
shows that while -u is correct, -u -l wrongly reports layers as different that are not.

strmxor -u -n=8 -d=11 gds/gf180mcu_ocd_sram_top.gds.gz gds/gf180mcu_ocd_sram_filled.gds.gz xor.hier_no_l.oas | tee hier_no_l.oas

  Layer      Output       Differences (hierarchical shape count)
  ----------------------------------------------------------------
  22/4       -            (no such layer in first layout)
  30/4       -            (no such layer in first layout)
  34/4       -            (no such layer in first layout)
  36/4       -            (no such layer in first layout)
  42/4       -            (no such layer in first layout)
  46/4       -            (no such layer in first layout)
  81/4       -            (no such layer in first layout)

strmxor -l -n=8 -d=11 -u gds/gf180mcu_ocd_sram_top.gds.gz gds/gf180mcu_ocd_sram_filled.gds.gz xor.hier.oas | tee hier.log

  Layer      Output       Differences (hierarchical shape count)
  ----------------------------------------------------------------
  22/4       22/4         1
  30/4       30/4         1
  34/4       34/4         1
  36/0       36/0         10130
  36/4       36/4         1
  42/4       42/4         1
  42/10      42/10        125
  46/4       46/4         1
  81/0       81/0         51766
  81/4       81/4         1
  204/10     204/10       5

feature request: it would be nice to have the flat shape count too.

running xor flat confirms that only the tile layers differ.

strmxor -l -n=8 -d=11 -m -p=500 gds/gf180mcu_ocd_sram_top.gds.gz gds/gf180mcu_ocd_sram_filled.gds.gz xor.flat.oas | tee flat.log

  Layer      Output       Differences (shape count)
  ----------------------------------------------------------------
  22/4       22/4         96858
  30/4       30/4         88975
  34/4       34/4         560748
  36/4       36/4         525834
  42/4       42/4         488536
  46/4       46/4         439208
  81/4       81/4         316790

[klayoutmatthias]

I added a patch that solves the original issue.

However, the most interesting part of that bug is that the -n 8 case is much slower than the -n 1 case!

The reason is that for multiple threads, the outputs are collected in 8 different containers that are consolidated. This step is single-threaded and takes too much time. It's somewhat more efficient to do that at the end (this is included in the patch). Still that takes too long.

With the patch and the given example:

• 8 threads: XOR takes about 7s, consolidation abouth 17s
• 1 thread: XOR takes about 20s, consolidation <1s

I will try to optimize that.

Matthias

[stefanottili]

  Layer      Output       Differences (shape count)
  ----------------------------------------------------------------
  22/4       22/4         1 (hierarchical) 96858 (flat)
  30/4       30/4         1 (hierarchical) 89067 (flat)
  34/4       34/4         1 (hierarchical) 560748 (flat)
  36/4       36/4         1 (hierarchical) 525834 (flat)
  42/4       42/4         1 (hierarchical) 488536 (flat)
  46/4       46/4         1 (hierarchical) 439208 (flat)
  81/4       81/4         1 (hierarchical) 316790 (flat)

Total: 18.24 (user) 0.54 (sys) 7.345675193 (wall) 829.14M (mem) -n=8
Total: 6.21 (user) 0.21 (sys) 6.426689267 (wall) 243.23M (mem)