Right now we have a very simple and basic pyramid generation scheme.
We
- walk over all features
- for every feature save
(prefix, geoms) e.g. [("1", [b0, .. ]), ("10", [b0, ..]), ..]
- once we have these tuples, union all values on a tuple-by-tuple basis
There are a lot of duplicate union calls going on here and the memory usage is pretty high. Instead what we should do is build up the pyramid from the bottom to the top recursively
- union all boxes with the same 5-char prefix (should be singleton)
- union all boxes with the same 4-char prefix using the boxes from the step before
- recurse
Note: before we do this, run on all of Germany once to see if we need this performance improvement. If we don't need it, let's not do it.