Skip to content

Build pyramid bottom-up to improve performance and memory usage #6

@daniel-j-h

Description

@daniel-j-h

Right now we have a very simple and basic pyramid generation scheme.

We

  • walk over all features
  • for every feature save (prefix, geoms) e.g. [("1", [b0, .. ]), ("10", [b0, ..]), ..]
  • once we have these tuples, union all values on a tuple-by-tuple basis

There are a lot of duplicate union calls going on here and the memory usage is pretty high. Instead what we should do is build up the pyramid from the bottom to the top recursively

  • union all boxes with the same 5-char prefix (should be singleton)
  • union all boxes with the same 4-char prefix using the boxes from the step before
  • recurse

Note: before we do this, run on all of Germany once to see if we need this performance improvement. If we don't need it, let's not do it.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions