-
Notifications
You must be signed in to change notification settings - Fork 10
Open
Labels
enhancementNew feature or requestNew feature or request
Description
Problem:
Oftentimes, travel cost matrices can get really big in large study areas. This creates a challenge of dealing with larger-than-memory data. The current version of {accessibility} cannot cope with this problem. Although the package is super efficient because it is built on top od {data.table}, the package still requires users to load the travel_matrix and land_use_data input to RAM memory, which can be a constraint for these larger-than-memory cases.
Proposed solution:
One solution to this would be to make the {accessibility} compatible with {arrow} (link). There are different ways to do this. The alternative I would suggest would use the following workfkow:
- If the user passes
travel_matrixandland_use_datainputs as data.tables or data.frames, then the package follows the current natural flow. - If the user passes a string path pointing to the location of
.parquetor.csvfiles that contain thetravel_matrixandland_use_datainputs, then the package follows a different flow to calculate the indicador using a{dplyr}+{arrow}syntax
Implications:
- For users, this only means more flexibility to work with larger-than-memory data and greater computation speed.
- For developers, this means we would need to make a few changes to check inputs, but most of the work would be translating the accessibility functions into a
{dplyr}syntax, which is generally super simple.
baarthur
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request