Skip to content

Conversation

@arnesetzer
Copy link

@arnesetzer arnesetzer commented Jul 29, 2025

👋 I did some awesome work for the Pelias project and would love for everyone to have a look at it and provide feedback.


Here's the reason for this change 🚀

I use the download routine from pelias to download the required osm data and extract the polylines from it. But since I don't want to import the osm data it is kinda annoying to delete them and then run the interpolation. Since the interpolation step takes quite some time it is very frustrating if you forgot this and wait that unused data is interpolated.


Here's what actually got changed 👏

Allow the pelias prepare interpolation additional arguments which data should only be used (osm, oa, tiger). So you can skip parts where data is present but not used. You can also mix up (e.g.. pelias prepare interpolation osm tiger).


Here's how others can test the changes 👀

  1. Build the docker container for interpolation and run it.
  2. Execute pelias prepare interpolation oa => only oa data should be interpolated

Reference PR: pelias/docker#385
Closes #290

@orangejulius
Copy link
Member

Another thing that has been requested many times in the past, such as #290 and elsewhere.

I think this PR looks ok but it adds a lot of length and duplication to the file. Can we have each step of the "full" interpolation process run in either the case where it was specified, or when no parameters were passed at all?

I appreciate the intent of retaining backwards compatibility by throwing the old code in an else unmodified, but I think it could lead to issues down the line if we update some places rather than others.

@arnesetzer
Copy link
Author

Removed the if/else construct and implemented the same functionality by checking the amount of additional arguments each time.

#joined catches all additional arguments given to the script into a single string
joined="$*"
if [[ "$joined" =~ "oa" || $# -eq 0 ]]; then
# run openaddresses conflation
echo "- conflating openaddresses"
$DIR/conflate_oa.sh;
fi
if [[ "$joined" =~ "osm" || $# -eq 0 ]]; then
# run openstreetmap conflation
echo "- conflating openstreetmap"
$DIR/conflate_osm.sh;
fi
if [[ "$joined" =~ "tiger" || $# -eq 0 ]]; then
# run tiger conflation
echo "- conflating tiger"
$DIR/conflate_tiger.sh;
fi
#If we don't hit any of the above, but additional arguments were given we don't run any conflation and exit with 22.
if [[ ! "$joined" =~ "tiger" ]] && [[ ! "$joined" =~ "osm" ]] && [[ ! "$joined" =~ "oa" ]] && [[ $# -gt 0 ]]; then
echo "No valid conflation source specified, running no conflations"; exit 22;
fi

@arnesetzer
Copy link
Author

Anything new here? imo is this PR ready for merge 😄

@orangejulius
Copy link
Member

@arnesetzer I've been thinking about this PR a bit more, and I realized adding more special commands to the pelias docker cli is something we'd ideally avoid. Historically, Pelias import config has been historically done mostly through pelias.json.

From a practical level, I realize most of the interpolation code is managed by a bunch of bash scripts, so reading pelias.json options is a bit unweildy (but not impossible).

Before we merge this I wanted to see if @missinglink agrees and thinks we should collectively invest the time to move some of this code into Javascript where it can be configured more easily. It would be more changes than this PR, but it would avoid creating one-off complexity in pelias/docker and keep things more in alignment.

@arnesetzer
Copy link
Author

Anything new here?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Skip importing OSM

2 participants