Fix precision of mapproj_match_offsets.txt lat/lon coords, general improvements to bundle_adjust text file output

[dshean]

In reviewing output with @bpurinton and @ShashankBice this morning, we realized that the lat/lon coordinates in the mapproj_match_offsets.txt text file (https://stereopipeline.readthedocs.io/en/latest/tools/bundle_adjust.html#registration-errors-on-the-ground) only have 4-5 decimal places, which offers limited precision (meters to 10s of meters, see https://en.wikipedia.org/wiki/Decimal_degrees#Precision). Other outputs like residuals csv have 7-8 decimal places. The precision for lat/lon coordinates throughout ASP should always be sufficient to preserve cm differences (at least 8).

Also, would it seems like we could combine some of these text files, adding columns to create a larger csv instead of creating separate text files? The issue is that match point order is not consistent across these different files, which complicates integration after the fact. In other words, if specified, just append the mapproj columns to the residuals csv, so we have one-to-one correspondence for each match point, rather than trying to figure this out after the fact.

Regardless, we should consider more self-consistent output text file formats created by bundle_adjust - now we have a mix of csv and txt with variable formatting. The use of these files is relatively limited, and if we have centralized functions to read and plot, impact on users is limited.

Including a header with column names (can use known escape character % or #) would simplify parsing and issues around reading csv files with different sets of columns based on user-specified flags.

[oleg-alexandrov]

This file was saved with 8 digits of precision as well, but since the lon-lat values have 3 digits before the decimal point, only 5 digits were saved after the decimal. I now made the total number of digits be 12 (so 9 after decimal point).

The reason different files are used is because other residual files do not assume a reference DEM is used in bundle adjustment.

Then, the pointmap.csv file has one residual for triangulated point, which may have many matches corresponding to it, while this file has one residual per pair of match points, so the bookkeeping is different.

I am open to sorting this out better going forward. For now I found this file to be of limited usefulness, with the stats.txt file having more value as that one returns one single measure of registration quality per image.

More work on this can happen going forward.

[dshean]

Great! Glad it was an easy fix.

Thanks for reminder about requirements for the different files. And agreed that the stats file contains lots of useful information. Let's revisit potential improvements later.

We started discussing ways to systematically compare the "observed" horizontal and vertical residuals in mapproj_match_offset.txt and pointmap.csv, respectively, with the "predicted" horizontal and vertical errors from propagation of the vendor estimates (ERRBIAS, ERRRAND for Maxar). In principle, the initial residuals should be similar to predicted errors, while the final residuals should be much smaller.

Still thinking about how we can update these vendor-provided error estimates for the adjusted cameras based on some of these metrics. We also need a general approach for bundle_adjust to account for known reference DEM (or GCP) horizontal and vertical uncertainty, beyond the current user-defined weights.