JUNOCam DEM tutorial - Different results than expected

[Pietro-Matt]

Hi !

I've been following the tutorial on how to produce DEMs and related products based on JUNOCam images (https://stereopipeline.readthedocs.io/en/stable/examples/junocam.html#junocam), but somehow my results are inconsistent with those shown there - and I'm pretty sure that the scripts are matching!

I attach here two different scripts, process_edit_Feb26.sh that corresponds to the 8.16.1 - 8.16.9 steps, and refinement_DTM_edit_Feb26.sh that corresponds to the steps from 8.16.10 till the end.

I also attach the two resulting DEM (unrefined and refined): as you can see many "holes" are present in the unrefined one combined_dem_2.tif, together with heavy artifacts in the form of striping, that then are transferred to the refined DEM combined_adj_dem_8.tif.

The refined DEM looks good enough, but I just can't understand why the results are somehow different from the tutorial, given that I've followed the described steps in detail.
Also, I'd like to process a Shape-from-Shading DEM based on these previous ones, and the artifacts/issues just propagate.

I hope the issue is clear enough, thanks in advance for any help!
Pietro

[oleg-alexandrov]

For the record, here's what you seem to get:

And the doc has this:

What you see is more erosion, so missing lines of pixels. The JunoCam images have little overlap, and one has to be careful to avoid erosion as then there is no overlap.

Note that you get that erosion only at top and bottom DEMs. I think if you add more DEMs in the sequence those areas will be covered too, then it will look almost as what I have.

One can also tweak some params to reduce erosion I think. So one can set --corr-kernel 3 3 (the default is 5 5 for asp_mgm), then do --subpixel-kernel 5 5 (instead of 7 7), and --edge-buffer-size 0 (the default is positive so erodes more).

The most important param here is --edge-buffer-size. There was a fix for the default for that too, but I forgot when I put it in the build.

I think adding more DEMs itself should be enough, but note that the DEMs really barely overlap. You can try the new params as backup option.

If you get better results with these I could update the doc too.

I did not try SfS on this. SfS is tricky. Here you have just single illuminaton. You can try, and you may get something plausible, but this will require some changes in parameters. See our sfs examples. Many of them have different smoothness weight, depending on the instrument and ground sample distance.

[Pietro-Matt]

Hi Oleg, thanks a lot for your quick and detailed answer.

I tried both ways (adding more JUNO framelets - I guess that's what you mean when you suggest to add more DEMs in the sequence - and tweaking the mentioned parameters in parallel_stereo. Unfortunately, neither provided much improvement.

1. This is the script process_edit_Feb26.sh in which I've added two (four) JUNO framelets at the top and bottom of the initial area (framelets JNCR_2021158_34C00001_V01_0008 and JNCR_2021158_34C00001_V01_0014 and corresponding ones for the JNCR_2021158_34C00002 image), without tweaking other parameters, obtaining this DEM combined_dem_4.tif

2. While when I tweak:

--corr-kernel 3 3 (the default is 5 5 for asp_mgm), then do --subpixel-kernel 5 5 (instead of 7 7), and --edge-buffer-size 0 (the default is positive so erodes more)

I get combined_dem_3.tif, which doesn't show much change compared to the initial result.

3. And, for the record, when I both add more JUNO framelets and tweak params, I get combined_dem_5.tif, which also doesn't look very good.

It seems like there is an inherent issue somewhere else in the script maybe?

I just don't get why I should get different results in the first place, given that the scripts are identical (I think, since I can't see the full tutorial script).

Do you have any other suggestions/ideas in this regard?

Thanks a lot again,
Pietro

[oleg-alexandrov]

Here is your combined2 on left, and combined 5 on right. To me this is suggesting that on the right not all possible overlapping stereo pairs were used. But I could be wrong.

The issue with having a hard time getting overlaps is real. But note that your solution on the left is not bad at all, and maybe there's still some pairs could be added to fill that gap.

One sanity check is to infill that gap in your best DEM, with dem_mosaic (see the doc for growing a DEM in its tool manual page) then mapproject the images onto that. Can blur that DEM with a sigma of 3 maybe too to avoid artifacts. That will give you an idea of which left and right image cover it best and then you can try running only that stereo pair rather than every single combination.

You can also try mapprojecting at a lower grid size, so not 1000 m but maybe 800 m pixel. That should hopefully give 20% more pixels and less erosion.

I am guessing you already spent quit some time on this, but what I am suggesting should be quick I guess since you would experiment with a single most promising stereo pair which would fill that gap. For this experiment you could use the original params which seems you got the smallest gap to start with.

[oleg-alexandrov]

To add, since you already mapproject, per the doc, there's no need to do that thing I wrote above with creating a new DEM. Just overlay your left and right mapprojected images onto the combined DEM, see which left-right stereo pair covers that gap you still see, and check that among your stereo pairs you do run that one. Then, mapprojecting at 800 m only that stereo pair which should be quick to do may result in more coverage I hope.

[Pietro-Matt]

Dear Oleg,

It worked!

Actually my script was pairing only the framelets with identical numbers, and not all that overlapped.

I fixed it and the result is pretty good, see below (:
galileo_ortho_6.tif

Thanks a lot for the kind help!
Pietro

[oleg-alexandrov]

Yes, I can see your new figure

There is still a little hole that could be in-filled for purpose of SfS if need be.

Further up and down the sequence the images start showing less detail and more of the planet edge, so results degrade somewhat. That is expected.

Glad things worked.

[Pietro-Matt]

Didn't realize that I just sent the orthoimage, so I also send the DEM if you want to have a look combined_dem_6.tif

There is still a little hole that could be in-filled for purpose of SfS if need be.

Speaking of SfS, I had very different results with using "sfs" or "parallel_sfs" programs, is it expected?
I will in any case now try again with the updated DEM

[oleg-alexandrov]

Not sure. I got another private report on this a few months ago and it seems things worked back then.

Maybe you can refine with jus a few iterations or keep some params fixed? I ran this exercise very many times on both Europa and Ganymede and it seems to have worked. Maybe can also try just very few images first and also with a large --heights-from-dem-uncertainty, say 10x more than in the doc, to try to nail it hard to the DEM. Should not be necessary but go figure.

Also take a look at the interest points if you get enough of them.

sfs and parallel_sfs should give similar enough results. parallel_sfs calls sfs on each tile and mosaics. You can simply peek at parallel_sfs run, inside there at subdirs for sfs, and inspect each individual DEM before mosaic. Maybe each individual DEM is fine but the mosaic is messed up? If so, it would be important for me to know. That mosaic tool had some big logic changes recently and I thought all was well but go figure.

BTW, the browser does not show the .tif files and I need to each time download them on Windows machine and copy to Linux machine to be able to see them. It may be easier to post screenshots inline.

[oleg-alexandrov]

Correction, to tighten things one needs small --heights-from-dem-uncerainty rather than big. Though I doubt that's the issue. The bundle adjustment, ideally, should make very localized changes in either case, it should just reduce distortion a bit. Running a single stereo pair in bundle adjust and then stereo and comparing to before should be a quick enough check. Also can peek inside the .json files and see how they they compare to before.

[Pietro-Matt]

Thanks a lot for the SfS tips Oleg!
I'll try them soon, but first I have to manage to get a good version of the refined JUNO DEM, as so far only the unrefined version of the DEM is coming out nicely (what I've shown earlier on)

As of now the refined DEM (point 8.16.10 in the tutorial) is not coming out as expected by running refinement_DTM_edit_Feb26.sh:

This one uses the same JUNO framelets stereo pairs as in the initial DEM processing script (points 8.16.1 to 8.16.9) that gave a result matching with that of the tutorial (shown here below):

Probably I'm just overseeing something, but can't understand what at the moment..

Could you maybe tell me what this sentence means in terms of script? From the tutorial point 8.16.10:

Lastly, stereo is run with the optimized model state camera files

I interpret it as in the code below, but the result is the not very satisfying one that I posted (the whole script is refinement_DTM_edit_Feb26.sh

pairs=(
  # Original pairs (09-13)
  "09 09"  "09 10"
  "10 10"  "10 11"  "10 09"
  "11 11"  "11 12"  "11 10"
  "12 12"  "12 13"  "12 11"  "12 10"  "12 09"
  "13 13"  "13 12"  "13 11"  "13 10"  "13 09"  
)

for p in "${pairs[@]}"; do
    set -- $p
    li=$1   # left index
    ri=$2   # right index

    l="JNCR_2021158_34C00001_V01_00${li}"
    r="JNCR_2021158_34C00002_V01_00${ri}"
    pref=stereo_final/${l}_${r}/run

    parallel_stereo                                 \
      map_adjusted/$l.map.tif map_adjusted/$r.map.tif \
        ba_rfne/adjusted/run-$l.adjusted_state.json \
        ba_rfne/adjusted/run-$r.adjusted_state.json \
        --ip-per-image 20000 \
        --stereo-algorithm asp_mgm \
        --subpixel-mode 9 \
        --subpixel-kernel 7 7 \
        --nodata-value 0 \
        ${pref} \
        ref/flat_dem_galileo.tif

[oleg-alexandrov]

I looked at your script. You are doing things correctly. It seems that you get a DEM with less distortion from before, but apparently not satisfactory enough. I can't tell for sure since you show no range of heights for your color images.

Things on the right appear more rough.

It is suggested to plot both DEMs with same colormap.

Maybe tighten --heghts-from-dem-uncertainty to 1000 but should not be needed.

I interpret it as in the code below, but the result is the not very satisfying one

Yeah, you are supposed to do bundle adjust, when you float all intrinsics, share all intrinsics, constrain with DEM, mapproject with adjusted cameras, and run stereo with adjusted cameras and newly mapprojected images. Which seems you are doing based on the script.

Also check if you have enough interest point matches on the right, where you get worse distortion. That can be done with stereo_gui --pairwise-matches. That tool manual has more info.

[Pietro-Matt]

It is suggested to plot both DEMs with same colormap.

Yes they have the same color map -1500 to 1500 m, same as the tutorial

[Pietro-Matt]

Maybe tighten --heghts-from-dem-uncertainty to 1000 but should not be needed.

I am a little confused, isn't this a command just for SfS? here I'm still in the previous refinement processing, not SfS yet. Basically processing the DEM that then will be used for SfS

[oleg-alexandrov]

No, --heghts-from-dem-uncertainty is for bundle_adjust. For now you have 5000, which maybe it is enough and maybe tigthening won't change things, but that is the only knob I can think about. Set to 1000 and see. Maybe also 2x more iterations?

Looks like your results show more variability in height both before and after bundle adjust, compared to the reference figure in https://stereopipeline.readthedocs.io/en/stable/examples/junocam.html#intrinsics-refinement.

I am at a bit of a loss. Can also check the before and after json file and see that all inrinsics and extrincs change. Can do a dif of these and look at camera position and distortion params.

To summarize, the trend goes the right way, but not enough for you, and I don't know why. At the end of day, 1500 m vertical error is 1.5 * GSD, so not totally terrible, but I thought it should be under 1000 m.

[Pietro-Matt]

No, --heghts-from-dem-uncertainty is for bundle_adjust

Ah yes I was confusing it with "--initial-dem-constraint-weight", sorry for the mess!

I am running the refined DEM script now with those two little tweaks on bundle_adjust, I'll let you know how it goes
Thanks a lot for the detailed help really :)