ListPlot in non-vector mode is showing log scale

[rocky]

Description

Recent changes seem to have broken non-vector-mode tick marks in ListPlot[]

How to Reproduce

mathicsscript --matplotlib -c 'ListPlot[{1, 1, 2}]'

Output Given

Expected behavior

No logplot scale, y axis goes from 0 to 2.

Your Environment

Current Mathics core development environment 9.0.1.dev0 and same for mathicsscript
mathics --version

Workarounds

Use last release ;-)

[bdlucas1]

See #1555 for analysis and fix.

We have a gap in testing, in that no (or maybe just a few?) tests check the generated SVG graphics. The tests in test_plot_detail.py can easily be enhanced to check that the SVG graphics for all the doctests aren't changed. If you'd like we can leave this issue open, assigned to me, to do that.

[rocky]

Thanks for fixing.

I will double-check the fix when I get a chance, sometime soon I hope.

[bdlucas1]

I also submitted PR #1556, which is #1555 plus SVG tests. Please double-check one or the other, depending on whether you're ok with adding the SVG tests. I think they will help prevent mishaps like this in the future.

[bdlucas1]

As noted on PR #1556, it's probably too fragile, so suggest just going with PR #1555 at this point.

[rocky]

As noted on PR #1556, it's probably too fragile, so suggest just going with PR #1555 at this point.

Ok. As often happens here, there are good ideas, e. g., more robust testing or using compilation (or a more aggressive transformation) to speed up computation. But these ideas need iteration over time to get right.

I haven't been following the testing process in detail, other than we're testing at the S-expression level (as opposed to a rendered SVG level), which is a good thing.

The obvious things that would seem to be needed are:

• some way to indicate tolerance on numeric values, and possibly
• checking counts of data items when things can vary widely (e. g., all we careabout are there are n points, or n points that range between, x and y.)
• and as I alluded to in another PR, a GitHub workflows CI regime that works independently of other tests so that certain environment aspects can be isolated (version of numpy used, special OS environments, and Python versions). Isolating CI testing also allows the graphics tests, which will probably grow a bit, to be run in parallel with the other tests rather than in sequence with them.

[bdlucas1]

Some thoughts and experiences regarding testing plotting functions:

There are three places in the graphics pipeline where we could do testing:

• The Graphics* expression that is output from Plot*. This is what the
  tests in this directory currently do, in more or less detail. The
  more detailed tests that were recently added are based on doing a
  simple diff of a pretty-printed indented outline-form text
  representation of the Graphics* output compared to a reference
  file.

  This was initially found to be fragile because of inconsequential
  numerical differences on different platforms, but that was brought
  under control by limiting the precision and type information of the
  text representation of the Graphics* expression.

  I have found this to be effective at finding problems and easy to
  understand - determine whether a change is acceptable or harmful -
  when a code change changes the output.

• Here the pipeline splits into multiple options:

  • SVG output via core. This tests both Plot*->Graphics* and
    Graphics*->SVG code. Simple textual comparison of actual vs
    expected result was tried in this PR and immediately found to be
    fragile due to the insane precision of the numbers emitted to
    the SVG.

    Possibly this could be brought under control by limiting the
    precision of the numbers (as was done for the above Graphics*
    based tests), which would require finding all the spots in SVG
    rendering where numbers are emitted. However after doing so
    interpreting differences might be difficult due to the complex
    structure of SVG.

    A pretty-printer that displays SVG in a deconstructed indented
    outline form (as we do for the above Graphics*) that could
    easily be diffed might facilitate this.

  • High-performance rendering output in m3d via
    Plotly+Panel. There's no direct equivalent of SVG in this
    pipeline. The closest would be the internal data structures used
    by Plotly+Panel, but I would expect that to be fragile, subject
    to change, and difficult to interpret.

• The final rendered image. This is of course what the user cares
  about. We don't have this in our core testing, but I have been doing
  this in m3d as I do development in core for both pipelines
  (Plot*->Graphics*->SVG->PNG and Plot*->Graphics*->Plotly->PNG) for
  all the doctests and more. I (somewhat to my surprise) have found
  these tests to be the most robust, because they are insensitive to
  inconsequential numerical or other differences in the Graphics* or
  the SVG, partly because images have very limited precision, both
  spatially and in color depth.

  As for interpreting the results, in m3d I have a semi-automated UI
  that, when it detects a difference, displays the actual image, the
  reference image, and a diff image side-by-side, with a button
  offering to accept the actual output and save it as the new
  reference output. I have found this to be pretty useful for
  development testing.

  I don't know how robust the generated images are from system to
  system. My hunch is they might be portable due to the limited
  precision of images. I think it might be worth the experiment to set
  up a separate CI workflow where we can install the required software
  to generate images (initially, and maybe only, via the core SVG
  path), and try it out.

I think these tests are all useful and are complementary. For example,
if the first Plot*->Graphics* tests show that some code change has
changed the Graphics* but you think it is inconsequential, or is
expected, and image-based comparison can serve as a check on whether
that is so. Conversely, lacking effective testing on the SVG output,
an image comparison can flag unintended changes in the SVG generation
code.

Regarding the idea to do more sophisticated comparisons than simple
textual diffs, I tried something in that direction in test_plot.py
(see eval_and_check_structure). I found that to be difficult to set
up, and difficult to interpret the results. Probably that's because I
wasn't doing it right, but I couldn't figure out a better way to do
it.

On the other hand I found the doctest-based tests that were recently
added to be easy to set up, easy to interpret, and they give broad coverage.