Finer-grained errors, ala PEP 657?

[jdtsmith]

PEP 657 added "fine-grained errors" which are similar to executing's AST-level text_range. But the default python traceback printing goes further, employing two types of error markers, ^ and ~. The latter is a "secondary" marker used for further detailed refinement for exceptions involving binary and indexing operators. Example:

File fge.py

x={}; x['a']={}; x['a']['b'] = None
x['a']['b']['c']['d'] = 1

Produces:

% python3.12 fge.py
Traceback (most recent call last):
  File "/Users/jdsmith/code/python/scraps/fge.py", line 2, in <module>
    x['a']['b']['c']['d'] = 1
    ~~~~~~~~~~~^^^^^
TypeError: 'NoneType' object is not subscriptable

while executing (here in iPython), shows:

One way to support this secondary refinement in a backwards-compatible way would be with an additional method text_range_secondary that computes the sub-region within text_range (if any) that could be highlighted distinctly.

[alexmojaki]

Do you want:

1. An improvement to IPython tracebacks?
2. To be able to compute these ranges for your own project?
3. To open a PR to executing/stack_data/IPython?

[alexmojaki]

@15r10nk I'm curious, do you have any idea how Python computes both ranges? I don't see extra positions in the instructions and it's not trivial for it to know which other instruction to look at.

[jdtsmith]

I guess 1+2. I have an iPython front end mode in Emacs I'm working on, so I'd be able to use it even before iPython adopts it for their ultratb output (or even if it never does). But I suspect IPython would adopt it, and I could possibly facilitate that.

In terms of computing the finer range, CPython does something pretty simple. I'm not sure how it gets the larger range to begin with, and how that compares to what you do with executing, but I haven't seen instances where they disagree (not that I've looked hard).

[alexmojaki]

In terms of computing the finer range, CPython does something pretty simple.

Ah, I didn't expect that they would parse the AST, I thought they only used the positions in instructions.

I'm not sure how it gets the larger range to begin with,

Positions are stored in bytecode instructions

and how that compares to what you do with executing,

Since Python 3.11, executing uses those stored positions.

For (2), you should be able to pretty much copy paste the CPython code since executing already gives you a Subscript/Binop node to work with.

[jdtsmith]

It's gotten more complicated in 3.13. I suppose it would be possible (if inefficient) to ask for the source_text, and pass it to _extract_caret_anchors_from_line_segment.