feat: 3.12 support for the PositionNodeFinder

[15r10nk]

fix #70

this merge-request adds support for the current 3.12.0a7

Unfortunately I had to skip test_module_files for the SentinelNodeFinder, because there where random failures.

@alexmojaki we should discuss how we want to proceed with this issue. I will try to fixe some issues with the node finder to get a better understanding. I hope it will not get to complicated.

[15r10nk]

reported the following issue python/cpython#109195.

source positions of LOAD_GLOBAL super includes the parentheses of super()

[alexmojaki]

Unfortunately I had to skip test_module_files for the SentinelNodeFinder, because there where random failures.

I tried taking a look in #73. At least in 3.9, test_module_files seems to have passed, but test_sample_files failed. I don't know if that's related to what you saw, I haven't investigated.

I also decided to drop support for Python < 3.8. The same is happening in gristlabs/asttokens#117 and we've suffered enough. Hopefully that helps.

[15r10nk]

hmm I looked also into these issues and fixed some of theme. However it seems that I forgot to push this branch.
I pushed it now #74.

I don't know if that's related to what you saw, I haven't investigated.

The thing is that this was some time ago and I can not remember if I was able to solve all issues. But if you want do deprecate python < 3.8 ❤️ #64 might be also interesting for you. This is a complete new NodeFinder implementation for >= 3.8 which works similar to the PositionNodeFinder. But I would like to review my own code first before you go into a deeper review. The algo is quite complicated and some documentation could improve the readability.

I work currently on pysource-codegen to generate really crazy python code. I used it so far to find several bugs in python 3.12 itself (mostly for the new type parameter syntax).

My (and propably your to) limiting factor is time 😄
I can continue to work on some executing issues when I'm done reporting the cpython 3.12 issues.

It would be really useful if you could tell me what you would prefer to be done first.

[alexmojaki]

The main priority is to release a version of executing that works with 3.12 before the final release, which is planned for 2nd October (https://peps.python.org/pep-0693/). If some odd new test failures in older versions have appeared, it's fine to do the minimum work required (e.g. skipping certain files) to make the build pass, as long as nothing becomes broken in a new way.

Thank you so much for all your help ❤️

[alexmojaki]

I have time to help at the moment, let me know if there's something that makes sense for me to look at without duplicating work.

[15r10nk]

Good news for you, I think the hardest part is over.
I was able to check most/all python files on my PC. I will go one more time through the commits and check if some things have changed since I tested the first alpha.

I'm currently on a short holiday trip with my family and will continue to work on this merge request next week. My best guess is that you will be able to review the code at the end of next week.

Thank you for your offer to help. Have a nice weekend. 😉

[15r10nk]

hi @alexmojaki, maybe you have a tip for me for the following problem.

https://github.com/alexmojaki/executing/actions/runs/6215351046/job/16868163417

It looks for me like the ast module misses some type information for 3.12. Do you have any tip how I could solve this issue?

[alexmojaki]

Just disable mypy for 3.12 for now. Looks like they're not ready either: python/mypy#15277

[15r10nk]

next problem @alexmojaki.

https://github.com/alexmojaki/executing/actions/runs/6215643716/job/16868740630

the tests for 3.9 fail because the sample_results have changed. The problem is that I get the same error for the master branch (also for different cpython minor versions).

I will try to solve this problem, but it would be easier when I would know why it happen.

[alexmojaki]

asttokens recently started supporting f-strings better, which resulted in 45d8ca2

But then that changed for Python 3.9: gristlabs/asttokens#104

So nothing is wrong in executing.

[alexmojaki]

You can update the samples with FIX_EXECUTING_TESTS=1 EXECUTING_SLOW_TESTS=1 tox -e py39, and those lines will go back to empty strings. Or you could add these lines to test_main.py:

if sys.version_info[:2] == (3, 9):
    # https://github.com/alexmojaki/executing/pull/71#issuecomment-1723634310
    import asttokens.util
    asttokens.util.fstring_positions_work = lambda: True

[15r10nk]

thank you, you saved me a lot of time.

[15r10nk]

hi @alexmojaki, I think it is time for a review.

[alexmojaki]

LGTM! Think it's ready?

[15r10nk]

I have added support for __iter__ and __next__. It is ready from my side if the next ci-run passes without errors.

[alexmojaki]

Released v2.0.0!