Regression testing pyright with open-source libraries

[erictraut]

For a long time, the maintainers of mypy have used this nifty tool called mypy_primer to run regression tests for every PR submitted to the mypy repo. This tool runs mypy on a large number of open-source typed Python libraries to determine whether a change to the mypy source code breaks something. The mypy_primer tool is also used by the maintainers of typeshed to detect regressions in typeshed stub files.

The author of mypy_primer (@hauntsaninja) has graciously extended the tool to support pyright. By using mypy_primer within pyright's CI workflows, we will now have better visibility into the impact of individual changes made to pyright.

For now, I've configured mypy_primer to run on just a handful of open-source libraries.

If you are a library maintainer and use pyright for your project, let us know if you'd like us to add your library to the pyright regression tests.

[hynek]

I'm somewhat surprised this thread is empty – given that https://github.com/hynek/svcs is kinda at the frontier of Python typing, it would be great if y'all could add it.

[erictraut]

@hauntsaninja, I agree this library would be a good addition to mypy_primer.

[hauntsaninja]

Added in hauntsaninja/mypy_primer#184 so all of pyright, mypy, ty, pyrefly, typeshed, basedpyright, basedmypy will observe regressions in their CI