Update setup.py packaging before dropping Python 3.4 support

[nitzmahone]

setup.py needs to be updated to a newer technology than distutils before we drop support for another Python version. setuptools should be the "easiest" to leave most of the existing custom code in place to support the extension build and integrated testing, etc, but there are other options that I'm sure @webknjaz would love to talk about ;).

We properly updated setup.py's metadata when dropping Python 2.6 and 3.3 support, however, our setup.py is still based on distutils, which doesn't understand python_requires (and prevents pypi from extracting the necessary metadata to ensure that unsupported Pythons will fall back to the last release that did support them). So today, anyone that's doing pip install pyyaml with an unsupported Python will install the latest release, but it will bomb out at install-time with something like PyYAML requires Python '>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*' but the running Python is 2.6.9 instead of installing a prior release that did support it.

Since we've released one that way, the fallback to 3.13 will never work again unless all the intermediate releases were pulled (uh, no ;) ). So either we say "tough noogies" or we release a single 5.x version with Python 2.6 and 3.3 support listed (on a packaging framework that supports the right metadata) to fix the glitch.

[nitzmahone]

also related: #210 (comment)

[webknjaz]

Setuptools is fine here, among alternatives I'd say Poetry looks nice, but when I checked it out it was lacking one feature which I personally wanted (grabbing a version from git tag). Yet, it gets better.

I think if pyyaml was to cover the most popular env cases with wheels it wouldn't matter a lot how setup.py is constructed because it would almost never be invoked given that users use modern enough Pip, which they should. Python 2.7 is shipped with up-to-date packaging tools so it should be fine to require them: https://github.com/python/cpython/tree/v2.7.16/Lib/ensurepip/_bundled.

The solution should start with pyproject.toml which would be a proper entry point for building things and setuptools is 100% compatible with it. It also provides us with a way to have easily replaceable build backends without ever affecting end-users.

Also, looking at the current setup.py, I'd say that we should move static stuff to setup.cfg and rip off test command which is widely discouraged in favor of the proper test runners.

[webknjaz]

Since we've released one that way, the fallback to 3.13 will never work

.post0 release can always fix this...

[4383]

@webknjaz some PR are currently under development to allow poetry to retrieve project version from git tag, I also propose my own version of this feature, but I think this kind of feature is not for now.

Else another alternative can be to use pbr, pbr is really straightforward to use and already have a lot of cool features (version, changelog, authors, requirements, etc...)

[webknjaz]

pbr is horrible. Please don't suggest it. I consider it a bug itself. Poetry is still dumb and PyPA members whom I trust don't like how it's done, so I'd not rush into using it. I only recommend what I've battle-tested myself.

[mcepl]

Also, could you please eliminate setup.py test and run tests in some more standardized way (python3 -munittest discover or pytest), please?

[nitzmahone]

Funny you mention it, I'd created a prototype pytest wrapper awhile back to surface pyyaml's "special" test suite as individual pytest tests- dusted it off and PR'd it at #588.

I think all the points of the original issue have been addressed in the past couple releases, so I'm closing...