pipoe cannot gather the requests package.

[chris-hibberd]

This is a great tool but unfortunately when trying to generate a set of recipes that includes requests as a dependency, I encounter an error. This can be reproduced by running for the requests package on its own:

build@hostname:~$ pipoe --python python3 -p requests
Gathering info:
  requests
   [ERROR] Failed to gather requests (
win-inet-pton; sys_platform == "win32" and python_version == "2.7" and extra == 'socks'
                                                                    ^
Parse error at line 1, column 68: expected EOF. trail: [wsp]
)
Generating recipes:

PREFERRED_VERSIONS are available in: ./python3-versions.inc

It looks like the parser imported from pep508_parser raises an exception when it encounters the requires_dist value 'win-inet-pton; sys_platform == "win32" and python_version == "2.7" and extra == \'socks\''.

Exception handling could be added around

d = parse_requires_dist(dep)

to ensure that the script completes and adds the other dependencies, I'm not familiar with pep508 to know whether the parser should be fixed as well.

I've had this issue as well.
For the latest release of requests (2.25.0) I can't reproduce this behavior anymore.

Besides requests I haven't seen this error with any other package.

[pwicks86]

This is an issue with a number of other packages, for example uvicorn. I think the underlying issue is that pep508_parser just implements the grammar specified in pep 508 directly, but pip actually accepts a wider range of inputs for requirements, especially things like: some-package; foo and bar and baz. It's not really clear to me that packages that do this are actually violating the pep 508, but at the end of the day it doesn't matter since pip accepts it. The fix is probably to change pipoe to use packaging directly, rather than to use pep508_parser.

[JonathanPlasse]

To fix the problem, you need to use parenthesis to follow the pep 508.
some-package; foo and (bar and baz)

[pwicks86]

To fix the problem, you need to use parenthesis to follow the pep 508.
some-package; foo and (bar and baz)

Sure, but you'd have to fix every package that's in violation of pep 508. Given that pip accepts a wider range of inputs that the standard allows, it seems like it makes more sense to use packaging directly as pip does, rather than try to fix the individual packages.