Skip to content

Conversation

@jtratner
Copy link
Contributor

@jtratner jtratner commented Apr 19, 2024

Follow-on to #143 . Makes swift and dxpy actually optional so people can opt into complex dependencies rather than being forced to use 'em, yay!

Tested this out in a codespace:

@jtratner ➜ /workspaces/stor (move-swift-and-dxpy-into-extras) $ poetry build
@jtratner ➜ /workspaces/stor (move-swift-and-dxpy-into-extras) $ for extratype in dxpy dxpy,swift swift; do rm -rf example-env/ && virtualenv example-env && source example-env/bin/activate && pip install 'dist/stor-4.2.0-py3-none-any.whl['$extratype']' && pip list > $extratype.txt && deactivate; done

New traceback:

Traceback (most recent call last):
  File "/workspaces/stor/stor/dx.py", line 21, in <module>
    import dxpy
ModuleNotFoundError: No module named 'dxpy'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/workspaces/stor/stor/base.py", line 47, in __new__
    cls = utils.find_dx_class(path)
  File "/workspaces/stor/stor/utils.py", line 273, in find_dx_class
    from stor.dx import DXPath, DXCanonicalPath, DXVirtualPath
  File "/workspaces/stor/stor/dx.py", line 25, in <module>
    raise utils.missing_storage_library_exception("dx", e) from e
ImportError: ModuleNotFoundError: No module named 'dxpy'
To use a 'dx' path, stor needs an additional python library. Please specify it as an extra in the installation.
i.e.,: `pip install stor[dx]` or `stor[dx] >= 5` in requirements.txt or `poetry add stor[dx]`.
Alternatively, change an existing pyproject.toml file to specify `stor = {version="5.0", extras = dx}`

And then showed results

`cat dxpy.txt`
@jtratner ➜ /workspaces/stor (move-swift-and-dxpy-into-extras) $ cat dxpy.txt 
Package            Version
------------------ -----------
argcomplete        3.3.0
boto3              1.34.90
botocore           1.34.90
cached-property    1.5.2
certifi            2024.2.2
charset-normalizer 3.3.2
dxpy               0.373.0
idna               3.7
jmespath           1.0.1
pip                24.0
psutil             5.9.8
python-dateutil    2.9.0.post0
requests           2.31.0
s3transfer         0.10.1
setuptools         69.5.1
six                1.16.0
stor               4.2.0
urllib3            2.1.0
websocket-client   0.54.0
wheel              0.43.0
`cat swift.txt`
@jtratner ➜ /workspaces/stor (move-swift-and-dxpy-into-extras) $ cat swift.txt 
Package               Version
--------------------- -----------
boto3                 1.34.90
botocore              1.34.90
cached-property       1.5.2
certifi               2024.2.2
charset-normalizer    3.3.2
debtcollector         3.0.0
idna                  3.7
iso8601               2.1.0
jmespath              1.0.1
keystoneauth1         5.6.0
msgpack               1.0.8
netaddr               1.2.1
netifaces             0.11.0
os-service-types      1.7.0
oslo.config           9.4.0
oslo.i18n             6.3.0
oslo.serialization    5.4.0
oslo.utils            7.1.0
packaging             24.0
pbr                   6.0.0
pip                   24.0
pyparsing             3.1.2
python-dateutil       2.9.0.post0
python-keystoneclient 5.4.0
python-swiftclient    4.5.0
PyYAML                6.0.1
requests              2.31.0
rfc3986               2.0.0
s3transfer            0.10.1
setuptools            69.5.1
six                   1.16.0
stevedore             5.2.0
stor                  4.2.0
tzdata                2024.1
urllib3               2.2.1
wheel                 0.43.0
wrapt                 1.16.0
`cat dxpy,swift.txt`
@jtratner ➜ /workspaces/stor (move-swift-and-dxpy-into-extras) $ cat dxpy,swift.txt 
Package               Version
--------------------- -----------
argcomplete           3.3.0
boto3                 1.34.90
botocore              1.34.90
cached-property       1.5.2
certifi               2024.2.2
charset-normalizer    3.3.2
debtcollector         3.0.0
dxpy                  0.373.0
idna                  3.7
iso8601               2.1.0
jmespath              1.0.1
keystoneauth1         5.6.0
msgpack               1.0.8
netaddr               1.2.1
netifaces             0.11.0
os-service-types      1.7.0
oslo.config           9.4.0
oslo.i18n             6.3.0
oslo.serialization    5.4.0
oslo.utils            7.1.0
packaging             24.0
pbr                   6.0.0
pip                   24.0
psutil                5.9.8
pyparsing             3.1.2
python-dateutil       2.9.0.post0
python-keystoneclient 5.4.0
python-swiftclient    4.5.0
PyYAML                6.0.1
requests              2.31.0
rfc3986               2.0.0
s3transfer            0.10.1
setuptools            69.5.1
six                   1.16.0
stevedore             5.2.0
stor                  4.2.0
tzdata                2024.1
urllib3               2.1.0
websocket-client      0.54.0
wheel                 0.43.0
wrapt                 1.16.0

dxpy pulls in a wide array of (often legacy) packages, skipping the import means
we only need to pull it in as necessary which hopefully makes library
more useful to others.
make dxpy into an extra as well
include with and without dxpy
Rather than leave the user hanging, show something helpful to the user
so they can resolve the situation by installing an additional package.

Left this as a separate function so it's easy to test the exception
generation fucntion doesn't break without needing to do painful testing
contortions. (which should be caught later on by test matrix)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants