Skip to content

Use of Swift storage #19

@caida-ricky

Description

@caida-ricky

I tried to use netacq-edge provider from CAIDA's network, and have swift credential setup. However, I got the following error when I tried to run the pyipmeta_test.py code in the test folder.

Testing netacq-edge with latest db...
Traceback (most recent call last):
  File "./pyipmeta_test.py", line 44, in <module>
    ipm = pyipmeta.IpMeta(providers=["netacq-edge"])
  File "build/bdist.linux-x86_64/egg/pyipmeta/pyipmeta.py", line 72, in __init__
  File "build/bdist.linux-x86_64/egg/pyipmeta/pyipmeta.py", line 130, in _reload
  File "build/bdist.linux-x86_64/egg/pyipmeta/dbidx.py", line 116, in __init__
  File "build/bdist.linux-x86_64/egg/pyipmeta/dbidx.py", line 137, in _load_index
swiftclient.exceptions.ClientException:
Auth versions 2.0 and 3 require python-keystoneclient, install it or use Auth
version 1.0 which requires ST_AUTH, ST_USER, and ST_KEY environment
variables to be set or overridden with -A, -U, or -K.

I had the keystoneclient installed. I can correctly list the objects with this code.

import os
from swiftclient.service import SwiftService, SwiftError
swift_opts = {
            # Apparently SwiftService by default checks only ST_AUTH_VERSION.
            # We emulate the swift CLI, and check three different variables.
            "auth_version": os.environ.get('ST_AUTH_VERSION',
                os.environ.get('OS_AUTH_VERSION',
                os.environ.get('OS_IDENTITY_API_VERSION', '1.0'))),
            }
with SwiftService(options=swift_opts) as swift:
    list_parts_gen = swift.list(container="datasets-external-netacq-edge-processed")
    for page in list_parts_gen:
        print(page["listing"])

But page["success"] was empty and and triggered line 136-137 in dbidx.py to throw an (unnecessary) error.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions