Home

Getting Started with Blockstore-client

NOTE: This article is a stub. It will be expanded upon soon.

The blockstore-client package contains a Python client library and command-line tool for interacting with Blockstore. The command-line tool is a thin wrapper around the library.

Installing

The blockstore-client package is built and installed with the standard distutils framework. The commands to do so are:

$ python ./setup.py build 
$ sudo python ./setup.py install

Using blockstore-cli

By default, the blockstore-cli tool will be installed to /usr/local/bin. Please see the Getting Started with Blockstore document on how to use it to interact with Blockstore.

Client API

The client library is the "rich" API endpoint for Blockstore. It provides the unified interface to Blockstore's RPC server, Blockstore's DHT, and the 3rd party name profile storage providers of the user's choice.

A Simple Name Resolver

from blockstore import config, client, drivers
import sys
import json

# ---------------------------
# Begin boilerplate
# ---------------------------

# load configuration 
conf = client.get_config()
if conf is None:
   print "Failed to load config"
   sys.exit(1)

# find storage drivers
storage_drivers = conf['storage_drivers']

if storage_drivers is None:
   print "No storage driver(s) defined in the config file.  Please set 'storage=' to a comma-separated list of %s" % ", ".join(drivers.DRIVERS))
   sys.exit(1)

# load and register each storage driver
for storage_driver in storage_drivers.split(","):
    storage_impl = client.load_storage( storage_driver )
    if storage_impl is None:
       print "Failed to load storage driver '%s'" % (storage_driver)
       sys.exit(1)

    rc = client.register_storage( storage_impl )
    if not rc:
       print "Failed to initialize storage driver '%s'" % (storage_driver))
       sys.exit(1)

# connect to blockstored
proxy = client.session(conf['blockstored_server'], conf['blockstored_port'])

# ---------------------------
# End boilerplate
# ---------------------------

# look up a name and print it out
name_rec = proxy.get_name_record( sys.argv[1] )
print json.dumps( name_rec, indent=4 )
sys.exit(0)

Client API

Terminology

• immutable data: This is content-addressed data whose hashes are written to the underlying blockchain in an OP_RETURN transaction, signed off by the writer's private key. The rationale behind providing "immutable" data is to provide a way of a writer to upload data that will not change, but whose integrity and owner can be confirmed by the blockchain. Immutable data can be hosted anywhere by anyone; a reader can verify that a given piece of data is authentic and owned by someone (given that someone's public key) by checking for its hash and signature in the OP_RETURN. The reader does not have to trust the source of the data; only the blockchain.

• mutable data: This is named data whose existence, ownership, and location are written to the underlying blockchain in an OP_RETURN transaction, signed off by the writer's public key. The rationale behind providing "mutable" data is to provide a way for a writer to upload data that will change frequently, but still guarantee to the reader that the data the reader gets is authentic and fresh. Mutable data must be hosted on one of a pre-defined set of servers, but the data's ownership, location, and any extra unchanging metadata are confirmed by the blockchain. The API for mutable data envelopes a read and write protocol that additionally ensures that readers will only receive authentic data, and that readers cannot be fed old versions of data once they have been seen for the first time.

• ancillary storage: General-purpose services like our DHT, or like IPFS, Syndicate, Tahoe-LAFS, Amazon S3, or even a simple FTP server. They merely hold copies of the data we send them; we construct the data such that they cannot serve data that a writer did not produce, and cannot serve an older version of data to a reader once the reader has seen a newer version.

• default ancillary storage provider: The blockstore client library can be configured to contact a default storage provider, specific to the application. Each method lets the caller choose how to contact ancillary storage of its choice, but in the context of a single application that hosts replicas of user data, this is redundant (hence the need to add a notion of a default provider).

• route: A JSON document encoding, among other things, the name and locations of mutable data (see below).

Route schema

{
   "id": string,
   "urls": [string],
   "pubkey": string
}

• id is an unchanging, unique identifier for the data.
• url is an array of strings that determine the location(s) of the current copies of the data. The data returned by url must encode enough information to construct a mutable data schema JSON document (see below). The easiest way is to simply send the JSON itself, but it's permitted to send something that the client can decode into one as well.
• pubkey is the optional public key of the writer, to be used in place of the public key in the blockchain.

Mutable data schema

{
   "id": string
   "data": string
   "ver": int
   "sig": string
}

• id is an unchanging identifier for this data.
• data is the string of data to fetch (base64-encoded).
• ver is an integer that represents the versioning information of the data. By default, it is a monotonically increasing version number that increments on each write.
• sig is the writer's signature over both data and ver.

API

• get_config( config_file=None ):

• load_storage( module_name ):

• register_storage( module ):

• session( blockstored_server, blockstored_port ):

• get_immutable( name, key ): Takes the name and the hash of the data (key),and returns the data. The method checks the blockchain to see that the data's hash (key) was signed by public key of the name owner. This method fetches the data from the ancillary storage system(s). Returns {"data": data_string}.

• get_mutable( name, id, ver_min=None, ver_max=None, ver_check=None ): Takes the name and unchanging id of the data, and returns the data. The method checks to see that the data was signed by the pubkey in the route (if given), or the name owner's public key. This method fetches the data from the ancillary storage system(s) by first fetching the route JSON record described above, and then fetching the data from the URLs given in the route JSON. It verifies the signature over the version and data, confirms that this version has not been seen before, and caches the version number locally. The caller only gets data back if (1) the signature matches, and (2) the data is not stale. The caller can optionally set lower and upper bounds on the version (ver_min, ver_max), and can optionally define a callback to decide whether or not the version is fresh (ver_check). The method returns a mutable data record (described above), as well as the route used to find it (set in the route field).

• put_immutable( name, data, privatekey, txid=None ): Takes the name of the user and the data, generates the data's hash, updates the name's profile to include the data's hash (i.e. puts the new profile's hash into a NAME_UPDATE operation on the blockchain), and replicates the data to ancillary storage. Note that immutable data is content-addressed--there is no notion of write-conflicts. In the event that replicating the data fails but the NAME_UPDATE succeeds, this method can be called multiple times with the txid field set to the NAME_UPDATE transaction's ID. In this mode, the data will only be replicated to ancillary storage; no profile update will occur. This method returns a JSON object that includes the transaction ID, set as txid.

• put_mutable( name, id, data, privatekey, create=True, txid=None, ver=None, make_ver=None ): Takes the name of the user, the unchanging id for the data, the data itself. Puts a new mutable data record in the name's profile, puts the new profile's hash into a NAME_UPDATE transaction if it does not exist already, generates a mutable data JSON document (see above), and replicates the JSON document to ancillary storage. By default, the version number of the mutable data will be incremented before it is replicated along with the data. If create is False, then the profile will not be updated (but the profile must have a mutable data record already). The caller can specify either an arbitrary verison number (ver) or a callback to generate a new version number (make_ver) to implement its own consistency controls. If this operation successfully broadcasts a NAME_UPDATE transaction but fails to replicate data, the caller can try to replicate again by setting txid to the successful NAME_UPDATE transaction's ID (in which case, no new transaction will be sent). This method returns a JSON object that includes the transaction ID, set as txid.

• delete_immutable( name, key, privatekey ): Takes the name of the user and the data's hash (key), and removes it from the user's profile and marks it as deleted in the blockchain. Also, this method removes the data from ancillary storage. Returns True if successful.

• delete_mutable( name, id, privatekey ): Takes the name of the user, the unchanging id for the data, and the private key, and will remove the mutable data and the mutable data JSON from ancillary storage and uncache the version. Returns True if successful.

Examples

Storage Drivers

To add support for a new ancillary storage provider, a developer must create a Python class or module with the following methods implemented:

• storage_init():

• make_mutable_url(data_id):

• get_immutable_handler(key):

• get_mutable_handler(data_id):

• put_immutable_handler(key, data, txid):

• put_mutable_handler(data_id, version, signature, data_json):

• delete_immutable_handler(key, txid):

• delete_mutable_handler(data_id, signature):

Examples