config-source

This package provides extensible configuration loading from various sources.

Features:

• Dict-like configuration loading from:
  • python dictionaries
  • python objects
  • python files
  • environment variables
  • JSON files
• Custom configuration sources and objects.

Basically config-source provides a way to register configuration loaders and call them by names. The loader accepts optional arguments, reads configuration from a specific source and populates a configuration object.

Example:

from config_source import DictConfig
from config_source import DictConfigLoader

config = DictConfig()
config.load_from('pyfile', '/path/to/config.py')
config.load_from('json', '/path/to/config.json')
config.load_from('env', prefix='MYCFG_')

loader = DictConfigLoader(config)
loader.load('/path/to/config.py')
loader.load('/path/to/config.json')
loader.load(SomeClassWithConfigs)

Usage

Out of the box you could use:

• Low level load_to() function.
• DictConfig class.
• DictConfigLoader class to assist in configurations loading.

load_to() calls a loader registered for a specific source and populates a config object passed to it:

load_to(config, 'source_name', ...)

• config - configuration object to populate.
• source_name - configuration source name.

DictConfig behaves like a regular python dictionary and provides load_from method to load configurations from various sources (it uses load_to() internally):

config = DictConfig()
config.load_from(<source_name>, *args, **kwargs)

• <source_name> - configuration source name;
• *args and **kwargs - arguments for configuration loader.

The following sources are provided out of the box for dict-like configurations.

Note: dict-like means any object with mapping interface can be used as configuration object:

config = {}
load_to(config, 'env', ...)

dictconfig = DictConfig()
dictconfig.load_from('env', ...)
load_to(dictconfig, 'env', ...)

• object - load configuration from a python object. It reads attributes with uppercase names:

  config.load_from('object', <object>)
  

  Example:

  class MyConfig:
      SECRET_KEY = 123
      DEBUG = False
  
  ...
  
  config.load_from('object', MyConfig)
  

• dict - load configuration from a python dictionary. Reads only uppercase keys:

  config.load_from('dict', <dict>)
  

  Example:

  myconfig = dict(SECRET_KEY=123, DEBUG=False)
  config.load_from('dict', myconfig)
  

• env - load configuration from current runtime environment:

  config.load_from('env', prefix=<name_prefix>, trim_prefix=True)
  

  • prefix - Environment variable name prefix.
  • trim_prefix - Include or not prefix to result config name

  Example:

  # Load vars with names MYCFG_*, like MYCFG_SECRET.
  config.load_from('env', prefix='MYCFG_')
  

• pyfile - load configuration from a python file. Reads only uppercase attributes:

  config.load_from('env', filename, silent=False)
  

  • filename - filename to load.
  • silent - Don't raise an error on missing files.

  Example:

  config.load_from('pyfile', 'config.py')
  

• json - load configuration from a json file. Reads only uppercase keys:

  config.load_from('json', filename, silent=False)
  

  • filename - filename to load.
  • silent - Don't raise an error on missing files.

  Example:

  config.load_from('json', '/path/to/config.json')
  

DictConfigLoader auto-detects source name from input configuration source:

loader = DictConfigLoader(config)
loader.load('/path/to/file.py')

# Same as:
config.load_from('pyfile', '/path/to/file.py')

You may subclass to extend auto-detection.

Add source

config_source decorator is used to register additional configuration sources:

from config_source import config_source

@config_source('source_name')
def myloader(config, arg1, arg2):
    config['XX'] = arg1 + arg2

config.load_from('source_name', 1, arg2=2)

Configuration loader must be a callable with at least one argument - configuration object to populate. Other arguments are optional and loader specific.

There is a possibility to register configuration sources by implementing a package with entry point:

setup(
    ...
    entry_points={'config_source.sources': '<source> = <package name>'},
    ...
)

In the package you use config_source decorator.

For more info on entry points see

• https://packaging.python.org/guides/creating-and-discovering-plugins/
• http://setuptools.readthedocs.io/en/latest/pkg_resources.html#entry-points
• http://setuptools.readthedocs.io/en/latest/setuptools.html#dynamic-discovery-of-services-and-plugins

Note: you could specify single entry point even if your package adds multiple sources.

Defaults

Instead of always passing parameters to configuration loaders you could set defaults in DictConfig:

config = DictConfig(defaults={
    'env': {'prefix': 'MYAPP_'},
    'pyfile': {'filename': '/path/to/file.py'}
})

# 'prefix' will be set to MYAPP_ for 'env' config source.
# Load from 'MYAPP_*' vars by default.
config.load_from('env')

# Load from 'MY_*' vars
config.load_from('env', 'MY_')

# Load from '/path/to/file.py' by default.
config.load_from('pyfile')

# Load from '/path/to/another/file.py'.
config.load_from('pyfile', '/path/to/another/file.py')

defaults is a map where keys are source names and values are keyword parameters to be passed to loaders.

Custom configuration type

You can register configuration source for specific type (by default it's a dict):

@config_source('source_name', config_type='mytype')
def mytype_loader(config):
    ....

config_type here is a string defining configuration object type.

Now you populate your config object using that loader:

load_to(config, 'source_name', config_type='mytype')

where config is object implementing mytype interface.

list configuration example:

from config_source import config_source, load_to


@config_source('object', config_type='list')
def object_loader(config, obj):
has = False
for key in dir(obj):
    if key.isupper():
        has = True
        config.append(getattr(obj, key))
return has


class MyConfig:
    SECRET = 1
    DEBUG = False


cfg = []
load_to(cfg, 'object', config_type='list')

# cfg = [1, False]

# Fails because by default it calls loader for 'dict' configuration.
# load_to(cfg, 'object')