Python load configuration from S3

You can use the configparser to load a configuration file on S3.

Assuming that you have the following configuration file on S3 as s3://bucket-name/config.ini:

[Default]
s3Location = s3://bucket-name/prefix/
sqlStatement = SELECT * FROM t ORDER BY ts DESC LIMIT 10

Below is the sample code to load the configuration:

import boto3
import configparser

"""
Getting the content from configuration file on S3
"""
s3Bucket = 'bucket-name'
s3Object = 'config.ini'
s3 = boto3.resource('s3')
obj = s3.Object(s3Bucket, s3Object)
content = ''
for line in obj.get()['Body']._raw_stream:
  content = content + line.decode("utf-8")
"""
Parsing configuration file
"""
config = configparser.ConfigParser()
config.read_string(content)
print(config['Default']['s3Location'])
print(config['Default']['sqlStatement'])