The dynamic configurator for your Python Project
Project description
██████╗ ██╗ ██╗███╗ ██╗ █████╗ ██████╗ ██████╗ ███╗ ██╗███████╗
██╔══██╗╚██╗ ██╔╝████╗ ██║██╔══██╗██╔════╝██╔═══██╗████╗ ██║██╔════╝
██║ ██║ ╚████╔╝ ██╔██╗ ██║███████║██║ ██║ ██║██╔██╗ ██║█████╗
██║ ██║ ╚██╔╝ ██║╚██╗██║██╔══██║██║ ██║ ██║██║╚██╗██║██╔══╝
██████╔╝ ██║ ██║ ╚████║██║ ██║╚██████╗╚██████╔╝██║ ╚████║██║
╚═════╝ ╚═╝ ╚═╝ ╚═══╝╚═╝ ╚═╝ ╚═════╝ ╚═════╝ ╚═╝ ╚═══╝╚═╝
dynaconf - The dynamic configurator for your Python Project
dynaconf a layered configuration system for Python applications - with strong support for 12-factor applications and extensions for Flask and Django.
Documentation: http://dynaconf.readthedocs.io/
Features
- Strict separation of settings from code (following 12-factor applications Guide).
- Define comprehensive default values.
- Store parameters in multiple file formats (toml recommended and json, yaml, ini and py also supported).
- Sensitive secrets like tokens and passwords can be stored in safe places like
.secrets
orvault server
. - Parameters can optionally be stored in external services like Redis server.
- Simple feature flag system.
- Layered [environment] system.
- Environment variables can be used to override parameters.
- Support for
.env
files to automate the export of environment variables. - Correct data types (even for environment variables).
- Have only one canonical settings module to rule all your instances.
- Drop in extension for Flask
app.config
object. - Drop in extension for Django
conf.settings
object. - Powerful $ dynaconf CLI to help you manage your settings.
- Customizable Validation System to ensure correct config parameters.
- Allow the change of dynamic parameters on the fly without the need to redeploy your application.
install Dynaconf
Python 3.x is required
# Default installation supports .py, .toml, .json file formats
# and also reading from environment variables (.env supported)
pip3 install dynaconf
Getting Started
Accessing config variables in your Python application
In your Python program wherever you need to access a settings variable do:
# Example of program to connect to some database
from someorm import connect
from dynaconf import settings # import `dynaconf.settings` canonical settings object
con = connect(
username=settings.USERNAME, # attribute style access
password=settings.get('PASSWORD'), # dict get style access
port=settings['PORT'], # dict item style access
timeout=settings.as_int('TIMEOUT'), # Forcing casting if needed
host=settings.get('HOST', 'localhost') # Providing defaults if key not defined
)
Where the settings values are stored?
Dynaconf aims to have a flexible and usable configuration system. Your applications can be configured via a configuration files, through environment variables, or both. Configurations are separated into environments: [development], [staging], [testing] and [production]. The working environment is selected via an environment variable.
Sensitive data like tokens, secret keys and password can be stored in .secrets.*
files and/or external storages like Redis
or vault
secrets server.
Besides the built-in optional support to redis as settings storage dynaconf allows you to create custom loaders and store the data wherever you want e.g: databases, memory storages, other file formats, nosql databases etc.
environments
At any point in time, your application is operating in a given configuration environment. By default there are four such environments:
- [development]
- [staging]
- [testing]
- [production]
You can also define [custom environment] and use the pseudo-envs [default] to provide comprehensive default values and [global] to provide global values to overrride in any other environment.
Without any action, your applications by default run in the [development] environment. The environment can be changed via the ÈNV_FOR_DYNACONF
environment variable. For example, to launch an application in the [staging] environment, we can run:
export ENV_FOR_DYNACONF=staging
or
ENV_FOR_DYNACONF=staging python yourapp.py
The settings files
An optional settings.{toml|py|json|ini|yaml}
file can be used to specify the configuration parameters for each environment. If it is not present, only the values from environment variables are used (.env file is also supported). Dynaconf searches for the file starting at the current working directory. If it is not found there, Dynaconf checks the parent directory. Dynaconf continues checking parent directories until the root is reached.
The recommended file format is TOML but you can choose to use any of .{toml|py|json|ini|yaml}.
The file must be a series of sections, at most one for [default], optionally one for each [environment], and an optional [global] section. Each section contains key-value pairs corresponding to configuration parameters for that [environment]. If a configuration parameter is missing, the value from [default] is used. The following is a complete settings.toml
file, where every standard configuration parameter is specified within the [default] section:
[default]
username = "admin"
port = 5000
host = "localhost"
message = "default message"
value = "default value"
[development]
username = "devuser"
[staging]
host = "staging.server.com"
[testing]
host = "testing.server.com"
[production]
host = "server.com"
[awesomeenv]
value = "this value is set for custom [awesomeenv]"
[global]
message = "This value overrides message of default and other envs"
The [global] pseudo-environment can be used to set and/or override configuration parameters globally. A parameter defined in a [global] section sets, or overrides if already present, that parameter in every environment. For example, given the following settings.toml
file, the value of address will be "1.2.3.4" in every environment:
[global]
address = "1.2.3.4"
[development]
address = "localhost"
[production]
address = "0.0.0.0"
NOTE: The [env] name and first level variables are case insensitive as internally dynaconf will always use upper case, that means [development] and [DEVELOPMENT] are equivalent and address and ADDRESS are also equivalent. This rule does not apply for inner data structures as dictionaries and arrays.
Settings file supported formats
By default toml is the recommended format to store your configuration, however you can switch to a different supported format.
# If you wish to include support for more sources
pip3 install dynaconf[yaml|ini|redis|vault]
# for a complete installation
pip3 install dynaconf[all]
Once the support is installed no extra configuration is needed to load data from those files, dynaconf will search for settings files in the root directory of your application looking for the following files in the exact order below:
[
'settings.py', '.secrets.py',
'settings.toml', 'settings.tml', '.secrets.toml', '.secrets.tml',
'settings.yaml', 'settings.yml', '.secrets.yaml', '.secrets.yml',
'settings.ini', 'settings.conf', 'settings.properties',
'.secrets.ini', '.secrets.conf', '.secrets.properties',
'settings.json', '.secrets.json',
# redis server if REDIS_FOR_DYNACONF_ENABLED=true
# vault server if VAULT_FOR_DYNACONF_ENABLED=true
# other sources if custom loaders are defined
# All environment variables prefixed with DYNACONF_
]
NOTE: Dynaconf works in an override mode based on the above order, so if you have multiple file formats with conflicting keys defined, the precedence will be based on the loading order.
Take a look at the example folder to see some examples of use with different file formats.
Storing sensitive secrets
To safely store sensitive data Dynaconf also searches for a .secrets.{toml|py|json|ini|yaml}
file to look for data like tokens and passwords.
example .secrets.toml
:
[default]
password = "sek@987342$"
The secrets file supports all the environment definitions supported in the settings file.
IMPORTANT: The reason to use a
.secrets.*
file is the ability to omit this file when commiting to the repository so a recommended.gitignore
should include.secrets.*
line.
Scafolding
Dynaconf provides a CLI to easily configure your application configuration, once dynaconf is installed go to the root directory of your application and run:
$ dynaconf init -v key=value -v otherkey=othervalue -s token=1234 -e production
The above command will create in the current directory
settings.toml
[default]
KEY = "default"
OTHERKEY = "default"
[production]
KEY = "value"
OTHERKEY = "othervalue"
also .secrets.toml
[default]
TOKEN = "default"
[production]
TOKEN = "1234"
The command will also create a .env
setting the working environment to [production]
ENV_FOR_DYNACONF="PRODUCTION"
And will include the .secrets.toml
in the .gitignore
# Ignore dynaconf secret files
.secrets.*
NOTE: refer to the documentation to see more CLI commands to manage your configuration files such as dynaconf list and dynaconf write
For sensitive data in production is recommended using http://vaultproject.io as dynaconf supports reading and writing values to vault servers, see vault loader
documentation
Environment variables
All configuration parameters, including custom envs and *_FOR_DYNACONF settings, can be overridden through environment variables. To override the configuration parameter {param}, use an environment variable named DYNACONF_{PARAM}. For instance, to override the "host" configuration parameter, you can run your application with:
DYNACONF_HOST=other.com python yourapp.py
.env files
If you don't want to declare the variables on every program call you can export DYNACONF_*
variables or put the values in .env
files located in the same directory as your settings files.
Configuration FOR_DYNACONF
The DYNACONF_ prefix is set by GLOBAL_ENV_FOR_DYNACONF and serves only to be used in environment variables to override config values.
This prefix itself can be changed to something more significant for your application, however we recommend kepping DYNACONF_ as your global env prefix.
NOTE: See the
Configuring Dynaconf
section in documentation to learn more on how to use.env
variables to configure dynaconf behavior.
Environment variables precedence and casting
Environment variables take precedence over all other configuration sources: if the variable is set, it will be used as the value for the parameter. Variable values are parsed as if they were TOML syntax. As illustration, consider the following examples:
DYNACONF_INTEGER=1
DYNACONF_FLOAT=3.14
DYNACONF_STRING=Hello
# Use extra quotes to force a string from other type
DYNACONF_STRING="'42'"
DYNACONF_STRING="'true'"
DYNACONF_STRING="Hello"
# booleans
DYNACONF_BOOL=true
DYNACONF_BOOL=false
# Arrays must be homogenous in toml syntax
DYNACONF_ARRAY=[1, 2, 3]
DYNACONF_ARRAY=[1.1, 2.2, 3.3]
DYNACONF_ARRAY=['a', 'b', 'c']
DYNACONF_DICT={key="abc",val=123}
# toml syntax does not allow `None/null` values so use
DYNACONF_NONE='@none None'
NOTE: Older versions of Dynaconf used the casting prefixes for env vars like
export DYNACONF_INTEGER='@int 123'
still works but this casting is deprecated in favor of using TOML syntax described above. To disable the@type
castingexport AUTO_CAST_FOR_DYNACONF=false
Boxed values
In Dynaconf values are Boxed, it means the dot notation can also be used to access dictionary members, example:
settings.toml
[default]
mysql = {host="server.com", port=3600, auth={user="admin", passwd=1234}}
You can now access
from dynaconf import settings
connect(
host=settings.MYSQL.host,
port=settings.MYSQL.port,
username=settings.MYSQL.auth.user,
passwd=settings.MYSQL.auth.get('passwd'),
)
External storages
Using Hashicorp Vault to store secrets
The https://www.vaultproject.io/ is a key:value store for secrets and Dynaconf can load variables from a Vault secret.
- Run a vault server Run a Vault server installed or via docker:
docker run -e 'VAULT_DEV_ROOT_TOKEN_ID=myroot' -p 8200:8200 vault
- Install support for vault in dynaconf
pip install dynaconf[vault]
- In your
.env
file or in environment variables do:
VAULT_FOR_DYNACONF_ENABLED=true
VAULT_FOR_DYNACONF_URL="http://localhost:8200"
VAULT_FOR_DYNACONF_TOKEN="myroot"
Now you can have keys like PASSWORD
and TOKEN
defined in the vault and
dynaconf will read it.
To write a new secret you can use http://localhost:8200 web admin and write keys
under the /secret/dynaconf
secret database.
You can also use the Dynaconf writer via console
$ dynaconf write vault -s password=123456
Using REDIS
1 Add the configuration for redis client in your .env
or export envvars
REDIS_FOR_DYNACONF_ENABLED=true
REDIS_FOR_DYNACONF_HOST=localhost
REDIS_FOR_DYNACONF_PORT=6379
You can now write variables direct in to a redis hash named DYNACONF_< env >
You can also use the redis writer
$ dynaconf write redis -v name=Bruno -v database=localhost -v port=1234
The above data will be recorded in redis as a hash:
DYNACONF_DYNACONF {
NAME='Bruno'
DATABASE='localhost'
PORT='@int 1234'
}
if you want to skip type casting, write as string intead of PORT=1234 use PORT="'1234'".
Data is read from redis and another loaders only once when dynaconf.settings
is first accessed
or when .setenv()
or using_env()
are invoked.
You can access the fresh value using settings.get_fresh(key)
There is also the fresh context manager
from dynaconf import settings
print(settings.FOO) # this data was loaded once on import
with settings.fresh():
print(settings.FOO) # this data is being freshly reloaded from source
And you can also force some variables to be fresh setting in your setting file
FRESH_VARS_FOR_DYNACONF = ['MYSQL_HOST']
or using env vars
export FRESH_VARS_FOR_DYNACONF='["MYSQL_HOST", "OTHERVAR"]'
Then
from dynaconf import settings
print(settings.FOO) # This data was loaded once on import
print(settings.MYSQL_HOST) # This data is being read from redis imediatelly!
Using programatically
Sometimes you want to override settings for your existing Package or Framework lets say you have a conf module exposing a config object and used to do:
from myprogram.conf import config
Now you want to use Dynaconf, open that conf.py
or conf/__init__.py
and do:
# coding: utf-8
from dynaconf import LazySettings
config = LazySettings(GLOBAL_ENV_FOR_DYNACONF="MYPROGRAM")
Now you can use export MYPROGRAM_FOO=bar
instead of DYNACONF_FOO=bar
Switching working environments
To switch the environment
programatically you can use setenv
or using_env
.
Using context manager
from dynaconf import settings
with settings.using_env('envname'):
# now values comes from [envmane] section of config
assert settings.HOST == 'host.com
Using env setter
from dynaconf import settings
settings.setenv('envname')
# now values comes from [envmane] section of config
assert settings.HOST == 'host.com'
settings.setenv()
# now working env are back to previous
Feature flag system (feature toggles)
Feature flagging is a system to dynamically toggle features in your application based in some settings value.
Learn more at: https://martinfowler.com/articles/feature-toggles.html
Example:
write flags to redis
$ dynaconf write redis -s NEWDASHBOARD=1 -e premiumuser
meaning: Any premium user has NEWDASHBOARD feature enabled!
In your program do:
usertype = 'premiumuser' # assume you loaded it from your database
# user has access to new dashboard?
if settings.flag('newdashboard', usertype):
activate_new_dashboard()
else:
# User will have old dashboard if not a premiumuser
activate_old_dashboard()
The value is ensured to be loaded fresh from redis server so features can be enabled or disabled at any time without the need to redeploy.
It also works with file settings but the recommended is redis as the data can be loaded once it is updated.
Framework Extensions
Flask Extension
Dynaconf providesa drop in replacement for app.config
This an extension makes your app.config
in Flask to be a dynaconf
instance.
from flask import Flask
from dynaconf import FlaskDynaconf
app = Flask(__name__)
FlaskDynaconf(app)
Now the app.config
will work as a dynaconf.settings
and FLASK_ will be the
global prefix for exporting environment variables.
export FLASK_DEBUG=true
export FLASK_INTVALUE=1
The working environment will also respect the FLASK_ENV
variable, so FLASK_ENV=development
to work
in development mode or FLASK_ENV=production
to switch to production.
NOTE: To use
$ dynaconf
CLI theFLASK_APP
must be defined.
Django Extension
Dynaconf a drop in replacement to django.conf.settings
This an extension makes your app.config
in Flask to be a dynaconf
instance.
In your django project's settings.py
include:
INSTALLED_APPS = [
'dynaconf.contrib.django_dynaconf',
...
]
NOTE: The extension must be included as the first INSTALLED_APP of the list
Now create your settings.{py|yaml|toml|ini|json}
in your project's root directory
(the same folder where manage.py
is located)
Now django.conf.settings will work as a dynaconf.settings
instance and DJANGO_ will
be the global prefix to export environment variables.
export DJANGO_DEBUG=true
export DJANGO_INTVALUE=1
It is recommended that all the django's internal config vars should be kept
in the settings.py
of your project, then application specific values your can
place in dynaconf's settings.toml
in the root (same folder as manage.py).
You can override settings.py values in the dynaconf settings file.
NOTE: To use
$ dynaconf
CLI theDJANGO_SETTINGS_MODULE
must be defined and the cli must be called from the same directory where manage.py is placed.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file dynaconf-1.0.0.tar.gz
.
File metadata
- Download URL: dynaconf-1.0.0.tar.gz
- Upload date:
- Size: 61.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 60d5f8830d5952fac7ac490f6173c4ae0d0e6d5cc7c297d533608d640a80c9fe |
|
MD5 | f050d832086083dd6279698e3f1b2cb7 |
|
BLAKE2b-256 | 46d51ce47ebd7df9fc3f96ad3cd8f63161909778c0ab2f4d6b134c360629ecb3 |
File details
Details for the file dynaconf-1.0.0-py2.py3-none-any.whl
.
File metadata
- Download URL: dynaconf-1.0.0-py2.py3-none-any.whl
- Upload date:
- Size: 66.2 kB
- Tags: Python 2, Python 3
- Uploaded using Trusted Publishing? No
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | a0393ddf59ad4272fb41136b553360bd61ef40e3de238baf22af9cfe60e784aa |
|
MD5 | 948cdaeafd67658829c3be0a3ee5aae1 |
|
BLAKE2b-256 | 4ed12d9c6d0e3f34fdf7c0d4f26f1f09c8890106f4c0581e421f592e1e9f9b17 |