configzen

configzen – easily create and maintain complex, statically-typed configurations with validation in Python.

What is this?

configzen is a good choice if you need to create complex configurations with schemas. Being based on pydantic, this tool will allow you to create configuration models for your configuration files, and then load, modify and save them with scope control. To see roughly how it works, check out the Features section.

Preprocessing

configzen provides built-in preprocessing directives to your configuration files, offering features such as extending configuration files directly from other configuration files (without writing any code). You might think of it as something that is analogous to Azure DevOps YAML templates, broadened to any from the supported configuration file formats (see Supported file formats). The directive ^copy may also be handy in quick conversions between the mentioned formats. See Preprocessing directives for more information.

Supported file formats

configzen uses anyconfig to serialize and deserialize data and does not operate on any protocol-specific entities. As an example result, comments in your configuration files are lost on save[^1], but you can exchange file formats without any hassle.

The following table shows the supported file formats, their requirements, file extensions, and the backend libraries used to accomplish this goal.

If your file extension is not recognized, you can register your own file extension by calling ConfigAgent.register_file_extension(file_extension, parser_name).

If your favorite backend library is not supported, please let me know by reporting it as an issue. Using custom backends is to be supported in the future.

[^1]: A suggested alternative for comments is to use the description parameter in your configuration models' fields: ConfigField(description=...). The provided field descriptions are included in JSON schemas generated by the default implementation of the ConfigModel.schema() method.

Features

Managing content

Having a YAML configuration file like this:

# database.yml
host: 127.0.0.1
port: 5432
user: postgres

You can create a configzen configuration model for it like this:

# config.py
from ipaddress import IPv4Address, IPv6Address
from configzen import ConfigModel, ConfigMeta, ConfigField


class DatabaseConfig(ConfigModel):
    host: IPv4Address | IPv6Address
    port: int
    user: str
    password: str = ConfigField(exclude=True)

    class Config(ConfigMeta):
        resource = "database.yml"
        env_prefix = "DB_"


db_config = DatabaseConfig.load()

With this code written, you can load your configuration from a file as well as from the environment variables DB_HOST, DB_PORT, DB_USER and DB_PASSWORD. Since password is a field created with the option exclude=True, it will not be included in the configuration's exported data: that guarantees that your password does never leak into database.yml on save – but you may still pass it through an environment variable (here – the mentioned DB_PASSWORD). Secret files are also supported, see the pydantic documentation section for more information.

pydantic will naturally take care of parsing and validating the loaded data. Configuration models inherit from the pydantic.BaseSettings class, so you can use all of its features: schema generation, type conversion, validation, etc.

There are additional features brought to you by configzen worth checking out, though.

You can use the db_config object defined above to access the configuration values:

>>> db_config.host
IPv4Address('127.0.0.1')

modify them, if the pydantic model validation allows it (<Your model>.Config.validate_assignment will be True by default):

>>> db_config.host = "0.0.0.0"
>>> db_config.host
IPv4Address('0.0.0.0')

as well as reload particular values, without touching the rest of the configuration:

>>> db_config.at("port").reload()
5432
>>> db_config
DatabaseConfig(host=IPv4Address('0.0.0.0'), port=5432, user='postgres', password='password')
>>> db_config.at("host").reload()
IPv4Address('127.0.0.1')
>>> db_config
DatabaseConfig(host=IPv4Address('127.0.0.1'), port=5432, user='postgres', password='password')

or reload the whole configuration:

>>> db_config.port = 1234
>>> db_config.reload()
DatabaseConfig(host=IPv4Address('127.0.0.1'), port=5432, user='postgres', password='password')

or save a particular value, without touching the rest of the configuration:

>>> db_config.host = "0.0.0.0"
>>> db_config.port = 443
>>> db_config
DatabaseConfig(host=IPv4Address('0.0.0.0'), port=443, user='postgres', password='password')
>>> db_config.at("host").save()
40
>>> db_config.reload()
DatabaseConfig(host=IPv4Address('0.0.0.0'), port=5432, user='postgres', password='password')

or save the whole configuration:

>>> db_config.save()
39

Preprocessing

To see supported preprocessing directives, see Supported preprocessing directives.

Basic usage

Having a base configuration file like this (base.json):

{
  "i18n": {
    "language": "en",
    "timezone": "UTC"
  },
  "app": {
    "debug": true,
    "expose": 8000
  }
}

create another configuration file like this, overriding desired sections as needed:

# production.yml
^extend: base.json

+app:
  debug: false

and load the production.yml configuration file. No explicit changes to the code indicating the use of the base.json file are needed.

Note: Using + in front of a key will update the section already defined at that key, instead of replacing it.

Notice how configuration file formats do not matter in configzen: you can extend JSON configurations with YAML, but that might be as well any other format among the supported ones (see the Supported file formats section).

The above example is equivalent to as if you used:

# production.yml
i18n:
  language: en
  timezone: UTC
app:
  debug: false
  expose: 8000

but with a significant difference: when you save the above configuration, the ^extend relation to the base configuration file base.json is preserved. This basically means that changes made in the base configuration file will apply to the configuration model instance loaded from the ^extend-ing configuration file. Any changes made locally to the model will result in + sections being automatically added to the exported configuration data.

Supported preprocessing directives

Interpolation

Basic interpolation

You can use interpolation in your configuration files:

cpu:
  cores: 4
num_workers: ${cpu.cores}

>>> from configzen import ConfigModel
...
>>> class CPUConfig(ConfigModel):
...     cores: int
...
>>> class AppConfig(ConfigModel):
...     cpu: CPUConfig
...     num_workers: int
...
>>> app_config = AppConfig.load("app.yml")
>>> app_config
AppConfig(cpu=CPUConfig(cores=4), num_workers=4)

Reusable configuration with namespaces

You can share independent configuration models as namespaces through inclusion:

# database.yml
host: ${app_config::db_host}
port: ${app_config::expose}

# app.yml
db_host: localhost
expose: 8000

>>> from configzen import ConfigModel, include
>>> from ipaddress import IPv4Address
>>>
>>> @include("app_config")
... class DatabaseConfig(ConfigModel):
...     host: IPv4Address
...     port: int
...
>>> class AppConfig(ConfigModel):
...     db_host: str
...     expose: int
...
>>> app_config = AppConfig.load("app.yml")
>>> app_config
AppConfig(db_host='localhost', expose=8000)
>>> db_config = DatabaseConfig.load("database.yml")
>>> db_config
DatabaseConfig(host=IPv4Address('127.0.0.1'), port=8000)
>>> db_config.dict()
{'host': IPv4Address('127.0.0.1'), 'port': 8000}
>>> db_config.export()  # used when saving
{'host': '${app_config::db_host}', 'port': '${app_config::expose}'}

You do not have to pass a variable name to @include, though. @include lets you overwrite the main interpolation namespace or one with a separate name (here: app_config) with configuration models, dictionaries and their factories.

Setup

In order to use configzen in your project, install it with your package manager, for example pip:

pip install configzen

If you are willing to contribute to the project, which is awesome, simply clone the repository and install its dependencies with poetry:

poetry install --with dev

License

MIT License

Contributing

Contributions are welcome! Feel free to open an issue or submit a pull request.

Credits

• @Lunarmagpie for crucial design tips and ideas.

Author

• bswck (contact: bswck.dev@gmail.com or via Discord bswck#8238)