Skip to main content

Easy dict-based script configuration with CLI support

Project description

GitlabCIPipeline GitlabCICoverage Appveyor Pypi Downloads

The main webpage for this project is: https://gitlab.kitware.com/utils/scriptconfig

The goal of scriptconfig is to make it easy to be able to define a CLI by simply defining a dictionary. Thie enables you to write simple configs and update from CLI, kwargs, and/or json.

The scriptconfig provides a simple way to make configurable scripts using a combination of config files, command line arguments, and simple Python keyword arguments. A script config object is defined by creating a subclass of Config with a default dict class attribute. An instance of a custom Config object will behave similar a dictionary, but with a few conveniences.

Project Design Goals:

  • Write Python programs that can be invoked either through the commandline or via Python itself.

  • Drop in replacement for any dictionary-based configuration system.

  • Intuitive parsing (currently working on this), ideally improve on argparse if possible. This means being able to easilly specify simple lists, numbers, strings, and paths.

To get started lets consider some example usage:

>>> import scriptconfig as scfg
>>> # In its simplest incarnation, the config class specifies default values.
>>> # For each configuration parameter.
>>> class ExampleConfig(scfg.Config):
>>>     default = {
>>>         'num': 1,
>>>         'mode': 'bar',
>>>         'ignore': ['baz', 'biz'],
>>>     }
>>> # Creating an instance, starts using the defaults
>>> config = ExampleConfig()
>>> # Typically you will want to update default from a dict or file.  By
>>> # specifying cmdline=True you denote that it is ok for the contents of
>>> # `sys.argv` to override config values. Here we pass a dict to `load`.
>>> kwargs = {'num': 2}
>>> config.load(kwargs, cmdline=False)
>>> assert config['num'] == 2
>>> # The `load` method can also be passed a json/yaml file/path.
>>> config_fpath = '/tmp/foo'
>>> open(config_fpath, 'w').write('{"num": 3}')
>>> config.load(config_fpath, cmdline=False)
>>> assert config['num'] == 3
>>> # It is possbile to load only from CLI by setting cmdline=True
>>> # or by setting it to a custom sys.argv
>>> config.load(cmdline=['--num=4'])
>>> assert config['num'] == 4
>>> # Note that using `config.load(cmdline=True)` will just use the
>>> # contents of sys.argv

Notice in the above example the keys in your default dictionary are command line arguments and values are their defaults. You can augment default values by wrapping them in scriptconfig.Value objects to encapsulate information like help documentation or type information.

>>> import scriptconfig as scfg
>>> class ExampleConfig(scfg.Config):
>>>     default = {
>>>         'num': scfg.Value(1, help='a number'),
>>>         'mode': scfg.Value('bar', help='mode1 help'),
>>>         'mode2': scfg.Value('bar', type=str, help='mode2 help'),
>>>         'ignore': scfg.Value(['baz', 'biz'], help='list of ignore vals'),
>>>     }
>>> config = ExampleConfig()
>>> # smartcast can handle lists as long as there are no spaces
>>> config.load(cmdline=['--ignore=spam,eggs'])
>>> assert config['ignore'] == ['spam', 'eggs']
>>> # Note that the Value type can influence how data is parsed
>>> config.load(cmdline=['--mode=spam,eggs', '--mode2=spam,eggs'])

Features

  • Serializes to json

  • Dict-like interface. By default a Config object operates independent of config files or the command line.

  • Can create command line interfaces

    • Can directly create an independent argparse object

    • Can use special command line loading using self.load(cmdline=True). This extends the basic argparse interface with:

      • Can specify options as either --option value or --option=value

      • Default config options allow for “smartcasting” values like lists and paths

      • Automatically add --config, --dumps, and --dump CLI options when reading cmdline via load.

Gotchas

CLI Values with commas:

When using scriptconfig to generate a command line interface, it uses a function called smartcast to try to determine input type when it is not explicitly given. If you’ve ever used a program that tries to be “smart” you’ll know this can end up with some weird behavior. The case where that happens here is when you pass a value that contains commas on the command line. If you don’t specify the default value as a scriptconfig.Value with a specified type, if will interpret your input as a list of values. In the future we may change the behavior of smartcast, or prevent it from being used as a default.

Boolean flags:

scriptconfig is currently strictly key-value. It does not support boolean flags (e.g. --flag), you must set it to a value (e.g. --flag=True).

FAQ

Question: How do I override the default values for a scriptconfig object using json file?

Answer: This depends if you want to pass the path to that json file via the command line or if you have that file in memory already. There are ways to do either. In the first case you can pass --config=<path-to-your-file> (assuming you have set the cmdline=True keyword arg when creating your config object e.g.: config = MyConfig(cmdline=True). In the second case when you create an instance of the scriptconfig object pass the default=<your dict> when creating the object: e.g. config = MyConfig(default=json.load(open(fpath, 'r'))). But the special --config --dump and --dumps CLI arg is baked into script config to make this easier.

TODO

  • [ ] Policy on nested heirachies (currently disallowed)

  • [ ] Policy on smartcast (currently enabled)

  • [ ] Policy on positional arguments (currently experimental)

    • [ ] Fixed length

    • [ ] Variable length

    • [ ] Can argparse be modified to always allow for them to appear at the beginning or end?

    • [ ] Can we get argparse to allow a positional arg change the value of a prefixed arg and still have a sane help menu?

  • [ ] Policy on boolean flags (needs exploration)

  • [ ] Improve over argparse’s default autogenerated help docs (needs exploration on what is possible with argparse and where extensions are feasible)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

scriptconfig-0.5.7.tar.gz (19.2 kB view details)

Uploaded Source

Built Distribution

scriptconfig-0.5.7-py2.py3-none-any.whl (17.1 kB view details)

Uploaded Python 2 Python 3

File details

Details for the file scriptconfig-0.5.7.tar.gz.

File metadata

  • Download URL: scriptconfig-0.5.7.tar.gz
  • Upload date:
  • Size: 19.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/49.3.1 requests-toolbelt/0.9.1 tqdm/4.48.2 CPython/3.8.5

File hashes

Hashes for scriptconfig-0.5.7.tar.gz
Algorithm Hash digest
SHA256 e281f2c9343ef1ec824060bd6e3b67de8f372f0b15a207509eb77b3a7fae0f6f
MD5 f6c92d37505717fe1eb4aab25483774d
BLAKE2b-256 6b2409b95adac77b79db1d0d374d5bb20f28c44fca7727a7b3a38d5d84efe32c

See more details on using hashes here.

File details

Details for the file scriptconfig-0.5.7-py2.py3-none-any.whl.

File metadata

  • Download URL: scriptconfig-0.5.7-py2.py3-none-any.whl
  • Upload date:
  • Size: 17.1 kB
  • Tags: Python 2, Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/49.3.1 requests-toolbelt/0.9.1 tqdm/4.48.2 CPython/3.8.5

File hashes

Hashes for scriptconfig-0.5.7-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 1bf74cbdf961618d312d88c8549f97b171517842c8acfe8d59e8a94d89af7b98
MD5 0515eeba43dc422e4bec66a958ede46f
BLAKE2b-256 4a80e9a0756d7d3a053c42d5c144bc032d526eb9612fc68471aee8be7e69bfee

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page