Cogs 0.1.1

Overview

Cogs is a toolkit for developing command-line utilities in Python. It handles common operations such as parsing command-line parameters, dispatching commands, loading configuration files and is targeted to developers, sysadmins, testers, or anybody else who needs to script their routine tasks.

Cogs is a free software licensed under MIT license. Cogs is written by Kirill Simonov from Prometheus Research, LLC.

Getting Started

You can install Cogs using PIP package manager:

# pip install Cogs

This operation installs a command-line utility cogs and a Python package of the same name. The cogs utility is a dispatcher which which lets you select and execute your scripts (called tasks). Let us show how to do it with a simple example.

In the current directory, make a subdirectory cogs.local and create a file __init__.py with the following content:

from cogs import task, log
import os

@task
def Hello(name=None):
    """greet the given entity (if not specified, the current user)"""
    if name is None:
        name = os.getlogin()
    log("Hello, %s!" % name.capitalize())

Now run:

$ cogs hello world
Hello, World!

$ cogs hello
Hello, Xi!

$ cogs help hello
HELLO - greet the given entity (if not specified, the current user)
Usage: cogs hello [<name>]

Cogs converts function Hello() into a command-line script with parameters inferred from the function signature, so then when you execute a command cogs hello world, you call a function Hello('world').

Loading Extensions

In this section, we describe how Cogs finds and loads extensions.

Cogs loads extensions from two places:

• cogs.local subdirectory in the current directory;

• all Python packages under cogs.extensions entry point.

The easiest way to add an extension is to create a cogs.local subdirectory and add your scripts there. The subdirectory must contain an __init__.py script, which is executed by Cogs on startup. The cogs.local subdirectory must be owned by the same user who runs the cogs script, or by root; otherwise it is ignored.

If you need to package and distribute your Cogs extensions, using cogs.local may be inconvenient. In this case, you may package your Cogs extensions as a regular Python distribution.

Suppose we want to pack the hello task as a separate package. Create a directory tree with the following structure:

Cogs-Hello/
    src/
        cogs_hello/
            __init__.py
    setup.py

The file cogs_hello/__init__.py contains the definition of the hello task and has the same content as cogs.local/__init__.py in our previous example.

The file setup.py contains the meta-data of the package and may have the following content:

from setuptools import setup

setup(
    name='Cogs-Hello',
    version='0.0.1',
    description="""A Cogs task to greet somebody""",
    packages=['cogs_hello'],
    package_dir={'': 'src'},
    install_requires=['Cogs'],
    entry_points={ 'cogs.extensions': ['Hello = cogs_hello'] },
)

Note the parameter entry_points in setup() invocation; it adds an entry point cogs.extensions named Hello that refers to package cogs_hello. On startup, Cogs finds and loads all packages defined for the entry point cogs.extensions.

Defining Tasks

A task can be created from a function or a class by augmenting it with the task decorator:

from cogs import task, argument, log, fail

@task
def Factorial(n):
    """calculate n!

    This task calculates the value of the factorial of the given
    positive number `n`.  Factorial of n, also known as n!, is
    defined by the formula:

        n! = 1*2*...*(n-1)*n
    """
    try:
        n = int(n)
    except ValueError:
        raise fail("n must be an integer")
    if n < 1:
        raise fail("n must be positive")
    f = 1
    for k in range(2, n+1):
        f *= k
    log("%s! = `%s`" % (n, f))

@task
class Fibonacci:
    """calculate the n-th Fibonacci number

    The n-th Fibonacci number `F_n` is defined by:

        F_0 = 0
        F_1 = 1
        F_n = F_{n-1}+F_{n-2} (n>1)
    """

    n = argument(int)

    def __init__(self, n):
        if n < 0:
            raise ValueError("n must be non-negative")
        self.n = n

    def __call__(self):
        p, q = 0, 1
        for k in range(self.n):
            p, q = p+q, p
        log("F_%s = `%s`" % (self.n, p))

You can now execute the tasks by running:

$ cogs factorial 10
10! = 3628800

$ cogs fibonacci 10
F_10 = 55

Cogs uses the name of the function or the class as the task identifier. The name is normalized: it is converted to lower case and has all underscore characters converted to the dash symbol.

If the task is derived from a function, the task arguments are inferred from the function signature. Cogs executes such a task by calling the function with the parsed command-line parameters.

If the task is derived from a class, the task arguments and options must be defined using argument() and option() descriptors. To execute a task, Cogs creates an instance of the class passing the task parameters as the constructor arguments. Then Cogs invokes the __call__ method of the instance. Thus the call of:

$ cogs factorial 10

is translated to:

Factorial('10')

and the call of:

$ cogs fibonacci 10

is translated to:

t = Fibonacci(10)
t()

The docstring of the function or the class becomes the task description:

$ cogs help factorial
FACTORIAL - calculate n!
Usage: cogs factorial <n>

This task calculates the value of the factorial of the given
positive number n.  Factorial of n, also known as n!, is
defined by the formula:

    n! = 1*2*...*(n-1)*n

$ cogs help fibonacci
Usage: cogs fibonacci <n>

The n-th Fibonacci number F_n is defined by:

    F_0 = 0
    F_1 = 1
    F_n = F_{n-1}+F_{n-2} (n>1)

A task derived from a function cannot have options. To add an option to a task derived from a class, use the option() descriptor. For example:

from cogs import task, argument, option
import sys, os

@task
class Write_Hello:

    name = argument(default=None)
    output = option(key='o', default=None)

    def __init__(self, name, output):
        if name is None:
            name = os.getlogin()
        self.name = name
        if output is None:
            self.file = sys.stdout
        else:
            self.file = open(output, 'w')

    def __call__(self):
        log("Hello, %s!" % self.name.capitalize(),
            file=self.file)

You can execute this task with option --output or -o to redirect the output to a file:

$ cogs write-hello world -o hello.txt

Configuration and Environment

Cogs allows you to define custom configuration parameters. For example:

from cogs import env, task, setting, log
import os

@setting
def Default_Name(name=None):
    """the name to use for greetings (if not set: login name)"""
    if name is None or name == '':
        name = os.getlogin()
    if not isinstance(name, str):
        raise ValueError("a string value is expected")
    env.add(default_name=name)

@task
def Hello_With_Configuration(name=None):
    if name is None:
        name = env.default_name
    log("Hello, %s!" % name.capitalize())

Now you could specify the name as a configuration parameter default-name. One way to do it is to use global option --default-name:

$ cogs --default-name=world hello-with-configuration

You could also pass a configuration parameter using an environment variable:

$ COGS_DEFAULT_NAME=world cogs hello-with-configuration

Alternatively, you can put parameters to a configuration file. In the current directory, create a file cogs.conf with the following content:

default-name: world

Now run:

$ cogs hello-with-configuration

Cogs reads configuration from the following locations:

• /etc/cogs.conf

• $PREFIX/etc/cogs.conf

• $HOME/.cogs/cogs.conf

• ./cogs.conf

• program environment

• command-line parameters

To create a new configuration parameter, wrap a function named after the parameter with the @setting decorator. The function must accept zero or one argument: the function is called without arguments if the parameter is not specified explicitly, and is called with the value of the parameter is it was set using one of the methods described above.

Cogs does not impose any rules on what to do with the parameter value, but we recommend to store the value in the global env variable. The call of env.add(default_name=name) adds a new parameter default_name which could then be accessed as env.default_name.

API Reference

The following functions and classes are defined in the package cogs:

@task

The @task decorator converts the wrapped function or class into a task. Task properties are inferred from the wrapped object as follows:

name

Generated from the function or the class name. The name is converted to lower case and all underscores are replaced with dashes.

documentation

Generated from the docstring. The first line of the docstring produces a one-line hint string, the rest of the docstring produces a multi-line help string.

arguments

When the task is inferred from a function, the arguments are generated from the function signature. Each function parameter becomes a task argument, those which have default values are optional.

If the task is inferred from a class, the arguments must be specified using the argument() descriptor.

options

A task inferred from a function has no options. A task inferred from a class may have options specified using the option() descriptor.

When a task is executed, the wrapped object is invoked according to the following rules:

• If the task is inferred from a function, parsed command-line parameters are passed as the function arguments.

• If the task is inferred from a class, command-line parameters are passed to the class constructor, then the __call__ method is called on the instance.

@setting

The @setting decorator converts the wrapped function to a configuration parameter, which properties are inferred from the function attributes.

The setting name is generated from the function name. The name is converted to lower case and has all underscores replaced with dashes.

The setting documentation is generated from the function docstring.

The function must be able to accept zero and one parameter. The function is called at startup with no parameters if the setting is not explicitly set by the user; otherwise it is called with the value of the setting. The function is responsible for storing the value in the env object.

argument(check, default, plural=False)

Describe a task argument.

check

A function which is called to check and/or transform the argument value. The function must return the transformed value or raise ValueError exception on error.

default

The default value to be used if the argument is optional and not specified. If this parameter is not set, the argument is mandatory.

plural

If set, the argument consumes all the remaining command-line parameters. Must be the last argument specified.

option(key, check, default, plural=False, value_name=None, hint=None)

Describe a task option.

key

A one-character shorthand.

check

A function called to check and transform the value of the option. The function must return the transformed value or raise ValueError exception on error.

default

The default value used when the option is not specified. If this parameter is not set, the option does not accept a value. Such an option is treated is a toggle and takes a value True if set and False if not set.

plural

If set, indicates that the option could be specified more than once.

value_name

The preferred name for the option value; used for the task description.

hint

A one-line description of the option; used for the task description.

env

A global object that keeps values of configuration parameters and other properties.

env.add(**keywords)

Add new parameters.

env.set(**keywords)

Set values for existing parameters.

env.push(**keywords)

Save the current state and set new values for existing parameters.

env.pop()

Restore a previously saved state of parameters and values.

env(**keywords)

A context manager for with statement. On entering, saves the current state and sets new parameter values. On exiting, restores the saved state.

log(*msgs, sep=' ', end='\n', file=sys.stdout)

Print the data to the standard output.

log() has the interface and behavior similar to the standard print() function.

log() supports styling: a substring of the form:

`...`

or:

:fmt:`...`

is colorized when displayed on a color terminal. The supported formats are: default (white), footnote (dark grey), warning (red), success (green).

debug(*msgs, sep=' ', end='\n', file=sys.stderr)

Print the data when the env.debug parameter is set. We recommend to accompany any permanent change to the filesystem or other system state with a respective debug() call.

Add command-line parameter --debug or set environment variable COGS_DEBUG=1 to see debug output.

warn(*msgs, sep=' ', end='\n', file=sys.stderr)

Display a warning. warn() should be used for reporting error conditions which do not prevent the script from continuing the job.

fail(*msgs, sep=' ', end='\n', file=sys.stderr)

Display an error message and return an exception object. It should be used in the following manner:

raise fail("no more beer in the refrigerator")

cp(src, dst)

Copy a file or a directory tree.

mv(src, dst)

Move a file or a directory tree.

rm(path)

Remove a file.

mktree(path)

Create all directories in the path.

rmtree()

Remove a directory tree.

exe(cmd)

Replace the current process with the given shell command.

sh(cmd, data=None, cd=None)

Execute a shell command with the given input and working directory.

pipe(cmd, data=None, cd=None)

Execute a shell command with the given input and working directory; return the command output.