yay 3.1.0

Add new lazy lookup method to the root config object. This is useful if you are building a lazy iterator over a dictionary key:

def example_iterator(self, expression):
    try:
        for item in expression.expand():
            resolved = item.resolve()
            create_obj_from_settings(resolved)
    except CreateObjectError:
        # You can get line and column from the item object!
    except yay.errors.Error:
        # Ideal place to wrap Yay errors in your apps exceptions

example_iterator(config.lookup("somekey"))

You can see here that the API lets you defer any messy “key not found” exceptions until the iterator needs to access they key - and the iterator already has apropriate exception handling.

Add additional test cases for backwards compatibility

Introduce a lazily evaluated mechanism for expanding the search path:

.search:
  - package://some.egg/
  - http://raw.github.com/isotoma/yay/

Any subsequent .import statements will use the modified search path.

If a package specified either as part of a .include or as part of a .search is not available on sys.path then an attempt will be made to install it.

Can pass config to opener backends using .config. For example, you might want to have the package:// opener to use your internal package repository:

.config:
    openers:
        packages:
            index: https://my-python-mirror/simple/
            username: joe
            password: penguin55

You can use variable subsitution and to define the password in a GPG encrypted yay file or to push the variable from application that is using Yay.

Yay openers support basic auth in the url. If used with .include then use of secrets to concatenate to hide the password is suggsted:

# Define ${password} in a GPG armored file
.include: home://.credentials.yay.gpg

.search:
  - https://username:${password}@svn.yourcompany.org/svn/cookbook/trunk

.include and .search may be a single scalar value rather than a list if you desire.

Allow expressions to return an empty dictionary:

result: ${site.vhosts else {}}

Added a string replacement function allowing people to do things like:

something: jolly good show
slug: ${replace(something, " ", "-")}

thereafter the value of slug will be jolly-good-show.

Support in operator during foreach:

result.foreach v in somevalues if v in somelist: ${v}.

Allow expressions to contain literal strings and floats:

result: ${site.description else "No description"}
result2: ${3.212121}

Allow expressions to return an empty list:

result: ${site.vhosts else []}

Allow equality checks of undefined keys to pass or fail appropriately without raising a NoMatching exception:

data:
  part1:
    a: 1
  part2:
    c: 2

res1.foreach p in data if data[p].a = 2: ${p}
res2.foreach p in data if data[p].c != 1: ${p}

res1 will be empty as neither part1 nor part2 have an a field with a value of 2, and res2 will contain part1 and part2 as neither have a c field wth a value of 1.

Add an undefined that raises a NoMatch exception when resolved. This allows one to filter by whether or not a key is present in a mapping.

data:

part1:

a: 1

part2:

c: 2

res.foreach p in data if data[p].c = undefined: ${p}

res will contain part1 as part1 doesn’t have a c field.

You can now pass an etag to the open method of Openers. This will throw a NotModified exception if etag matching shows the request URI is unchanged. This can be used to implement caching strategies and reduce the amount of data that yay pulls over network connections:

>>> from yay.openers import Openers
>>> fp = Openers().open("/tmp/somefile, etag="1234535462356456fa")
NotModified: File '/tmp/somefile' not modified

All openers will now have an etag property. This property can be None if a suitable etag cannot be provided. Otherwise it should be some hash that guarantees the state of the file so that later requests can just ask “has it changed since it was in this state”. To get an etag:

>>> from yay.openers import Openers
>>> fp = Openers().open("/tmp/somefile")
>>> print fp.etag
1234535462356456fa

You can inject Python objects into the Yay Config object, as long as Yay knows how to box them. If you had a Yay file like this:

hello: abc
result: ${hello}

And the following python code:

from yay.config import Config
c = Config()
c.load_uri(example_file)
c.add(dict(hello="xyx"))

Then the following assertion is true:

c.get() == dict(hello="xyz", result="xyz")

There is now a home:// URL handler. For my laptop this means:

import yay
yay.load_uri("home://foo/bar.yay")

Is equivalent to:

import yay
yay.load_uri("/home/john/foo/bar.yay")

Improve searchpath handling to better cope with URIs.

Add support for else keyword in expressions as per YEP2:

foo:
    bar: ${foo.baz else 52}

This is useful for providing sensible defaults in your recipes.

The loop block is now autoflattening. This means that constructs that yield lists of lists will be turned into 1-dimensional lists by .foreach. For example:

somelist:
  - - - a
      - b
    - c
  - d

someotherlist.foreach var in somelist: ${var}

The list someotherlist would now contain:

somelist:
 - a
 - b
 - c
 - d

The previous behaviour can be obtained with the chain modifier:

somelist.foreach var in somelist chain: ${var}

Add a simple macro language. This is highly EXPERIMENTAL and subject to frequent syntax tweaks. You can define a reusable block of configuration using the .define statement. An example of this in Yaybu to create a reusable virtualenv step:

.define Virtualenv:
  - Directory:
      name: ${venv.location}
      owner: ${venv.owner}
  - Execute:
      name: create-virtualenv-${venv.location}
      command: virtualenv --no-site-packages ${venv.location}
      creates: ${venv.location}/bin/pip
  - Execute:
      name: install-requirements-${venv.location}
      command: ${venv.location}/bin/pip install -r ${venv.requirements}

The macro is callable inline as part of an expression. For example:

resources.append:
  .foreach venv in virtualenvs: ${Virtualenv!}

Equally you can do this:

.define Virtualenv:
  - Directory:
      name: ${location}
      owner: ${owner}
# SNIP

resources.append:
    Virtualenv!:
      location: /tmp/example
      owner: dave
      requirements: /tmp/requirements.txt

This release refactors ProtectedString to make it more useful outside Yay. Instead of importing yay.protectedstring.ProtectedString you should import yay.String

You can now pass strings to the constructor and they will automatically be wrapped. Previously you had to pass StringPart’s to the constructor. That is now an internal implementation detail.

There is now an extend() function. This accepts lists that contains a mixture of raw strings and other String objects:

s = String("echo")
y = String("supersekritpassword", secret=True)
s.extend(["a", "b", y, "d"])

There is now an as_list() function. This returns all the parts you have added to a string. This is useful if you are building a command line to pass to subprocess:

s.String(["someprogram", "--pasword"])
s.add_secret("password")

# Log the obfuscated version but execute with the real password
log.info(s.protected)
p = subprocess.Popen(s.as_list(secret=False), cwd="/")

There is now a secret call in Yay for exercising secret Yay without needing to use GPG. This is mainly for test purposes, but might be useful if you have strings which arent closely guarded secrets yet you dont want them show in logs. To use it you do something like this:

resources.append:
  - Checkout:
      name: /checkouts/mycode
      repository: http://github.com/whatever
      scm_password.secret: mypassword

This example is from Yaybu and would stop Yaybu from logging your SVN password.

Any stream objects returned by the Openers() API now have a len property.

The internal Openers() API can now load from any package on the Python path:

fp = Openers().open("package://some.egg/hello.txt")

The internal Openers() API now has support for a search path:

fp = Openers(searchpath=['file:///home/john', 'http://google.com']).open("foo.txt")

The Openers() API now supports https://