slapos.recipe.template

Templating recipe with remote resource support.

Project description

Template recipe which supports remote resource.

Inspired by collective.recipe.template, with minimum set of features, but with (hopefully) safer buildout-based templating.

“jinja2” entry point allows rendering jinja2 templates.

Usage

Getting started

You can start by a simple buildout:

>>> write('buildout.cfg',
... '''
... [buildout]
... parts = template
...
... [template]
... recipe = slapos.recipe.template
... url = template.in
... output = template.out
...
... [section]
... option = value
... ''')

And a simple template:

>>> write('template.in', '${section:option}')

We run buildout:

>>> run_buildout()
Installing template.

And the output file has been parsed by buildout itself:

>>> cat('template.out')
value

Full options

There is two non required options:

md5sum: Check the integrity of the input file.
mode: Specify the filesystem permissions in octal notation.

Check file integrity

Let’s write a file template:

>>> write('template.in', '${buildout:parts}')

Compute its MD5 sum:

>>> from hashlib import md5
>>> with open('template.in', 'rb') as f:
...   md5sum = md5(f.read()).hexdigest()

Write the buildout.cfg using slapos.recipe.template:

>>> write('buildout.cfg',
... '''
... [buildout]
... parts = template
...
... [template]
... recipe = slapos.recipe.template
... url = template.in
... output = template.out
... md5sum = ''' + md5sum + '''
... ''')

And run buildout, and see the result:

>>> run_buildout()
Uninstalling template.
Installing template.

>>> cat('template.out')
template

If the md5sum doesn’t match, the buildout fail:

>>> write('buildout.cfg',
... '''
... [buildout]
... parts = template
...
... [template]
... recipe = slapos.recipe.template
... url = template.in
... output = template.out
... md5sum = 0123456789abcdef0123456789abcdef
... ''')
>>> run_buildout()
While:
  Installing.
  Getting section template.
  Initializing section template.
Error: MD5 checksum mismatch for local resource at 'template.in'.

Specify filesystem permissions

You can specify the mode of the written file:

>>> write('template.in', '${buildout:installed}')

>>> write('buildout.cfg',
... '''
... [buildout]
... parts = template
...
... [template]
... recipe = slapos.recipe.template
... url = template.in
... output = template.out
... mode = 0627
... ''')

>>> run_buildout()
Uninstalling template.
Installing template.

And the generated file with have the right permissions:

>>> import os, stat
>>> print("0%o" % stat.S_IMODE(os.stat('template.out').st_mode))
0627

Fetching template source from an URL

You can fetch resources from an URL:

>>> server_data = tmpdir('server_data')
>>> server_url = start_server(server_data)

>>> write(server_data, 'template.in', '${buildout:parts}')
>>> write('buildout.cfg', '''
... [buildout]
... parts = template
...
... [template]
... recipe = slapos.recipe.template
... url = %stemplate.in
... md5sum = %s
... output = template.out
...
... ''' % (server_url, md5sum))

>>> run_buildout()
Downloading http://localhost/template.in
Cannot download http://localhost/template.in from network cache.
Uninstalling template.
Installing template.

>>> cat('template.out')
template

Section dependency

You can use other part of buildout in the template. This way this parts will be installed as dependency:

>>> write('template.in', '${dependency:foobar}')
>>> write('buildout.cfg', '''
... [buildout]
... parts = template
...
... [template]
... recipe = slapos.recipe.template
... url = template.in
... output = template.out
...
... [dependency]
... foobar = dependency content
... recipe = zc.buildout:debug
... ''')

>>> run_buildout()
Uninstalling template.
Installing dependency.
  foobar='dependency content'
  recipe='zc.buildout:debug'
Installing template.

This way you can get options which are computed in the __init__ of the dependent recipe.

Let’s create a sample recipe modifying its option dict:

>>> write('setup.py',
... '''
... from setuptools import setup
...
... setup(name='samplerecipe',
...       entry_points = {
...           'zc.buildout': [
...                'default = main:Recipe',
...           ],
...       }
...      )
... ''')
>>> write('main.py',
... '''
... class Recipe(object):
...
...     def __init__(self, buildout, name, options):
...         options['data'] = 'foobar'
...
...     def install(self):
...         return []
... ''')

Let’s just use buildout.cfg using this egg:

>>> write('template.in', '${sample:data}')
>>> write('buildout.cfg',
... '''
... [buildout]
... develop = .
... parts = template
...
... [template]
... recipe = slapos.recipe.template
... url = template.in
... output = template.out
...
... [sample]
... recipe = samplerecipe
... ''')
>>> run_buildout()
Develop: '/sample-buildout/.'
Uninstalling template.
Uninstalling dependency.
Installing sample.
Installing template.
>>> cat('template.out')
foobar

Jinja2 usage

Getting started

Example buildout demonstrating some types:

>>> write('buildout.cfg',
... '''
... [buildout]
... parts = template
...
... [template]
... recipe = slapos.recipe.template:jinja2
... template = foo.in
... rendered = foo
... context =
...     key     bar          section:key
...     key     recipe       :recipe
...     raw     knight       Ni !
...     import  json_module  json
...     section param_dict   parameter-collection
...
... [parameter-collection]
... foo = 1
... bar = bar
...
... [section]
... key = value
... ''')

And according Jinja2 template (kept simple, control structures are possible):

>>> write('foo.in',
...     '{{bar}}\n'
...     'Knights who say "{{knight}}"\n'
...     '${this:is_literal}\n'
...     '${foo:{{bar}}}\n'
...     'swallow: {{ json_module.dumps(("african", "european")) }}\n'
...     'parameters from section: {{ param_dict | dictsort }}\n'
...     'Rendered with {{recipe}}\n'
...     'UTF-8 text: привет мир!\n'
...     'Unicode text: {{ "你好世界" }}\n'
... )

We run buildout:

>>> run_buildout()
Installing template.

And the template has been rendered:

>>> cat('foo')
value
Knights who say "Ni !"
${this:is_literal}
${foo:value}
swallow: ["african", "european"]
parameters from section: [('bar', 'bar'), ('foo', '1')]
Rendered with slapos.recipe.template:jinja2
UTF-8 text: привет мир!
Unicode text: 你好世界

Parameters

Mandatory:

template: Template url/path, as accepted by zc.buildout.download.Download.__call__ . For very short template, it can make sense to put it directly into buildout.cfg: the value is the template itself, prefixed by the string “inline:” + an optional newline.
rendered: Where rendered template should be stored.

Optional:

context

Jinja2 context specification, one variable per line, with 3 whitespace-separated parts: type, name and expression. Available types are described below. “name” is the variable name to declare. Expression semantic varies depending on the type.

Available types:

raw: Immediate literal string.
key: Indirect literal string.
import: Import a python module.
section: Make a whole buildout section available to template, as a dictionary.

Indirection targets are specified as: [section]:key . It is possible to use buildout’s buit-in variable replacement instead instead of key type, but keep in mind that different lines are different variables for this recipe. It might be what you want (factorising context chunk declarations), otherwise you should use indirect types.

md5sum

Template’s MD5, for file integrity checking. By default, no integrity check is done.

mode

Mode, in octal representation (no need for 0-prefix) to set output file to. This is applied before storing anything in output file.

once

Path of a marker file to prevents rendering altogether.

extensions

Jinja2 extensions to enable when rendering the template, whitespace-separated. By default, none is loaded.

import-delimiter

Delimiter character for in-temlate imports. Defaults to /. See also: import-list

import-list

Declares a list of import paths. Format is similar to context. “name” becomes import’s base name.

Available types:

rawfile: Literal path of a file.
file: Indirect path of a file.
rawfolder: Literal path of a folder. Any file in such folder can be imported.
folder: Indirect path of a folder. Any file in such folder can be imported.

encoding

Encoding for input template and output file. Defaults to utf-8.

FAQ

Q: How do I generate ${foo:bar} where foo comes from a variable ?

A: {{ '${' ~ foo_var ~ ':bar}' }}: This is required as jinja2 fails parsing “${{{ foo_var }}:bar}”. Though, jinja2 succeeds at parsing “${foo:{{ bar_var }}}” so this trick isn’t needed for that case.

Use jinja2 extensions

>>> write('foo.in',
... '''{% set foo = ['foo'] -%}
... {% do foo.append(bar) -%}
... {{ foo | join(', ') }}''')
>>> write('buildout.cfg',
... '''
... [buildout]
... parts = template
...
... [template]
... recipe = slapos.recipe.template:jinja2
... template = foo.in
... rendered = foo
... context = key bar buildout:parts
... # We don't actually use all those extensions in this minimal example.
... extensions = jinja2.ext.do jinja2.ext.loopcontrols
...   jinja2.ext.with_
... ''')
>>> run_buildout()
Uninstalling template.
Installing template.

>>> cat('foo')
foo, template

Check file integrity

Compute template’s MD5 sum:

>>> write('foo.in', '{{bar}}')
>>> from hashlib import md5
>>> with open('foo.in', 'rb') as f:
...     md5sum = md5(f.read()).hexdigest()
>>> write('buildout.cfg',
... '''
... [buildout]
... parts = template
...
... [template]
... recipe = slapos.recipe.template:jinja2
... template = foo.in
... rendered = foo
... context = key bar buildout:parts
... md5sum = ''' + md5sum + '''
... ''')
>>> run_buildout()
Uninstalling template.
Installing template.

>>> cat('foo')
template

If the md5sum doesn’t match, the buildout fail:

>>> write('buildout.cfg',
... '''
... [buildout]
... parts = template
...
... [template]
... recipe = slapos.recipe.template:jinja2
... template = foo.in
... rendered = foo
... context = key bar buildout:parts
... md5sum = 0123456789abcdef0123456789abcdef
... ''')
>>> run_buildout()
Uninstalling template.
Installing template.
While:
  Installing template.
Error: MD5 checksum mismatch for local resource at 'foo.in'.

Specify filesystem permissions

You can specify the mode for rendered file:

>>> write('template.in', '{{bar}}')
>>> write('buildout.cfg',
... '''
... [buildout]
... parts = template
...
... [template]
... recipe = slapos.recipe.template:jinja2
... template = template.in
... rendered = foo
... context = key bar buildout:parts
... mode = 205
... ''')
>>> run_buildout()
Installing template.

And the generated file with have the right permissions:

>>> import os, stat
>>> print("0%o" % stat.S_IMODE(os.stat('foo').st_mode))
0205

Note that Buildout will not allow you to have write permission for others and will silently remove it (i.e a 207 mode will become 205).

Fetching resources from URLs

You can specify the resource to fetch from an URL:

>>> server_data = tmpdir('server_data')
>>> server_url = start_server(server_data)

>>> write(server_data, 'foo.in', '{{bar}}')
>>> write('buildout.cfg',
... '''
... [buildout]
... parts = template
...
... [template]
... recipe = slapos.recipe.template:jinja2
... template = %sfoo.in
... rendered = foo
... context = key bar buildout:parts
... md5sum = %s
... ''' % (server_url, md5sum))
>>> run_buildout()
Uninstalling template.
Installing template.
Downloading http://localhost/foo.in
Cannot download http://localhost/foo.in from network cache.

>>> cat('foo')
template

Template imports

Here is a simple template importing an equaly-simple library:

>>> write('template.in', '''
... {%- import "library" as library -%}
... {{ library.foo() }}
... ''')
>>> write('library.in', '{% macro foo() %}FOO !{% endmacro %}')

To import a template from rendered template, you need to specify what can be imported:

>>> write('buildout.cfg', '''
... [buildout]
... parts = template
...
... [template]
... recipe = slapos.recipe.template:jinja2
... template = template.in
... rendered = bar
... import-list = rawfile library library.in
... ''')
>>> run_buildout()
Uninstalling template.
Installing template.
>>> cat('bar')
FOO !

Just like context definition, it also works with indirect values:

>>> write('buildout.cfg', '''
... [buildout]
... parts = template
...
... [template-library]
... path = library.in
...
... [template]
... recipe = slapos.recipe.template:jinja2
... template = template.in
... rendered = bar
... import-list = file library template-library:path
... ''')
>>> run_buildout()
Uninstalling template.
Installing template.
>>> cat('bar')
FOO !

This also works to allow importing from identically-named files in different directories:

>>> write('template.in', '''
... {%- import "dir_a/1.in" as a1 -%}
... {%- import "dir_a/2.in" as a2 -%}
... {%- import "dir_b/1.in" as b1 -%}
... {%- import "dir_b/c/1.in" as bc1 -%}
... {{ a1.foo() }}
... {{ a2.foo() }}
... {{ b1.foo() }}
... {{ bc1.foo() }}
... ''')
>>> mkdir('a')
>>> mkdir('b')
>>> mkdir(join('b', 'c'))
>>> write(join('a', '1.in'), '{% macro foo() %}a1foo{% endmacro %}')
>>> write(join('a', '2.in'), '{% macro foo() %}a2foo{% endmacro %}')
>>> write(join('b', '1.in'), '{% macro foo() %}b1foo{% endmacro %}')
>>> write(join('b', 'c', '1.in'), '{% macro foo() %}bc1foo{% endmacro %}')

All templates can be accessed inside both folders:

>>> write('buildout.cfg', '''
... [buildout]
... parts = template
...
... [template-library]
... path = library.in
...
... [template]
... recipe = slapos.recipe.template:jinja2
... template = template.in
... rendered = bar
... import-list =
...     rawfolder dir_a a
...     rawfolder dir_b b
... ''')
>>> run_buildout()
Uninstalling template.
Installing template.
>>> cat('bar')
a1foo
a2foo
b1foo
bc1foo

It is possible to override default path delimiter (without any effect on final path):

>>> write('template.in', r'''
... {%- import "dir_a\\1.in" as a1 -%}
... {%- import "dir_a\\2.in" as a2 -%}
... {%- import "dir_b\\1.in" as b1 -%}
... {%- import "dir_b\\c\\1.in" as bc1 -%}
... {{ a1.foo() }}
... {{ a2.foo() }}
... {{ b1.foo() }}
... {{ bc1.foo() }}
... ''')
>>> write('buildout.cfg', r'''
... [buildout]
... parts = template
...
... [template-library]
... path = library.in
...
... [template]
... recipe = slapos.recipe.template:jinja2
... template = template.in
... rendered = bar
... import-delimiter = \
... import-list =
...     rawfolder dir_a a
...     rawfolder dir_b b
... ''')
>>> run_buildout()
Uninstalling template.
Installing template.
>>> cat('bar')
a1foo
a2foo
b1foo
bc1foo

Section dependency

You can use other part of buildout in the template. This way this parts will be installed as dependency:

>>> write('buildout.cfg', '''
... [buildout]
... parts = template
...
... [template]
... recipe = slapos.recipe.template:jinja2
... template = inline:{{bar}}
... rendered = foo
... context = key bar dependency:foobar
...
... [dependency]
... foobar = dependency content
... recipe = zc.buildout:debug
... ''')

>>> run_buildout()
Uninstalling template.
Installing dependency.
  foobar='dependency content'
  recipe='zc.buildout:debug'
Installing template.

This way you can get options which are computed in the __init__ of the dependent recipe.

Let’s create a sample recipe modifying its option dict:

>>> write('setup.py',
... '''
... from setuptools import setup
...
... setup(name='samplerecipe',
...       entry_points = {
...           'zc.buildout': [
...                'default = main:Recipe',
...           ],
...       }
...      )
... ''')
>>> write('main.py',
... '''
... class Recipe(object):
...
...     def __init__(self, buildout, name, options):
...         options['data'] = 'foobar'
...
...     def install(self):
...         return []
... ''')

Let’s just use buildout.cfg using this egg:

>>> write('buildout.cfg',
... '''
... [buildout]
... develop = .
... parts = template
...
... [template]
... recipe = slapos.recipe.template:jinja2
... template = inline:
...   {{bar}}
... rendered = foo
... context = key bar sample:data
...
... [sample]
... recipe = samplerecipe
... ''')
>>> run_buildout()
Develop: '/sample-buildout/.'
Uninstalling template.
Uninstalling dependency.
Installing sample.
Installing template.
>>> cat('foo')
foobar

Avoiding file re-creation

Normally, each time the section is installed/updated the file gets re-generated. This may be undesirable in some cases.

once allows specifying a marker file, which when present prevents template rendering.

>>> import os
>>> write('buildout.cfg',
... '''
... [buildout]
... parts = template
...
... [template]
... recipe = slapos.recipe.template:jinja2
... template = inline:dummy
... rendered = foo_once
... once = foo_flag
... ''')
>>> run_buildout() # doctest: +ELLIPSIS
Uninstalling template.
Uninstalling sample.
Getting distribution for 'samplerecipe'.
Got samplerecipe 0.0.0.
Installing template.
The template install returned None.  A path or iterable os paths should be returned.
warning: install_lib: '...' does not exist -- no Python modules to install
<BLANKLINE>
zip_safe flag not set; analyzing archive contents...

Template was rendered:

>>> cat('foo_once')
dummy

And canary exists:

>>> os.path.exists('foo_flag')
True

Remove rendered file and re-render:

>>> import os
>>> os.unlink('foo_once')
>>> with open('buildout.cfg', 'a') as f:
...     f.writelines(['extra = useless'])
>>> run_buildout()
Uninstalling template.
Installing template.
The template install returned None.  A path or iterable os paths should be returned.
Unused options for template: 'extra'.

Template was not rendered:

>>> os.path.exists('foo_once')
False

Removing the canary allows template to be re-rendered:

>>> os.unlink('foo_flag')
>>> with open('buildout.cfg', 'a') as f:
...     f.writelines(['moreextra = still useless'])
>>> run_buildout()
Uninstalling template.
Installing template.
The template install returned None.  A path or iterable os paths should be returned.
Unused options for template: 'extra'.
>>> cat('foo_once')
dummy

It’s also possible to use the same file for rendered and once:

>>> write('buildout.cfg',
... '''
... [buildout]
... parts = template
...
... [template]
... recipe = slapos.recipe.template:jinja2
... template = inline:initial content
... rendered = rendered
... once = ${:rendered}
... ''')
>>> run_buildout() # doctest: +ELLIPSIS
Uninstalling template.
Installing template.
The template install returned None.  A path or iterable os paths should be returned.

Template was rendered:

>>> cat('rendered')
initial content

When buildout options are modified, the template will not be rendered again:

>>> with open('buildout.cfg', 'a') as f:
...     f.writelines(['template = inline:something different'])

>>> run_buildout()
Uninstalling template.
Installing template.
The template install returned None.  A path or iterable os paths should be returned.

Even though we used a different template, the file still contain the first template:

>>> cat('rendered')
initial content

4.6 (2021-06-08)

Fix leaking temporary files with templates from URLs

4.5 (2020-01-08)

jinja2: Prevent ‘once’ from overwriting ‘rendered’

4.4 (2019-01-24)

jinja2: add bytes and six

4.3 (2018-01-25)

jinja2: Compile the same source only once and use compiled source next time.

4.2 (2017-12-12)

jinja2: try to not rewrite on update if there’s no change

4.1 (2017-10-18)

Fix $$ escaping in basic templates.

4.0 (2017-10-13)

jinja2: read template at install/update and fix ‘mode’ option
Add support for Python 3.

3.0 (2017-05-23)

jinja2: make ‘import’ return the leaf module instead of the root one.

2.10 (2017-01-18)

jinja2: Add support for render-once.

2.9 (2015-11-18)

jinja2: Add support for non-ascii templates. Encoding for input/output and imported files can be set via new “encoding” parameter which defaults to utf-8.

2.8 (2015-06-25)

jinja2: new assert function.

2.7 (2015-05-18)

jinja2: fix display of source in traceback when there is an error in the root template (or in instance parameters).

2.6 (2014-11-26)

jinja2: add many built-in functions from Python.

2.5 (2013-08-07)

Fix file import with Jinja2 >= 2.7

2.4.3 (2013-08-02)

jinja2: add support for inline templates.

2.4.2 (2012-08-21)

jinja2: Mode shall be used instead of umask. [Vincent Pelletier]

jinja2: Add jinja2 “import” directive support. [Vincent Pelletier, Timothee Lacroix]

Added rawfile and rawfolder types. [Vincent Pelletier, Timothee Lacroix]

Reworked loader classes [Vincent Pelletier]

2.4.1 (2012-08-01)

jinja2: Make “context” parameter really optional. [Vincent Pelletier]

2.4 (2012-06-01)

Provide access to zc.buildout.buildout.dumps when it exists. [Vincent Pelletier]

Fix missing jinja2 entry point documentation in pacakge description [Vincent Pelletier]

2.3 (2012-03-29)

Add jinja2 entry point with jinja2 template support. [Vincent Pelletier]

2.2 (2011-10-12)

Include missing files in package. [Łukasz Nowak]

2.1 (2011-10-12)

Description update. [Łukasz Nowak]

2.0 (2011-10-12)

Dropping collective.recipe.template dependency. [Romain Courteaud, Antoine Catton]

1.1 (2011-05-30)

Moved out from slapos.cookbook in order to minimise depenencies [Łukasz Nowak]

Project details

Release history Release notifications | RSS feed

5.1

Oct 24, 2022

5.0

Feb 3, 2022

This version

4.6

Jun 9, 2021

4.5

Jan 8, 2021

4.4

Jan 27, 2020

4.3

Jan 25, 2018

4.2

Dec 11, 2017

4.1

Oct 18, 2017

4.0

Oct 13, 2017

3.0

Jun 3, 2017

2.10

Jan 18, 2017

2.9

Nov 18, 2015

2.8

Jun 25, 2015

2.7

May 19, 2015

2.6

Dec 8, 2014

2.5

Aug 7, 2013

2.4.3

Aug 2, 2013

2.4.2

Aug 21, 2012

2.4.1

Aug 1, 2012

2.4

Jun 1, 2012

2.3

Mar 29, 2012

2.2

Oct 12, 2011

2.1

Oct 12, 2011

2.0

Oct 12, 2011

1.1

May 30, 2011

1.0.dev-r4150 pre-release

Feb 8, 2011

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

slapos.recipe.template-4.6.tar.gz (22.6 kB view hashes)

Uploaded Jun 9, 2021 Source

Hashes for slapos.recipe.template-4.6.tar.gz

Hashes for slapos.recipe.template-4.6.tar.gz
Algorithm	Hash digest
SHA256	`37f7a6a80db48c48c2ff7ff19cec41885b1fb12500725c32d6716c2a5f82eb04`
MD5	`4ca8ec83235db10413eee2e89f03ada2`
BLAKE2b-256	`816be68aede63ebc6897d2e68ae699518fe7f297f7971e573e5bd14c4910ac75`

slapos.recipe.template 4.6

Navigation

Verified details

Maintainers

Unverified details

Meta

Classifiers

Project description

Usage

Getting started

Full options

Check file integrity

Specify filesystem permissions

Fetching template source from an URL

Section dependency

Jinja2 usage

Getting started

Parameters

FAQ

Use jinja2 extensions

Check file integrity

Specify filesystem permissions

Fetching resources from URLs

Template imports

Section dependency

Avoiding file re-creation

4.6 (2021-06-08)

4.5 (2020-01-08)

4.4 (2019-01-24)

4.3 (2018-01-25)

4.2 (2017-12-12)

4.1 (2017-10-18)

4.0 (2017-10-13)

3.0 (2017-05-23)

2.10 (2017-01-18)

2.9 (2015-11-18)

2.8 (2015-06-25)

2.7 (2015-05-18)

2.6 (2014-11-26)

2.5 (2013-08-07)

2.4.3 (2013-08-02)

2.4.2 (2012-08-21)

2.4.1 (2012-08-01)

2.4 (2012-06-01)

2.3 (2012-03-29)

2.2 (2011-10-12)

2.1 (2011-10-12)

2.0 (2011-10-12)

1.1 (2011-05-30)

Project details

Verified details

Maintainers

Unverified details

Meta

Classifiers

Release history Release notifications | RSS feed

Download files

Source Distribution