i18ndude 4.2

Introduction

i18ndude performs various tasks related to ZPT’s, Python Scripts and i18n.

Compatibility

i18ndude works with Python 2.6 and 2.7. It is expected to work with Python 2.4 and 2.5 too, but it may be tricky to get the proper versions of its dependencies that work with those Python versions.

i18ndude uses a few zope packages. If you install i18ndude using a buildout, you may want to pin their versions to the ones in the current latest ztk (Zope Tool Kit) versions, as that is what we test with: http://download.zope.org/zopetoolkit/index/1.1.6/ztk-versions.cfg

If you use a separate buildout for i18ndude, you may want to update the zope.tal version to 4.0.0 or higher, with fixes a possible UnicodeDecodeError while printing a warning when a msgid already exists. If you happen to hit that corner case, the new version should help.

If you add i18ndude to a buildout that handles a Zope, CMF or Plone site, you probably already have versions pinned so then you do not need to add an extends line.

You can also use pip or easy_install to install it. With pip you may want to create a requirements.txt based on that ztk versions list.

i18ndude

usage: i18ndude [-h]
                {find-untranslated,rebuild-pot,merge,sync,filter,admix,list,trmerge}
                ...

i18ndude performs various tasks related to ZPT's, Python Scripts
and i18n.

Its main task is to extract translation strings (msgids) into a
.pot file (with the 'rebuild-pot' command), and sync the .pot file
with .po files (with the 'sync' command).

Call i18ndude with one of the listed subcommands followed by
--help to get help for that subcommand.

optional arguments:
  -h, --help            show this help message and exit

subcommands:
  {find-untranslated,rebuild-pot,merge,sync,filter,admix,list,trmerge}

find-untranslated

usage: i18ndude find-untranslated [-h] [-s] [-n] [files [files ...]]

    Provide a list of ZPT filenames and I will output a report of places
    where I suspect untranslated messages, i.e. tags for which
    "i18n:translate" or "i18n:attributes" are missing.

    If you provide the -s option, the report will only contain a summary
    of errors and warnings for each file (or no output if there are no
    errors or warnings). If you provide the -n option, the report will
    contain only the errors for each file.

    You can mark tags to be ignored for this translation check by
    setting the "i18n:ignore" attribute on the tag. Same for
    attributes with "i18n:ignore-attributes". Note that i18ndude may
    be happy with this, but your template engine may fail when trying
    to render a template containing those ignore hints.  You need
    Chameleon 2.23 or higher, or the to be released zope.tal 4.1.2.


positional arguments:
  files            list of ZPT filenames

optional arguments:
  -h, --help       show this help message and exit
  -s, --silent     The report will only contain a summary of errors and
                   warnings for each file (or no output if there are no errors
                   or warnings).
  -n, --nosummary  The report will contain only the errors for each file.

rebuild-pot

usage: i18ndude rebuild-pot [-h] [--wrap | --no-wrap] [--width NUMBER] -p
                            filename [-c domain] [-m filename]
                            [--merge2 filename]
                            [--exclude "<ignore1> <ignore2> ..."]
                            [path [path ...]]

    Given a pot-file via the --pot option you can specify one or more
    directories which including all sub-folders will be searched for
    PageTemplates (*.*pt) and Python scripts (*.*py).

    Make sure you have a backup copy of the original pot-file in case
    you need to fill back in ids by hand.

    If you specify a domain in --create I will create the pot file and
    look for messages for that domain.  Otherwise I will take the
    domain from the Domain header in the given pot file and keep the
    headers from the file as base for a new pot file.

    If you give me an additional pot-file with the --merge <filename>
    option, I try to merge these msgids into the target-pot file
    afterwards. If a msgid already exists in the ones I found in the
    ZPTs, I'll warn you and ignore that msgid. I take the mime-header
    from this additional pot-file. If you provide a second pot-file via
    --merge2 <filename> I'll merge this into the first merge's result

    You can also provide a list of filenames (or regular expressions for
    filenames) which should not be included by using the --exclude argument,
    which takes a whitespace delimited list of files (or regular expressions
    for files).


positional arguments:
  path

optional arguments:
  -h, --help            show this help message and exit
  --wrap                Wrap long lines.
  --no-wrap             Do not wrap long lines. This is the default.
  --width NUMBER        Set output page width. Default is 79.
  -p filename, --pot filename
  -c domain, --create domain
  -m filename, --merge filename
  --merge2 filename
  --exclude "<ignore1> <ignore2> ..."

merge

usage: i18ndude merge [-h] [--wrap | --no-wrap] [--width NUMBER] -p filename
                      -m filename [--merge2 filename]

    Given a pot-file via the --pot option and a second
    pot-file with the --merge <filename> option, I try to merge
    these msgids into the target-pot file. If a msgid already
    exists, I'll warn you and ignore that msgid.

    If you provide a --merge2 <filename> I'll first merge this one
    in addition to the first one.


optional arguments:
  -h, --help            show this help message and exit
  --wrap                Wrap long lines.
  --no-wrap             Do not wrap long lines. This is the default.
  --width NUMBER        Set output page width. Default is 79.
  -p filename, --pot filename
  -m filename, --merge filename
  --merge2 filename

sync

usage: i18ndude sync [-h] [--wrap | --no-wrap] [--width NUMBER] -p potfilename
                     pofilename [pofilename ...]

    Given a pot-file with the --pot option and a list of po-files I'll
    remove from the po files those message translations of which the
    msgids are not in the pot-file and add messages that the pot-file has
    but the po-file doesn't.


positional arguments:
  pofilename

optional arguments:
  -h, --help            show this help message and exit
  --wrap                Wrap long lines.
  --no-wrap             Do not wrap long lines. This is the default.
  --width NUMBER        Set output page width. Default is 79.
  -p potfilename, --pot potfilename

filter

usage: i18ndude filter [-h] [--wrap | --no-wrap] [--width NUMBER] file1 file2

    Given two pot-files I will write a copy of file1 to stdout with all
    messages removed that are also in file2, i.e. where msgids match.


positional arguments:
  file1
  file2

optional arguments:
  -h, --help      show this help message and exit
  --wrap          Wrap long lines.
  --no-wrap       Do not wrap long lines. This is the default.
  --width NUMBER  Set output page width. Default is 79.

admix

usage: i18ndude admix [-h] [--wrap | --no-wrap] [--width NUMBER] file1 file2

    Given two po-files I will look for translated entries in file2 that
    are untranslated in file1. I add these translations (msgstrs) to
    file1. Note that this will not affect the number of entries in file1.
    The result will be on stdout.


positional arguments:
  file1
  file2

optional arguments:
  -h, --help      show this help message and exit
  --wrap          Wrap long lines.
  --no-wrap       Do not wrap long lines. This is the default.
  --width NUMBER  Set output page width. Default is 79.

list

usage: i18ndude list [-h] -p product [product ...] [-t]

    This will create a simple listing that displays how much of the
    combined products pot's is translated for each language. Run this
    from the directory containing the pot-files. The product name is
    normally a domain name.


optional arguments:
  -h, --help            show this help message and exit
  -p product [product ...], --products product [product ...]
  -t, --table

trmerge

usage: i18ndude trmerge [-h] [--wrap | --no-wrap] [--width NUMBER] [-i]
                        [--no-override]
                        file1 file2

    Given two po-files I will update all translations from file2 into
    file1. Missing translations are added.

    If a translation was fuzzy in file1, and there is a nonempty translation
    in file2, the fuzzy marker is removed.

    Fuzzy translations in file2 are ignored.

    The result will be on stdout.  If you want to update the first
    file in place, use a temporary file, something like this:

      i18ndude trmerge file1.po file2.po > tmp_merge && mv tmp_merge file1.po


positional arguments:
  file1
  file2

optional arguments:
  -h, --help          show this help message and exit
  --wrap              Wrap long lines.
  --no-wrap           Do not wrap long lines. This is the default.
  --width NUMBER      Set output page width. Default is 79.
  -i, --ignore-extra  Ignore extra messages: do not add msgids that are not in
                      the original po-file. Only update translations for
                      existing msgids.
  --no-override       Do not override translations, only add missing
                      translations.

4.2 (2017-06-21)

New:

• In find-untranslated, do not report items that get replaced by Chameleon syntax. So <span>${view/test}</span> will no longer get flagged as missing a translation. (Note that you still can add i18n:translate if it makes sense, like Plone does for translating the dynamically calculated review state.) [Netroxen, maurits]

• Find untranslated attributes now also checks for ‘placeholder’ attributes on input tags.

4.1 (2016-12-02)

New:

• Allow use of regular expressions for –exclude parameter. For example, use *.py to exclude all python files. This doesn’t break existing behavior. Do remember to use quotes around the expression. [laulaz, maurits]

4.0.1 (2015-12-03)

Fixes:

• Fixed some reported line numbers in find-untranslated. Fixes issue #34. [maurits]

4.0.0 (2015-12-03)

New:

• Extract strings from zcml. Issue #28 [maurits]

• No longer print two blank lines at the end of .po and .pot files. [maurits]

• In the find-untranslated command, first try to parse a template as xml, which is good for non-html files. If that fails, try to parse it as html with a little help from the lxml HTMLPaser, which handles html5 code much better. If that fails, use our trusty home grown common.prepare_xml function, which treats everything as old html. Note that we still use xml.sax as the core parser here. Issue #15 [maurits]

• Ignore hidden files in the find-untranslated command. Issue #29 [maurits]

• Use lxml instead of xml.etree or elementtree for parsing GenericSetup xml files. [maurits]

3.4.5 (2015-11-05)

New:

• First try the original zope.tal parser. Only when this fails we try our own parser/generator. [maurits]

• Support Chameleon unnamed attributes without crashing. For example: tal:attributes="python:{'data-something': 'chameleon-only'}" [maurits]

• Support chameleon attributes tal:switch and tal:case. Fixes issue #24. [ale-rt]

3.4.4 (2015-11-04)

Fixes:

• Check tal:condition correctly when it is in a tal:something tag. [maurits]

• In find-untranslated only ignore tal:condition="nothing", not other conditions. Fixes issue #16. [maurits]

• Improved the prepare_xml function. This tries to work around templates that miss the usual boiler plate, like xmlns:i18n="http://xml.zope.org/namespaces/i18n". But there were some silly errors in it. This refs issue #16. [maurits]

3.4.3 (2015-09-01)

• Fix nosummary option from find-untranslated. It was reporting wrong information. [gforcada]

3.4.2 (2015-07-16)

• Fix encoding errors with wrapAndQuoteString. [thet]

• Pep8. [thet]

3.4.1 (2015-06-25)

• Releasing as Python wheel too. [maurits]

• Fixed wrapping when string contains newline. Issue #13 [maurits]

3.4.0 (2014-11-27)

• Drop Python 2.6 support. It may still work, but the tests only run on Python 2.7. Note that it is fine to use one central i18ndude command for all your projects, no matter what Python version they are using. [janjaapdriessen, maurits]

• For the find-untranslated feature, add the possibility to mark a tag to be ignored by setting the “i18n:ignore” attribute on the tag. Also works for attributes with the “i18n:ignore-attributes” attribute. [janjaapdriessen]

3.3.5 (2014-08-05)

• Avoid AttributeError: ‘NoneType’ object has no attribute ‘comments’ when a .po file is missing an empty msgid and msgstr near the top. This is fixed automatically, although it will override some headers. [maurits]

3.3.4 (2014-07-02)

• Wrap first line correctly. Fixes #9. [gforcada]

3.3.3 (2013-11-27)

• Package housekeeping. [hvelarde]

3.3.2 (2013-11-27)

• trmerge: do not override when the mixin translation is fuzzy. [maurits]

• trmerge: add --no-override argument. This means: do not override translations, only add missing translations. [maurits]

• trmerge: add --ignore-extra option. This ignores extra msgids in the second po-file. [maurits]

3.3.1 (2013-10-18)

• Update script.py in some cases “arguments.exclude” is None. [giacomos]

• Fixed optional parameter exclude. [shylux]

3.3.0 (2013-10-13)

• Add command line documentation to long description of package. [maurits]

• Add options --wrap, --no-wrap and --width=NUMBER to all commands that write files. Use these to determine whether long lines are wrapped and at which width. Default width is 79. By default we do NOT wrap, because we have never wrapped before. This may change in the future, so if you really want to be sure to not wrap when using a future i18ndude version, you can add --no-wrap now. https://github.com/collective/i18ndude/issues/3 [maurits]

• Fix the list command to also work in a locales structure. [maurits]

• Fix an error in the merge command where the --merge option would be used as value for the --merge2 option as well, if that option itself was unused. This led to unneeded warnings. [maurits]

• The --create domain option of rebuild-pot is now optional. If not given, i18ndude reads the domain from the given .pot file. It was always optional, but the documentation did not show it and it did not work. [maurits]

• Update the command line options handling. You can now get the help for individual commands by calling them with the --help option. [maurits]

• Return exit code 1 when the called function gives an error. This currently only has an effect when calling find-untranslated. https://github.com/collective/i18ndude/issues/1 [maurits]

• Moved code to https://github.com/collective/i18ndude [maurits]

• Backslash escape added to msgid when it includes double quotes. [taito]

• Add trmerge command to merge po files. Custom tailored for transifex. [do3cc]

3.2.2 (2010-12-11)

• Encode key to utf-8 for the Merge-Warning message to avoid a UnicodeEncodeError. [mikerhodes]

3.2.1 (2010-10-31)

• Fixed making POT file for DOUBLE BYTE strings on default. [terapyon]

3.2 (2010-09-04)

• Replaced internal odict implementation by the ordereddict package. (implementation backported from Python 2.7) [vincentfretin]

3.1.3 (2010-09-04)

• Avoid UnicodeDecodeError when printing warning message in add(). [rnix]

3.1.2 (2010-02-14)

• elementtree is only required for Python < 2.5. [vincentfretin]

• Fixed tests (patch provided by John Trammell). [vincentfretin]

3.1.1 (2009-11-22)

• Strip “src” only once in the pathname for the comments. Example: before it generated the following comment “#: archetypes.referencebrowserwidget/” which was not so useful. Now it generates “archetypes.referencebrowserwidget/src/archetypes/referencebrowserwidget/…” [vincentfretin]

3.1 (2009-10-31)

• Support for explicit msgids in GSReader. [vincentfretin]

• Better handling of msgid references. Keep all the references in PTReader and PYReader. In POWriter, normalize and sort the references, write only MAX_OCCUR (default is 3) references. You can set MAX_OCCUR=None if you want all references to be written to the generated POT file. Only the first reference is written in case of several references to the same file but with different line number. [vincentfretin]

• Depend now on zope.tal 3.5.2 to print a warning when msgid already exists in catalog with a different default message. Simplified PTReader code. Check for msgid with different default in GSReader, PYReader and in the merged catalog (ptctl, pyctl, gsctl). [vincentfretin]

• Fix behaviour when dealing with broken xml files to be parsed. [afd]

3.0 (2008-11-13)

• No changes. [hannosch]

For older changes, see docs/ChangeLog.