darker 1.7.1

What?

This utility reformats and checks Python source code files. However, when run in a Git repository, it compares an old revision of the source tree to a newer revision (or the working tree). It then

• only applies reformatting in regions which have changed in the Git working tree between the two revisions, and

• only reports those linting messages which appeared after the modifications to the source code files.

The reformatters supported are:

• Black for code reformatting

• isort for sorting imports

• flynt for turning old-style format strings to f-strings

See Using linters below for the list of supported linters.

To easily run Darker as a Pytest plugin, see pytest-darker.

To integrate Darker with your IDE or with pre-commit, see the relevant sections below in this document.

Why?

You want to start unifying code style in your project using Black. Maybe you also like to standardize on how to order your imports, or do static type checking or other static analysis for your code.

However, instead of formatting the whole code base in one giant commit, you’d like to only change formatting when you’re touching the code for other reasons.

This can also be useful when contributing to upstream codebases that are not under your complete control.

Partial formatting is not supported by Black itself, for various good reasons, and so far there hasn’t been a plan to implemented it either (134, 142, 245, 370, 511, 830). However, in September 2021 Black developers started to hint towards adding this feature after all (1352). This might at least simplify Darker’s algorithm substantially.

But for the time being, this is where darker enters the stage. This tool is for those who want to do partial formatting right now.

Note that this tool is meant for special situations when dealing with existing code bases. You should just use Black and isort as is when starting a project from scratch.

You may also want to still consider whether reformatting the whole code base in one commit would make sense in your particular case. You can ignore a reformatting commit in git blame using the blame.ignoreRevsFile config option or --ignore-rev on the command line. For a deeper dive into this topic, see Avoiding ruining git blame in Black documentation, or the article Why does Black insist on reformatting my entire project? from Łukasz Langa (@ambv, the creator of Black). Here’s an excerpt:

“When you make this single reformatting commit, everything that comes after is semantic changes so your commit history is clean in the sense that it actually shows what changed in terms of meaning, not style. There are tools like darker that allow you to only reformat lines that were touched since the last commit. However, by doing that you forever expose yourself to commits that are a mix of semantic changes with stylistic changes, making it much harder to see what changed.”

How?

To install or upgrade, use:

pip install --upgrade darker~=1.7.1

Or, if you’re using Conda for package management:

conda install -c conda-forge darker~=1.7.1 isort
conda update -c conda-forge darker

Note: It is recommended to use the ‘~=’ “compatible release” version specifier for Darker. See Guarding against Black compatibility breakage for more information.

The darker <myfile.py> or darker <directory> command reads the original file(s), formats them using Black, combines original and formatted regions based on edits, and writes back over the original file(s).

Alternatively, you can invoke the module directly through the python executable, which may be preferable depending on your setup. Use python -m darker instead of darker in that case.

By default, darker just runs Black to reformat the code. You can enable additional features with command line options:

• -i / --isort: Reorder imports using isort. Note that isort must be run in the same Python environment as the packages to process, as it imports your modules to determine whether they are first or third party modules.

• -f / --flynt: Also convert string formatting to use f-strings using the flynt package

• -L <linter> / --lint <linter>: Run a supported linter (see Using linters)

New in version 1.1.0: The -L / --lint option. New in version 1.2.2: Package available in conda-forge. New in version 1.7.0: The -f / --flynt option

Example

This example walks you through a minimal practical use case for Darker.

First, create an empty Git repository:

$ mkdir /tmp/test
$ cd /tmp/test
$ git init
Initialized empty Git repository in /tmp/test/.git/

In the root of that directory, create the ill-formatted Python file our_file.py:

if True: print('hi')
print()
if False: print('there')

Commit that file:

$ git add our_file.py
$ git commit -m "Initial commit"
[master (root-commit) a0c7c32] Initial commit
 1 file changed, 3 insertions(+)
 create mode 100644 our_file.py

Now modify the first line in that file:

if True: print('CHANGED TEXT')
print()
if False: print('there')

You can ask Darker to show the diff for minimal reformatting which makes edited lines conform to Black rules:

$ darker --diff our_file.py
--- our_file.py
+++ our_file.py
@@ -1,3 +1,4 @@
-if True: print('CHANGED TEXT')
+if True:
+    print("CHANGED TEXT")
print()
if False: print('there')

Alternatively, Darker can output the full reformatted file (works only when a single Python file is provided on the command line):

$ darker --stdout our_file.py

if True:
    print("CHANGED TEXT")
print()
if False: print('there')

If you omit the --diff and --stdout options, Darker replaces the files listed on the command line with partially reformatted ones as shown above:

$ darker our_file.py

Now the contents of our_file.py will have changed. Note that the original print() and if False: ... lines have not been reformatted since they had not been edited!

if True:
    print("CHANGED TEXT")
print()
if False: print('there')

You can also ask Darker to reformat edited lines in all Python files in the repository:

$ darker .

Or, if you want to compare to another branch (or, in fact, any commit) instead of the last commit:

$ darker --revision master .

Customizing darker, Black, isort, flynt and linter behavior

darker invokes Black and isort internals directly instead of running their binaries, so it needs to read and pass configuration options to them explicitly. Project-specific default options for darker itself, Black and isort are read from the project’s pyproject.toml file in the repository root. isort does also look for a few other places for configuration.

Mypy, Pylint, Flake8 and other compatible linters are invoked as subprocesses by darker, so normal configuration mechanisms apply for each of those tools. Linters can also be configured on the command line, for example:

darker -L "mypy --strict" .
darker --lint "pylint --errors-only" .

flynt (option -f / --flynt) is also invoked as a subprocess, but passing command line options to it is currently not supported. Configuration files need to be used instead.

Darker does honor exclusion options in Black configuration files when recursing directories, but the exclusions are only applied to Black reformatting. Isort and linters are still run on excluded files. Also, individual files explicitly listed on the command line are still reformatted even if they match exclusion patterns.

For more details, see:

• Black documentation about pyproject.toml

• isort documentation about config files

• public GitHub repositories which install and run Darker

• flynt documentation about configuration files

The following command line arguments can also be used to modify the defaults:

-r REV, --revision REV

Revisions to compare. The default is HEAD..:WORKTREE: which compares the latest commit to the working tree. Tags, branch names, commit hashes, and other expressions like HEAD~5 work here. Also a range like main...HEAD or main... can be used to compare the best common ancestor. With the magic value :PRE-COMMIT:, Darker works in pre-commit compatible mode. Darker expects the revision range from the PRE_COMMIT_FROM_REF and PRE_COMMIT_TO_REF environment variables. If those are not found, Darker works against HEAD. Also see --stdin-filename= for the :STDIN: special value.

--diff

Don’t write the files back, just output a diff for each file on stdout. Highlight syntax if on a terminal and the pygments package is available, or if enabled by configuration.

-d, --stdout

Force complete reformatted output to stdout, instead of in-place. Only valid if there’s just one file to reformat. Highlight syntax if on a terminal and the pygments package is available, or if enabled by configuration.

--stdin-filename PATH

The path to the file when passing it through stdin. Useful so Darker can find the previous version from Git. Only valid with --revision=<rev1>..:STDIN: (HEAD..:STDIN: being the default if --stdin-filename is enabled).

--check

Don’t write the files back, just return the status. Return code 0 means nothing would change. Return code 1 means some files would be reformatted.

-f, --flynt

Also convert string formatting to use f-strings using the flynt package

-i, --isort

Also sort imports using the isort package

-L CMD, --lint CMD

Also run a linter on changed files. CMD can be a name or path of the linter binary, or a full quoted command line with the command and options. Linters read their configuration as normally, and aren’t affected by -c / --config. Linter output is syntax highlighted when the pygments package is available if run on a terminal and or enabled by explicitly (see --color).

-c PATH, --config PATH

Ask black and isort to read configuration from PATH. Note that other tools like flynt, Mypy, Pylint and Flake8 won’t use this configuration file.

-v, --verbose

Show steps taken and summarize modifications

-q, --quiet

Reduce amount of output

--color

Enable syntax highlighting even for non-terminal output. Overrides the environment variable PY_COLORS=0

--no-color

Disable syntax highlighting even for terminal output. Overrides the environment variable PY_COLORS=1

-S, --skip-string-normalization

Don’t normalize string quotes or prefixes

--no-skip-string-normalization

Normalize string quotes or prefixes. This can be used to override skip_string_normalization = true from a configuration file.

--skip-magic-trailing-comma

Skip adding trailing commas to expressions that are split by comma where each element is on its own line. This includes function signatures. This can be used to override skip_magic_trailing_comma = true from a configuration file.

-l LENGTH, --line-length LENGTH

How many characters per line to allow [default: 88]

-t VERSION, --target-version VERSION

[py33|py34|py35|py36|py37|py38|py39|py310|py311] Python versions that should be supported by Black’s output. [default: per-file auto-detection]

-W WORKERS, --workers WORKERS

How many parallel workers to allow, or 0 for one per core [default: 1]

To change default values for these options for a given project, add a [tool.darker] or [tool.black] section to pyproject.toml in the project’s root directory, or to a different TOML file specified using the -c / --config option. For example:

[tool.darker]
src = [
    "src/mypackage",
]
revision = "master"
diff = true
check = true
isort = true
flynt = true
lint = [
    "pylint",
]
line-length = 80                  # Passed to isort and Black, override their config
log_level = "INFO"

[tool.black]
line-length = 88                  # Overridden by [tool.darker] above
skip-magic-trailing-comma = false
skip-string-normalization = false
target-version = ['py311']
exclude = "test_*\.py"
extend_exclude = "/generated/"
force_exclude = ".*\.pyi"

[tool.isort]
profile = "black"
known_third_party = ["pytest"]
line_length = 88                  # Overridden by [tool.darker] above

While isort reads all of its options from the configuration file, Black only honors the ones listed above when called by darker. Other tools are invoked as subprocesses and use their configuration mechanisms unmodified.

Be careful to not use options which generate output which is unexpected for other tools. For example, VSCode only expects the reformat diff, so lint = [ ... ] can’t be used with it.

New in version 1.0.0:

• The -c, -S and -l command line options.

• isort is configured with -c and -l, too.

New in version 1.1.0: The command line options

• -r / --revision

• --diff

• --check

• --no-skip-string-normalization

• -L / --lint

New in version 1.2.0: Support for

• commit ranges in -r / --revision.

• a [tool.darker] section in pyproject.toml.

New in version 1.2.2: Support for -r :PRE-COMMIT: / --revision=:PRE_COMMIT:

New in version 1.3.0: The --skip-magic-trailing-comma and -d / --stdout command line options

New in version 1.5.0: The -W / --workers, --color and --no-color command line options

New in version 1.7.0: The -t / --target-version command line option

New in version 1.7.0: The -f / --flynt command line option

Editor integration

Many editors have plugins or recipes for integrating Black. You may be able to adapt them to be used with darker. See editor integration in the Black documentation.

set autoread
autocmd BufWritePost *.py silent :!darker %

(use-package reformatter
  :hook ((python-mode . darker-reformat-on-save-mode))
  :config
  (reformatter-define darker-reformat
    :program "darker"
    :stdin nil
    :stdout nil
    :args (list "-q" input-file))

Using as a pre-commit hook

New in version 1.2.1

To use Darker locally as a Git pre-commit hook for a Python project, do the following:

1. Install pre-commit in your environment (see pre-commit Installation for details).

2. Create a base pre-commit configuration:

   pre-commit sample-config >.pre-commit-config.yaml

3. Append to the created .pre-commit-config.yaml the following lines:

   - repo: https://github.com/akaihola/darker
     rev: 1.7.1
     hooks:
       - id: darker

4. install the Git hook scripts and update to the newest version:

   pre-commit install
   pre-commit autoupdate

When auto-updating, care is being taken to protect you from possible incompatibilities introduced by Black updates. See Guarding against Black compatibility breakage for more information.

If you’d prefer to not update but keep a stable pre-commit setup, you can pin Black and other reformatter/linter tools you use to known compatible versions, for example:

- repo: https://github.com/akaihola/darker
  rev: 1.7.1
  hooks:
    - id: darker
      args:
        - --isort
        - --lint mypy
        - --lint flake8
        - --lint pylint
      additional_dependencies:
        - black==22.12.0
        - isort==5.11.4
        - mypy==0.990
        - flake8==5.0.4
        - pylint==2.15.5

- repo: https://github.com/akaihola/darker
  rev: 1.7.1
  hooks:
    - id: darker
      args: [--isort]
      additional_dependencies:
        - isort~=5.9

GitHub Actions integration

You can use Darker within a GitHub Actions workflow without setting your own Python environment. Great for enforcing that modifications and additions to your code match the Black code style.

name: Lint

on: [push, pull_request]

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
        with:
          fetch-depth: 0
      - uses: actions/setup-python@v4
      - uses: akaihola/darker@1.7.1
        with:
          options: "--check --diff --isort --color"
          src: "./src"
          version: "~=1.7.1"
          lint: "flake8,pylint==2.13.1"

Using linters

One way to use Darker is to filter linter output to only those linter messages which appeared after the modifications to source code files, as well as old messages which concern modified lines. Darker supports any linter with output in one of the following formats:

<file>:<linenum>: <description>
<file>:<linenum>:<col>: <description>

Most notably, the following linters/checkers have been verified to work with Darker:

• Mypy for static type checking

• Pylint for generic static checking of code

• Flake8 for style guide enforcement

• cov_to_lint.py for test coverage

New in version 1.1.0: Support for Mypy, Pylint, Flake8 and compatible linters.

New in version 1.2.0: Support for test coverage output using cov_to_lint.py.

To run a linter, use the --lint / -L command line option with the linter command or a full command line to pass to a linter. Some examples:

• -L flake8: enforce the Python style guide using Flake8

• -L "mypy --strict": do static type checking using Mypy

• --lint="pylint --ignore='setup.py'"": analyze code using Pylint

• -L cov_to_lint.py: read .coverage and list non-covered modified lines

Note: Full command lines aren’t fully tested on Windows. See issue #456 for a possible bug.

Darker also groups linter output into blocks of consecutive lines separated by blank lines. Here’s an example of cov_to_lint.py output:

$ darker --revision 0.1.0.. --check --lint cov_to_lint.py src
src/darker/__main__.py:94:  no coverage:             logger.debug("No changes in %s after isort", src)
src/darker/__main__.py:95:  no coverage:             break

src/darker/__main__.py:125: no coverage:         except NotEquivalentError:

src/darker/__main__.py:130: no coverage:             if context_lines == max_context_lines:
src/darker/__main__.py:131: no coverage:                 raise
src/darker/__main__.py:132: no coverage:             logger.debug(

Syntax highlighting

Darker automatically enables syntax highlighting for the --diff, -d/--stdout and -L/--lint options if it’s running on a terminal and the Pygments package is installed.

You can force enable syntax highlighting on non-terminal output using

• the color = true option in the [tool.darker] section of pyproject.toml of your Python project’s root directory,

• the PY_COLORS=1 environment variable, and

• the --color command line option for darker.

You can force disable syntax highlighting on terminal output using

• the color = false option in pyproject.toml,

• the PY_COLORS=0 environment variable, and

• the --no-color command line option.

In the above lists, latter configuration methods override earlier ones, so the command line options always take highest precedence.

Guarding against Black compatibility breakage

Darker accesses some Black internals which don’t belong to its public API. Darker is thus subject to becoming incompatible with future versions of Black.

To protect users against such breakage, we test Darker daily against the Black main branch and strive to proactively fix any potential incompatibilities through this process. If a commit to Black main branch introduces an incompatibility with Darker, we will release a first patch version for Darker that prevents upgrading Black and a second patch version that fixes the incompatibility. A hypothetical example:

1. Darker 9.0.0; Black 35.12.0 -> OK

2. Darker 9.0.0; Black main (after 35.12.0) -> ERROR on CI test-future workflow

3. Darker 9.0.1 released, with constraint Black<=35.12.0 -> OK

4. Black 36.1.0 released, but Darker 9.0.1 prevents upgrade; Black 35.12.0 -> OK

5. Darker 9.0.2 released with a compatibility fix, constraint removed; Black 36.1.0 -> OK

If a Black release introduces an incompatibility before the second Darker patch version that fixes it, the first Darker patch version will downgrade Black to the latest compatible version:

1. Darker 9.0.0; Black 35.12.0 -> OK

2. Darker 9.0.0; Black 36.1.0 -> ERROR

3. Darker 9.0.1, constraint Black<=35.12.0; downgrades to Black 35.12.0 -> OK

4. Darker 9.0.2 released with a compatibility fix, constraint removed; Black 36.1.0 -> OK

To be completely safe, you can pin both Darker and Black to known good versions, but this may prevent you from receiving improvements in Black.

It is recommended to use the ‘~=’ “compatible release” version specifier for Darker to ensure you have the latest version before the next major release that may cause compatibility issues.

See issue #382 and PR #430 for more information.

How does it work?

Darker takes a git diff of your Python files, records which lines of current files have been edited or added since the last commit. It then runs Black and notes which chunks of lines were reformatted. Finally, only those reformatted chunks on which edited lines fall (even partially) are applied to the edited file.

Also, in case the --isort option was specified, isort is run on each edited file before applying Black. Similarly, each linter requested using the –lint <command> option is run, and only those linting messages are displayed which appeared after the modifications to the source code files, or which were there already before but now fall on modified lines.

License

BSD. See LICENSE.rst.

Prior art

• black-macchiato

• darken (deprecated in favor of Darker; thanks Carreau for inspiration!)

Interesting code formatting and analysis projects to watch

The following projects are related to Black or Darker in some way or another. Some of them we might want to integrate to be part of a Darker run.

• blacken-docs – Run Black on Python code blocks in documentation files

• blackdoc – Run Black on documentation code snippets

• velin – Reformat docstrings that follow the numpydoc convention

• diff-cov-lint – Pylint and coverage reports for git diff only

• xenon – Monitor code complexity

• pyupgrade – Upgrade syntax for newer versions of the language (see #51)

• yapf – Google’s Python formatter

• yapf_diff – apply yapf or other formatters to modified lines only

Contributors ✨

See README.rst for the list of contributors.

This project follows the all-contributors specification. Contributions of any kind are welcome!

GitHub stars trend