Docstring Auditor

Meet Docstring Auditor, your go-to solution for maintaining precise and up-to-date Python code documentation. Tired of misleading or outdated docstrings? Docstring Auditor harnesses the power of large language models to analyze and critique your docstrings, ensuring they align with your code's true purpose. Accessible to both experts and novices, Docstring Auditor is your ultimate companion for clear, concise, and informative docstrings. Say hello to better code documentation!

Motivation

Recognizing the need for a reliable tool to address the challenge of keeping code documentation in sync with evolving codebases, we developed Docstring Auditor to tackle this issue head-on. Our motivation was to create an accessible, user-friendly solution that empowers developers to maintain clear and up-to-date documentation with ease, enhancing their workflow and reducing misunderstandings.

Docstring Auditor leverages the advanced capabilities of GPT-4, a powerful language model designed to deeply understand both code and natural language. By incorporating GPT-4 into our tool, Docstring Auditor examines the docstrings in your Python code, identifying discrepancies between the documentation and the actual code implementation. The analysis covers errors, warnings, and potential improvements, providing valuable critiques and suggestions to help you keep your documentation accurate and coherent. Docstring Auditor not only ensures that technical details, such as variables and types, are consistent, but it also verifies that the docstrings' meanings are in harmony with the code's functionality.

With Docstring Auditor, you can trust that your documentation stays relevant, informative, and accessible to all members of your team, making collaboration smoother and more efficient than ever before.

Docstring Auditor can be utilized as a GitHub Action, a Docker container command-line tool, or Python package.

Features

• Analyzes Python functions' docstrings in a given file
• Identifies errors, warnings, and possible improvements
• Provides detailed critiques and suggestions for better docstrings
• Powered by OpenAI's GPT for accurate and insightful analysis
• Easy to use command-line interface

Example of Docstring Auditor in Action

Let's say you have a Python file called example.py with the following content:

def compute(a, b):
    """
    Add two numbers.

    Parameters
    ----------
    a : int or float
        The first number to be added or from which 'b' will be subtracted.
    b : int or float
        The second number to be added or subtracted.

    Returns
    -------
    int or float
        The result of the addition operation.
    """
     if a > 0:
        return a + b
     else:
        return a - b

To analyze the docstring of the add function, run:

docstring-auditor example.py

Docstring Auditor will then provide you with a detailed analysis of the docstring, including any errors, warnings, and suggestions for improvement. The output may look like...

Processing function 1 of 1...
--------------------------------------------------------------------------------
An error was found in the function: compute

The docstring states that the function adds two numbers, but the code also performs subtraction if 'a' is less than or equal to 0. The docstring should accurately describe both addition and subtraction operations.

A warning was found in the function: compute

The docstring does not follow the numpydoc style completely. The summary line should be a one-line summary, and the extended description should be provided in a separate paragraph.

A proposed solution to these concerns is:

"""
Add or subtract two numbers based on the value of 'a'.

This function performs addition if 'a' is greater than 0, and subtraction if 'a' is less than or equal to 0.

Parameters
----------
a : int or float
    The first number to be added or from which 'b' will be subtracted.
b : int or float
    The second number to be added or subtracted.

Returns
-------
int or float
    The result of the addition or subtraction operation.
"""

GitHub Action

Docstring Auditor can be used as a GitHub Action to automatically analyze the docstrings in your Python codebase. To use Docstring Auditor as a GitHub Action, add the following to your workflow file:

  - name: Docstring Auditor
    uses: agencyenterprise/docstring-auditor@main
    with:
      openaiApiKey: ${{ secrets.OPENAI_API_KEY }}
      path: .

For an example of how to use Docstring Auditor, see this workflow.

Docker

Docstring Auditor can be used with Docker

1. Install Docker
2. Run the following command:

# If your code lives in the directory /Path/to/code
# And you wish to analyse all files in that directory
docker run -it --rm -e OPENAI_API_KEY=sk-XXXX -v /Path/to/code:/repo docstring-auditor

# If your code lives in the directory /Path/to/code
# And you wish to analyse a file called module/file.py
docker run -it --rm -e OPENAI_API_KEY=sk-XXXX -v /Path/to/code:/repo docstring-auditor module/file.py

Usage

If you use the docker command above, Docstring Auditor will analyse all python files in your directory. If you wish for it to analyse a single file, pass in the file name with the repo prefix. For example, to analyse the file in src/module/file.py...

docker run -it --rm -e OPENAI_API_KEY=sk-XXXX -v /Path/to/code:/repo docstring-auditor src/module/file.py

The tool will then analyze the functions' docstrings in the specified file and display the critiques and suggestions for improvement.

Local Installation

You can also run Docstring Auditor locally by following these steps:

1. Install Python 3.6+
2. Install Git
3. Clone the repository: git clone git@github.com:agencyenterprise/docstring-auditor.git
4. Setup hatch: pip install hatch
5. Run the package hatch run docstring-auditor /path/to/your/python_file.py

Limitations

1. There is currently no checking for malformed json from GPT4. Occasionaly, GPT4 will return a bad json and the program will crash.
2. There is no rate limiting on the GPT4 calls.

Contributing

We welcome contributions to Docstring Auditor! If you'd like to contribute, please fork the repository and submit a pull request with your changes. We also appreciate bug reports and feature requests submitted through the GitHub issues page.