capone 1.0.0

# Capone

_Never let your books land you in the pen._


`Capone` is a library that provides double-entry bookkeeping (the foundation of
all modern accounting) for Django with the ability to link each recorded
transaction to zero or more other Django models as evidence for that
transaction.

## Introduction

In double-entry bookkeeping (DEB), all recordable events (purchases, sales,
equipment depreciation, bad debt markdowns, etc.) are tracked as "ledger
entries" or "transactions" in "ledgers". Each ledger entry is made up of one
or more "credit" and one or more "debit" entries. For the sake of this brief
example, you can think of credits as increasing the amount of money recorded in
a ledger and a debit decreasing it. With that assumption, the central idea behind
double-entry bookkeeping is that the sum of every ledger entry's debits must
equal the sum of its credits. `capone` implements a double-entry bookkeeping
system by providing an API for checking that all created entries satisfy this
condition or rolling back the transaction if not.

In addition to this standard bookkeeping functionality, `capone` also allows
any number of arbitrary objects to be attached, via generic foreign keys, to
a ledger entry as "evidence" for that transaction's having happened. For
instance, a transaction recording a bank deposit paying for several medical
tests at a time from an insurance company to your medical testing company could
be linked to the original `Order` objects that recorded the test. `capone`
also provides an API for the efficient querying of ledger entries by evidence.

For more information on the concept of double-entry bookkeeping itself, we
recommend the Wikipedia article:
https://en.wikipedia.org/wiki/Double-entry_bookkeeping_system.


## Local Development

### Setup:

First, you must set up your working environment:

make setup

This will build a local virtualenv and all other requirements for local
development.


### Running Commands:

The following commands are available for interacting with the app:

To start a shell instance so that you can interact with the app via the ORM:

make shell

Note: before any of the following instructions, you may have to run `make
develop` to set up a postgres database for this app.

First, activate a virtualenv so that your commands have access to the
environment built by `make setup`:

From the repository root, run:

source .venv/bin/activate

Then you should be free to run

./manage.py makemigrations --settings=capone.tests.settings

or any other `manage.py` command, even those in the Makefile.

To run individual tests, use the following:

./manage.py test --settings=capone.tests.settings capone.tests

Notice the `--settings=capone.tests.settings` argument: because this repository
is a django sub-module, it wouldn't make sense for it to come with its own
default `settings.py` file. Instead, it ships with one used to run its tests.
To use `manage.py`, we have to pass an import path to the settings file
explicitly.


## Models

Let's introduce the models provided by `capone` and how they relate to one
another.

Note that all objects in this library have `created_at` and `modified_at`
fields that are `auto_now_add` and `auto_now`, respectively.

### Accounting Models

The models in this section are those that correspond most to well known
accounting concepts, i.e. those involved in keeping accounts using the
principles of double-entry bookkeeping. They model ledgers, journal entries,
credits and debits, and any metadata one wishes to store with these objects.


#### Ledger

A `Ledger` is the top-most level of organization of information in double-entry
bookkeeping as well as the `capone` app. Most ledgers have names familiar to
those with any knowledge of accounting, such as "revenue" or "accounts
receivable".

`Ledgers` are synonymous with the accounting concept of an "account", so you
may see references to accounts in this documentation or elsewhere in the
accounting literature.

As a data structure, a `Ledger` in this library is little more than a name,
description, and unique number: `LedgerEntries` (see below) point to a `Ledger`
to represent their being "in" a `Ledger`. `Transactions` (see below also) that
are "between" two `Ledgers` have a `LedgerEntry` pointing to one `Ledger` and
another `LedgerEntry` pointing to the other `Ledger`.

##### `increased_by_debits`

`Ledger` also has the sometimes confusing field `increased_by_debits`. All
`Ledgers` are of one of two types: either debits increase the "value" of an
account or credits do. By convention, asset and expense accounts are of the
former type, while liabilities, equity, and revenue are of the latter: in
short, an increase to an "asset"-type account is a debit, and an increase to
a "liability" or "equity"-type account is a credit.

Here's a handy mnemonic for the two types of accounts: The accounting equation
says (by definition) that:

assets == liabilities + owner equity

The terms on the right of the equals sign are increased by debits, and terms on
the left of the equals sign are decreased by debits. We can therefore use the
accounting equation to know whether to use debits or credits to model an
increase in a ledger.

**So because debits and credits mean different things in different types of
accounts, we can have a transaction with an "equal and opposite" credit and
debit pair of the same currency amount, but that still represents a net
increase in the value of a company: a debit in Accounts Receivable and a credit
in Revenue increases both accounts while satisfying the accounting equation.**

Currently, field `increased_by_debits` is not used by the code in `capone` but
is provided as a convenience to users who might wish to incorporate this
information into an external report or calculation.


#### Transaction

A `Transaction` is a record of a discrete financial action, represented by
a collection of debits and credits whose sums equal one another. Practically
all models in `capone` link to or through `Transaction`: in a sense you could
say it's the main model provided by `capone`. A `Transaction` can sometimes be
referred to as a "journal entry".

The `Transaction` model records debits and credits by linking to
`LedgerEntries`, which include currency amounts of the proper sign, and those
`LedgerEntries` themselves point to `Ledger`. In other words, `Transaction`
and `Ledger` are linked in a many-to-many fashion by going through
`LedgerEntry` as a custom through model. The "proper sign" part is taken care
of by the `credit` and `debit` convenience methods (see examples below).

`Transactions` should never be deleted. Instead, a new `Transaction` with
debits and credits swapped should be created using
`capone.api.actions.void_transaction` to negate the effect of the `Transaction`
you'd like to remove. The `voids` field on the new `Transaction` will
automatically be filled in with the old `Transaction` you wish to remove. By
this method, you'll never have to delete data from your system as a part of
normal operation, which mimics one of the many benefits of traditional,
non-computerized double-entry bookkeeping.

`Transaction` also has the following fields to provide metadata for each transaction:

- `created_by`: The user who created this `Transaction`.
- `notes`: A free-form text field for adding to a `Transaction` any
information not expressed in the numerous metadata fields.
- `posted_timestamp`: The time a `Transaction` should be considered valid
from. `capone.api.actions.create_transaction` automatically deals with
filling in this value with the current time. You can change this value to
post-date or back-date `Transactions` because `created_at` will always
represent the true object creation time.
- `transaction_id`: A Universally Unique Identifier (UUID) for the
`Transaction`, useful for unambiguously referring to a `Transaction`
without using primary keys or other database internals.
- `type`: A user-defined type for the `Transaction` (see the `TransactionType`
model below).


#### TransactionType

A `TransactionType` is a user-defined, human-readable "type" for
a `Transaction`, useful for sorting, aggregating, or annotating `Transactions`.
The default `TransactionType` is `MANUAL`, which is created automatically by
the library, but you can define others, say for bots or certain classes of
users.

Currently, `TransactionType` is not used by the code in `capone` but is
provided as a convenience to users who might wish to incorporate this
information into an external report or calculation.


#### LedgerEntry

`LedgerEntries` represent single debit or credit entries in a single `Ledger`.
`LedgerEntries` are grouped together into `Transactions` (see above) with the
constraint that the sum of all credit and debit `LedgerEntries` for a given
`Transaction` must equal zero.

`LedgerEntries` have a field `entry_id`, which is a UUID for unambiguously
referring to a single `LedgerEntry`.


### Evidence Models

The models in this section deal with adding evidence to `Transactions` and
searching over that evidence.


#### TransactionRelatedObject

A `TransactionRelatedObject` (`TRO`) represents the "evidence" relationship that
makes the `capone` library more useful. A `TRO` links a `Transaction` to an arbitrary
object in the larger app that this library is used in using a generic foreign
key. One `TRO` links one `Transaction` and one arbitrary object, so we make as
many `TROs` as we want pieces of evidence. There are several convenience
methods in `capone.api.queries` for efficiently querying over `Transactions`
based on evidence and evidence objects based on their `Transactions` (see
examples below).


#### LedgerBalance

A `LedgerBalance` is similar to a `TRO` in that it allows linking `ledger`
objects with objects from the wider app that the library is used in via generic
foreign keys. The purpose of `LedgerBalance` is to denormalize for more
efficient querying the current sum of debits and credits for an object in
a specific Ledger. Therefore, there is only one `LedgerBalance` for each
`(ledger, related_object)` tuple.

You should never have to manually create or edit a `LedgerBalance`: doing so,
as well as keeping them up-to-date, is handled by `capone` internals. For the
same reasons, deleting them is not necessary or a good idea.

The purpose of `LedgerBalance` can best be demonstrated by considering the
deceptively simple query, "how many Orders (a non-`capone` model we presumably
created in the app where we include `capone` as a library) have an Accounts
Receivable balance greater than zero?" One would have to calculate the ledger
balance over literally the product of all ledgers and all non-`capone` objects
in the database, and then filter them for all those with balances above zero,
to answer this question, which is obviously too expensive. By keeping track of
the per-`Ledger` balance for each object used as evidence in a `Transaction`,
we can much more easily make these queries with reasonable overhead.


## Usage

### Creating Ledgers

Let's start by creating two common ledger types, "Accounts Receivable" and
"Revenue", which usually have transactions between themselves:

```
>>> from capone.models import Ledger
>>> ar = Ledger.objects.create(name='Accounts Receivable', number=1, increased_by_debits=True)
<Ledger: Ledger Accounts Receivable>
>>> revenue = Ledger.objects.create(name='Revenue', number=2, increased_by_debits=True)
<Ledger: Ledger Revenue>
```

Both of these accounts are asset accounts, so they're both increased by debits.
Please consult the double-entry bookkeeping Wikipedia article or the
explanation for `increased_by_debits` above for a more in-depth explanation of
the "accounting equation" and whether debits increase or decrease an account.

Also, note that the default convention in `capone` is to store debits as
positive numbers and credits as negative numbers. This convention is common
but completely arbitrary. If you want to switch the convention around, you can
set `DEBITS_ARE_NEGATIVE` to `True` in your settings.py file. By default, that
constant doesn't need to be defined, and if it remains undefined, `capone` will
interpret its value as `False`.


### Faking Evidence Models

Now let's create a fake Order, so that we have some evidence for these ledger
entries, and a fake User, so we'll have someone to blame for these transactions:

```
>>> from capone.tests.factories import OrderFactory
>>> order = OrderFactory()
>>> from capone.tests.factories import UserFactory
>>> user = UserFactory()
```


### Creating Transactions

We're now ready to create a simple transaction:

```
>>> from capone.api.actions import create_transaction
>>> from capone.api.actions import credit
>>> from capone.api.actions import debit
>>> from decimal import Decimal
>>> from capone.models import LedgerEntry
>>> txn = create_transaction(user, evidence=[order], ledger_entries=[LedgerEntry(amount=debit(Decimal(100)), ledger=ar), LedgerEntry(amount=credit(Decimal(100)), ledger=revenue)])
>>> txn.summary()
{
u'entries': [
'LedgerEntry: $100.0000 in Accounts Receivable',
'LedgerEntry: $-100.0000 in Revenue',
],
u'related_objects': [
'TransactionRelatedObject: Order(id=1)',
]
}
```

Note that we use the helper functions `credit` and `debit` with positive
numbers to keep the signs consistent in our code. There should be no reason to
use negative numbers with `capone`.

Note also that the value for the credit and debit is the same: $100. If we
tried to create a transaction with mismatching amounts, we would get an error:

```
>>> create_transaction(user, evidence=[order], ledger_entries=[LedgerEntry(amount=debit(Decimal(100)), ledger=ar), LedgerEntry(amount=credit(Decimal(101)), ledger=revenue)])
---------------------------------------------------------------------------
TransactionBalanceException Traceback (most recent call last)

[...]

TransactionBalanceException: Credits do not equal debits. Mis-match of -1.
```

So the consistency required of double-entry bookkeeping is automatically kept.

There are many other options for `create_transaction`: see below or its
docstring for details.


### Ledger Balances

`capone` keeps track of the balance in each ledger for each evidence object in
a denormalized and efficient way. Let's use this behavior to get the balances
of our ledgers as well as the balances in each ledger for our `order` object:


```
>>> from capone.api.queries import get_balances_for_object

>>> get_balances_for_object(order)
defaultdict(<function <lambda> at 0x7fd7ecfa96e0>, {<Ledger: Ledger Accounts Receivable>: Decimal('100.0000'), <Ledger: Ledger Revenue>: Decimal('-100.0000')})

>>> ar.get_balance()
Decimal('100.0000')

>>> revenue.get_balance()
Decimal('-100.0000')
```


### Voiding Transactions

We can also void that transaction, which enters a transaction with the same
evidence but with all values of the opposite sign:

```
>>> void = void_transaction(txn, user)
<Transaction: Transaction 9cd85014-c588-43ff-9532-a6fc2429069e>

>>> void_transaction(txn, user)
---------------------------------------------------------------------------
UnvoidableTransactionException Traceback (most recent call last)

[...]

UnvoidableTransactionException: Cannot void the same Transaction #(e0842107-3a5b-4487-9b86-d1a5d7ab77b4) more than once.

>>> void.summary()
{u'entries': ['LedgerEntry: $-100.0000 in Accounts Receivable',
'LedgerEntry: $100.0000 in Revenue'],
u'related_objects': ['TransactionRelatedObject: Order(id=1)']}

>>> txn.voids

>>> void.voids
<Transaction: Transaction e0842107-3a5b-4487-9b86-d1a5d7ab77b4>
```

Note the new balances for evidence objects and `Ledgers`:

```
>>> get_balances_for_object(order)
defaultdict(<function <lambda> at 0x7fd7ecfa9758>, {<Ledger: Ledger Accounts Receivable>: Decimal('0.0000'), <Ledger: Ledger Revenue>: Decimal('0.0000')})

>>> ar.get_balance()
Decimal('0.0000')

>>> revenue.get_balance()
Decimal('0.0000')
```


### Transaction Types

You can label a `Transaction` using a foreign key to the `TransactionType` to,
say, distinguish between manually made `Transactions` and those made by a bot,
or between `Transactions` that represent two different types of financial
transaction, such as "Reconciliation" and "Revenue Recognition".

By default, `Transactions` are of a special, auto-generated "manual" type:

```
>>> txn.type
<TransactionType: Transaction Type Manual>
```

but you can create and assign `TransactionTypes` when creating `Transactions`:

```
>>> from capone.models import TransactionType
>>> new_type = TransactionType.objects.create(name='New type')
>>> txn = create_transaction(user, evidence=[order], ledger_entries=[LedgerEntry(amount=debit(Decimal(100)), ledger=ar), LedgerEntry(amount=credit(Decimal(100)), ledger=revenue)], type=new_type)
>>> txn.type
<TransactionType: Transaction Type New type>
```


### Querying Transactions

#### Getting Balances

`Transaction` has a `summary` method to summarize the data on the many models
that can link to it:

```
>>> txn.summary()
{u'entries': ['LedgerEntry: $100.0000 in Accounts Receivable',
'LedgerEntry: $-100.0000 in Revenue'],
u'related_objects': ['TransactionRelatedObject: Order(id=1)']}
```

To get the balance for a `Ledger`, use its `get_balance` method:

```
>>> ar.get_balance()
Decimal('100.0000')
```

To efficiently get the balance of all transactions with a particular object as
evidence, use `get_balances_for_objects`:

```
>>> get_balances_for_object(order)
defaultdict(<function <lambda> at 0x7fd7ecfa9230>, {<Ledger: Ledger Accounts Receivable>: Decimal('100.0000'), <Ledger: Ledger Revenue>: Decimal('-100.0000')})
```

`Transactions` are validated before they are created, but if you need to do
this manually for some reason, use the `validate_transaction` function, which
has the same prototype as `create_transaction`:

```
>>> validate_transaction(user, evidence=[order], ledger_entries=[LedgerEntry(amount=debit(Decimal(100)), ledger=ar), LedgerEntry(amount=credit(Decimal(100)), ledger=revenue)], type=new_type)
>>> validate_transaction(user, evidence=[order], ledger_entries=[LedgerEntry(amount=debit(Decimal(100)), ledger=ar), LedgerEntry(amount=credit(Decimal(101)), ledger=revenue)], type=new_type)
---------------------------------------------------------------------------
TransactionBalanceException Traceback (most recent call last)
<ipython-input-64-07b6d139bb37> in <module>()
----> 1 validate_transaction(user, evidence=[order], ledger_entries=[LedgerEntry(amount=debit(Decimal(100)), ledger=ar), LedgerEntry(amount=credit(Decimal(101)), ledger=revenue)], type=new_type)

/home/hunter/capone/capone/api/queries.pyc in validate_transaction(user, evidence, ledger_entries, notes, type, posted_timestamp)
67 if total != Decimal(0):
68 raise TransactionBalanceException(
---> 69 "Credits do not equal debits. Mis-match of %s." % total)
70
71 if not ledger_entries:

TransactionBalanceException: Credits do not equal debits. Mis-match of -1.
```

### Queries

Along with the query possibilities from the Django ORM, `capone` provides
`Transaction.filter_by_related_objects` for finding `Transactions` that are
related to certain models as evidence.

```
>>> Transaction.objects.count()
5

>>> Transaction.objects.filter_by_related_objects([order]).count()
5

>>> order2 = OrderFactory()

>>> create_transaction(user, evidence=[order2], ledger_entries=[LedgerEntry(amount=debit(Decimal(100)), ledger=ar), LedgerEntry(amount=credit(Decimal(100)), ledger=revenue)])
<Transaction: Transaction 68a4adb1-b898-493f-b5f3-4fe7132dd28d>

>>> Transaction.objects.filter_by_related_objects([order2]).count()
1
```

`filter_by_related_objects` is defined on a custom `QuerySet` provided for
`Transaction`, so calls to it can be chained like ordinary `QuerySet` function
calls:

```
>>> create_transaction(user, evidence=[order2], ledger_entries=[LedgerEntry(amount=debit(Decimal(100)), ledger=ar), LedgerEntry(amount=credit(Decimal(100)), ledger=revenue)])
<Transaction: Transaction 92049712-4982-4718-bc71-a405b0d762ac>

>>> Transaction.objects.filter_by_related_objects([order2]).count()
2

>>> Transaction.objects.filter_by_related_objects([order2]).filter(transaction_id='92049712-4982-4718-bc71-a405b0d762ac').count()
1
```

`filter_by_related_objects` takes an optional `match_type` argument, which is
of type `MatchType(Enum)` that allows one to filter in different ways, namely
whether the matching transactions may have "any", "all", "none", or "exactly"
the evidence provided, determined by `MatchTypes` `ANY`, `ALL`, `NONE`, and
`EXACT`, respectively.


### Asserting over Transactions

For writing tests, the method
`assert_transaction_in_ledgers_for_amounts_with_evidence` is provided for
convenience. As its name implies, it allows asserting the existence of exactly
one `Transaction` with the ledger amounts, evidence, and other fields on Ledger
provided to the method.

```
>>> create_transaction(user, evidence=[order], ledger_entries=[LedgerEntry(amount=debit(Decimal(100)), ledger=ar), LedgerEntry(amount=credit(Decimal(100)), ledger=revenue)])
<Transaction: Transaction b3e73f1d-6b10-4597-b19b-84800839d5b3>
>>> with assert_raises(Transaction.DoesNotExist):
... assert_transaction_in_ledgers_for_amounts_with_evidence(ledger_amount_pairs=[(revenue.name, credit(Decimal(100))), (ar.name, debit(Decimal(100)))], evidence=[])
...
>>> assert_transaction_in_ledgers_for_amounts_with_evidence(ledger_amount_pairs=[(revenue.name, credit(Decimal(100))), (ar.name, debit(Decimal(100)))], evidence=[order])
>>> with assert_raises(Transaction.DoesNotExist):
... assert_transaction_in_ledgers_for_amounts_with_evidence(ledger_amount_pairs=[(revenue.name, credit(Decimal(100))), (ar.name, debit(Decimal(100)))], evidence=[order])
...
Traceback (most recent call last):
File "<console>", line 2, in <module>
File "/usr/lib/python2.7/unittest/case.py", line 116, in __exit__
"{0} not raised".format(exc_name))
AssertionError: DoesNotExist not raised
```

You can see
`capone.tests.test_assert_transaction_in_ledgers_for_amounts_with_evidence` for
more examples!


## Image Credits

Image courtesy [Officer](https://commons.wikimedia.org/wiki/User:Officer) on [Wikipedia](https://commons.wikimedia.org/wiki/File:Al_Capone_in_Florida.jpg).
This work was created by a government unit (including state, county, and municipal government agencies) of the U.S. state of Florida. It is a public record that was not created by an agency which state law has allowed to claim copyright and is therefore in the public domain in the United States.