partridge 0.11.0

Partridge
=========


.. image:: https://img.shields.io/pypi/v/partridge.svg
:target: https://pypi-hypernode.com/pypi/partridge

.. image:: https://img.shields.io/travis/remix/partridge.svg
:target: https://travis-ci.org/remix/partridge


Partridge is python library for working with `GTFS <https://developers.google.com/transit/gtfs/>`__ feeds using `pandas <https://pandas.pydata.org/>`__ DataFrames.

The implementation of Partridge is heavily influenced by our experience at `Remix <https://www.remix.com/>`__ ingesting, analyzing, and debugging thousands of GTFS feeds from hundreds of agencies.

At the core of Partridge is a dependency graph rooted at ``trips.txt``. Disconnected data is pruned away according to this graph when reading the contents of a feed. The root node can optionally be filtered to create a view of the feed specific to your needs. It's most common to filter a feed down to specific dates (``service_id``), routes (``route_id``), or both.

.. figure:: dependency-graph.png
:alt: dependency graph


Philosphy
---------

The design of Partridge is guided by the following principles:

**As much as possible**

- Favor speed
- Allow for extension
- Succeed lazily on expensive paths
- Fail eagerly on inexpensive paths

**As little as possible**

- Do anything other than efficiently read GTFS files into DataFrames
- Take an opinion on the GTFS spec

Usage
-----

**Reading a feed**

.. code:: python

import datetime
import partridge as ptg

path = 'path/to/sfmta-2017-08-22.zip'

service_ids_by_date = ptg.read_service_ids_by_date(path)

date = datetime.date(2017, 9, 25)
service_ids = service_ids_by_date[date]

feed = ptg.feed(path, view={
'trips.txt': {
'service_id': service_ids,
'route_id': '12300',
},
})

assert service_ids == set(feed.trips.service_id)

len(feed.stops)
# 88

feed.routes.head()
# route_id agency_id route_short_name route_long_name route_desc route_type \
# 12300 SFMTA 18 46TH AVENUE NaN 3
#
# route_url route_color route_text_color
# NaN NaN NaN


**Extracting a new feed**

.. code:: python

import partridge as ptg

inpath = 'gtfs.zip'
outpath = 'gtfs-slim.zip'

date, service_ids = ptg.read_busiest_date(inpath)

ptg.writers.extract_feed(inpath, outpath, {'trips.txt': {'service_id': service_ids}})

assert service_ids == set(ptg.feed(outpath).trips.service_id)


Features
--------

- Surprisingly fast :)
- Load only what you need into memory
- Built-in support for resolving service dates
- Easily extended to support fields and files outside the official spec
(TODO: document this)
- Handle nested folders and bad data in zips
- Predictable type conversions

Installation
------------

.. code:: console

pip install partridge

Thank You
---------

I hope you find this library useful. If you have suggestions for
improving Partridge, please open an `issue on
GitHub <https://github.com/remix/partridge/issues>`__.


History
=======

0.11.0 (2018-08-01)
-------------------
* Fix major performance issue related to encoding detection. Thank you to @cjer for reporting the issue and advising on a solution.


0.10.0 (2018-04-30)
-------------------

* Improved handling of non-standard compliant file encodings
* Only require functools32 for Python < 3
* ``ptg.parsers.parse_date`` no longer accepts dates, only strings


0.9.0 (2018-03-24)
------------------

* Improves read time for large feeds by adding LRU caching to ``ptg.parsers.parse_time``.


0.8.0 (2018-03-14)
------------------

* Gracefully handle completely empty files. This change unifies the behavior of reading from a CSV
with a header only (no data rows) and a completely empty (zero bytes)
file in the zip.


0.7.0 (2018-03-09)
------------------

* Fix handling of nested folders and zip containing nested folders.
* Add ``ptg.get_filtered_feed`` for multi-file filtering.


0.6.1 (2018-02-24)
------------------

* Fix bug in ``ptg.read_service_ids_by_date``. Reported by @cjer in #27.


0.6.0 (2018-02-21)
------------------

* Published package no longer includes unnecessary fixtures to reduce the size.
* Naively write a feed object to a zip file with ``ptg.write_feed_dangerously``.
* Read the earliest, busiest date and its ``service_id``'s from a feed with ``ptg.read_busiest_date``.
* Bug fix: Handle ``calendar.txt``/``calendar_dates.txt`` entries w/o applicable trips.


0.6.0.dev1 (2018-01-23)
-----------------------

* Add support for reading files from a folder. Thanks again @danielsclint!


0.5.0 (2017-12-22)
------------------

* Easily build a representative view of a zip with ``ptg.get_representative_feed``. Inspired by `peartree <https://github.com/kuanb/peartree/blob/3bfc3f49ae6986d6020913b63c8ee32582b3dcc3/peartree/paths.py#L26>`_.
* Extract out GTFS zips by agency_id/route_id with ``ptg.extract_{agencies,routes}``.
* Read arbitrary files from a zip with ``feed.get('myfile.txt')``.
* Remove ``service_ids_by_date``, ``dates_by_service_ids``, and ``trip_counts_by_date`` from the feed class. Instead use ``ptg.{read_service_ids_by_date,read_dates_by_service_ids,read_trip_counts_by_date}``.


0.4.0 (2017-12-10)
------------------

* Add support for Python 2.7. Thanks @danielsclint!


0.3.0 (2017-10-12)
------------------

* Fix service date resolution for raw_feed. Previously raw_feed considered all days of the week from calendar.txt to be active regardless of 0/1 value.


0.2.0 (2017-09-30)
------------------

* Add missing edge from fare_rules.txt to routes.txt in default dependency graph.


0.1.0 (2017-09-23)
------------------

* First release on PyPI.