Tools for cleaning pandas DataFrames
Project description
pyjanitor is a Python implementation of the R package janitor, and provides a clean API for cleaning data.
Why janitor?
Originally a port of the R package, pyjanitor has evolved from a set of convenient data cleaning routines into an experiment with the method chaining paradigm.
Data preprocessing usually consists of a series of steps that involve transforming raw data into an understandable/usable format. These series of steps need to be run in a certain sequence to achieve success. We take a base data file as the starting point, and perform actions on it, such as removing null/empty rows, replacing them with other values, adding/renaming/removing columns of data, filtering rows and others. More formally, these steps along with their relationships and dependencies are commonly referred to as a Directed Acyclic Graph (DAG).
The pandas API has been invaluable for the Python data science ecosystem, and implements method chaining of a subset of methods as part of the API. For example, resetting indexes (.reset_index()), dropping null values (.dropna()), and more, are accomplished via the appropriate pd.DataFrame method calls.
Inspired by the ease-of-use and expressiveness of the dplyr package of the R statistical language ecosystem, we have evolved pyjanitor into a language for expressing the data processing DAG for pandas users.
Functionality
Current functionality includes:
Cleaning columns name (multi-indexes are possible!)
Removing empty rows and columns
Identifying duplicate entries
Encoding columns as categorical
Splitting your data into features and targets (for machine learning)
Adding, removing, and renaming columns
Coalesce multiple columns into a single column
Date conversions (from matlab, excel, unix) to Python datetime format
Expand a single column that has delimited, categorical values into dummy-encoded variables
Concatenating and deconcatenating columns, based on a delimiter
Syntactic sugar for filtering the dataframe based on queries on a column
Experimental submodules for finance, biology, chemistry, engineering, and pyspark
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file pyjanitor-0.20.5.tar.gz.
File metadata
- Download URL: pyjanitor-0.20.5.tar.gz
- Upload date:
- Size: 65.5 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.9.1 pkginfo/1.5.0.1 requests/2.10.0 setuptools/40.6.3 requests-toolbelt/0.8.0 tqdm/4.29.1 CPython/3.6.8
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 55e0480d848572df0267ccb52c8fe19ad0b53364a8fd70798298dece98660218 |
|
| MD5 | 9b6e688db7c76ceb12d71c682e310ffb |
|
| BLAKE2b-256 | 95144c589d180f83da61d821077966591bfc46866ce1b4b5351df5b563e77378 |
File details
Details for the file pyjanitor-0.20.5-py3-none-any.whl.
File metadata
- Download URL: pyjanitor-0.20.5-py3-none-any.whl
- Upload date:
- Size: 63.0 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.9.1 pkginfo/1.5.0.1 requests/2.10.0 setuptools/40.6.3 requests-toolbelt/0.8.0 tqdm/4.29.1 CPython/3.6.8
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 9770793aa6c2d8e896319ab896e4a16068ca3d237ff7647e182dfa484c705274 |
|
| MD5 | 688844026828b18d6c685fe2b43b429c |
|
| BLAKE2b-256 | c3b38928ea97893cf2af7ec25799bf4f40f74ebd9b47879f717c1b97ab4f82ee |
