Opinionated JSON to CSV converter
Project description
Flatterer. Making JSON flatterer
Introduction
An opinionated JSON to CSV/XLSX converter which tries to make a useful relational output for data analysis.
It aims to be fast and memory efficient.
Install
pip install flatterer
Flatterer requires Python 3.6 or greater. It is written as a python extension in Rust but has binaries (wheels) for linux (x64), macos (x64 and universal) and windows (x64, x86). On other platforms a rust toolchain will need to be installed.
Example JSON
Say you have a JSON data like this named games.json
:
[
{
"id": 1,
"title": "A Game",
"releaseDate": "2015-01-01",
"platforms": [
{"name":"Xbox"},
{"name":"Playstation"}
],
"rating": {
"code": "E",
"name": "Everyone"
}
},
{
"id": 2,
"title": "B Game",
"releaseDate": "2016-01-01",
"platforms": [
{"name":"PC"}
],
"rating": {
"code": "E",
"name": "Everyone"
}
}
]
Running Flatterer
Run the above file with flatterer.
flatterer games.json games_dir
Output Files
By running the above you will get the following files:
tree games_dir
games_dir/
├── csv
│ ├── games.csv
│ └── platforms.csv
├── data_package.json
├── fields.csv
└── ...
Main Table
games.csv
contains:
_link | _link_games | id | rating_code | rating_name | releaseDate | title |
---|---|---|---|---|---|---|
1 | 1 | 1 | E | Everyone | 2015-01-01 | A Game |
2 | 2 | 2 | E | Everyone | 2016-01-01 | B Game |
Special columns _link
and _link_games
are generated. _link
is the primary key there unique per game.
Also the rating
sub-object is promoted to this table it has a one-to-one relationship with games
.
Sub-object properties are separated by '_'.
One To Many Table
platforms
is an array so is a one-to-many with games therefore needs its own table:
platforms.csv
contains:
_link | _link_games | name |
---|---|---|
1.platforms.0 | 1 | Xbox |
1.platforms.1 | 1 | Playstation |
2.platforms.0 | 2 | PC |
Link Fields
_link
is the primary key for the platforms
table too. Every table contains a _link_games
field to easily join to the main games
table.
If there was a sub-array of platforms
then that would have _link
, _link_games
and _link_platforms
fields.
To generalize this the _link__<table_name>
fields joins to the _link
field of <table_name>
i.e the _link__<table_name>
are the foreign keys refrencing <table_name>._link
.
Fields CSV
fields.csv
contains some metadata about the output tables:
table_name | field_name | field_type | count | field_title |
---|---|---|---|---|
platforms | _link | text | 3 | _link |
platforms | _link_games | text | 3 | _link_games |
platforms | name | text | 3 | name |
games | _link | text | 2 | _link |
games | _link_games | text | 2 | _link_games |
games | id | number | 2 | id |
games | rating_code | text | 2 | rating_code |
games | rating_name | text | 2 | rating_name |
games | releaseDate | date | 2 | releaseDate |
games | title | text | 2 | title |
The field_type
column contains a type guess useful for inserting into a database. The field_title
is the column heading in the CSV file or XLSX tab, which is initally the same as the field_name.
After editing this file then you can rerun the transform:
flatterer games.json new_games_dir -f myfields.csv --only-fields
This can be useful for renameing columns, rearranging the field order or if you want to remove some fields the --only-fields
flag will only include the fields in the edited file.
data_package.json
contains metadata in the Tabular Datapackge Spec
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for flatterer-0.8.0-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 990d33118b992f9ce4109daef4fb84a5bfb209833c8563010e813fcb46b8e3da |
|
MD5 | cd55cc7a044cf7a868a7a0e6c5028032 |
|
BLAKE2b-256 | 7850e0e13827a2a38ea2a7f90ff743502da8987e857cc97b54f58db66d44c4d7 |
Hashes for flatterer-0.8.0-cp310-none-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e0702e2f90c59b7f7363e8293480400c3912ad73ff5cde7a0690d4b318f25c25 |
|
MD5 | 1fc7c17bbab46d8c34b5e418c9b95f81 |
|
BLAKE2b-256 | 4e577e727de2a6df78dcf5a4c89736ab8db35e8fa506e81660cc6b265a9ceb49 |
Hashes for flatterer-0.8.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | df88c32789a090b0faaa7498f496ef95eb29adc2dba5438a55ed7e42049b6d7e |
|
MD5 | efa36c61fe97bda1345806fa1f532de7 |
|
BLAKE2b-256 | 94daa81273603066fadb3409545b2135c4981c93e80dce1216bb01a18da138a7 |
Hashes for flatterer-0.8.0-cp310-cp310-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b8b125ee163c95f69c24ee230ecb6b7e3190ebe51339da520a65688c5ecf7085 |
|
MD5 | 30ba4ed2cd6222a5c65fe9840bb13020 |
|
BLAKE2b-256 | 10331ef81641094c69c3bf9eb5388e1f3cafea6258f2c1243714345fc578e49f |
Hashes for flatterer-0.8.0-cp310-cp310-macosx_10_7_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0cf0eb4c3362267803d60ccef4ef5bcab0d1e2dd1fc71a2d25d6f027fe632500 |
|
MD5 | ff42b23cc5f903598a14444abc7796a9 |
|
BLAKE2b-256 | 550663e36d490d6b3f2c3cb92a789969c121810cb6e02715c148413d621e64fe |
Hashes for flatterer-0.8.0-cp39-none-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b5ffa3cdb96719447036c4cee93279860e054bf1da4e05849665cf56375eeeae |
|
MD5 | f7c1827edb6d30c80b3e26037d7d7925 |
|
BLAKE2b-256 | de55760be0bc8b2aab0090325396f821235a90fffc9073a0217acb23a95c4207 |
Hashes for flatterer-0.8.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6da96127e94ca4d451729f34009cbba57bd909f2a79991710d1debf14e071912 |
|
MD5 | 41cbed9ad9a9508aa8d224912d41066a |
|
BLAKE2b-256 | d104f8fb607aae50d887c25bd06d67f11b74fcbdf570f07775528c4919cb2ae5 |
Hashes for flatterer-0.8.0-cp39-cp39-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 64b2bfb35f8481e391a5f5a73db792848f4ec7b64670e308bc6973c447baf99a |
|
MD5 | 4ee143033bd9528ece1f6e26929e286c |
|
BLAKE2b-256 | a897c1a7720d2df3ff263a60953044a28f3c34e7a08c462f7294f3519b1ab1e0 |
Hashes for flatterer-0.8.0-cp39-cp39-macosx_10_7_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 55fb516180fd72f95f5d081338c01d61755f191bc542a9db563883ec350c6d78 |
|
MD5 | d0d191ccd7ad89688b6f08b4ea24017f |
|
BLAKE2b-256 | 3b0a97841967f40d7483670c55ed48d4d5c7cc6d63f68b03902fe1cfa783e7c3 |
Hashes for flatterer-0.8.0-cp38-none-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | beeeef88421b3fae1fe8e24415e041c852997d43b409b31de9a2da0d10aa4eff |
|
MD5 | b40c87c9396db131ede02912edd3246c |
|
BLAKE2b-256 | 3ff75fbfb374dced00b824329f5aca86d220526a5b18a539d8377820aa5004c6 |
Hashes for flatterer-0.8.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f2ae29ca0d69edbd061590ddab6a4c6ff84da0c9d86fbd70d6fd8de534959919 |
|
MD5 | d816087772c59f837a1ce16a0226a690 |
|
BLAKE2b-256 | 94bdc583e75ab3d5ec8cf7704375e898f5b4eb7d0a34773aa22f309e8d5fabbd |
Hashes for flatterer-0.8.0-cp38-cp38-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f27f01b56a676ef46589d82e35a735f2a544021b7a7f5db6c3926dd979b8ac55 |
|
MD5 | 6f4f8ba1e9d73d35fffdd2858023bad3 |
|
BLAKE2b-256 | eabca62f96d6c2b3ef718a3d62b006b73411d738c6fd54bd73b59e34456b41af |
Hashes for flatterer-0.8.0-cp38-cp38-macosx_10_7_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cb27ec2edeba88164219e8a906c40eb753c32afd0e82a6a3feae6c4ff6ed7d58 |
|
MD5 | f0ad24255ded5120d84c45bab2601ed6 |
|
BLAKE2b-256 | 5e9a65a90d59237e187558ba393987062b098753f6aee854acd24c3c43cb3257 |
Hashes for flatterer-0.8.0-cp37-none-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 917c19f22e293d14cb5733ec6a6cde1b91729990b1818bcd3f9858a8e24f0370 |
|
MD5 | ed3fd884dd768b4095fdb6b810081f25 |
|
BLAKE2b-256 | 8a00a6aa989389314bbd19909e508eb64ef3de404f3d6d22dea41674ca3c3ae1 |
Hashes for flatterer-0.8.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4ed0bd117ac2bc79dd7ff0e0cd7abe6230be1c159c67cfc6f6e9666e33f871af |
|
MD5 | 55ae9c46b9286561b29602e0b0cde7b0 |
|
BLAKE2b-256 | bb0d2fcc08f72d89bac40eb95c9b45417ac063686de3a9bc859b04200eb58440 |
Hashes for flatterer-0.8.0-cp37-cp37m-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 39f7af6feba7b427aeb37ba4c79becf20866e09da81aa20f48fbf7cb2a7e3847 |
|
MD5 | 7fb209c3e1b253905f6e7a93da65f17a |
|
BLAKE2b-256 | aa3b04c84be3850de2e5b7dc690dbd40e88229ea038377d2b56182663d12dada |
Hashes for flatterer-0.8.0-cp37-cp37m-macosx_10_7_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 37a28c97226cfb86a85729999365405f3df306ee083357962761d81c00ab651b |
|
MD5 | f95f930ed76958665a981eb32693e1b2 |
|
BLAKE2b-256 | b0747f3885bc80ef45b69ae606ef5e9680cd9527ee05a82fb6d2bcc0287ae030 |
Hashes for flatterer-0.8.0-cp36-none-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0059edbf91b85866d854d5f16633fff1339e11f07d603c4a36eaebd6eea997e3 |
|
MD5 | bd882b71d41a255d784f36baed99962c |
|
BLAKE2b-256 | 11551e94bf63a5b0ade84432bcc581f5a075ea44a24a7918549b9207e765b4e8 |
Hashes for flatterer-0.8.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a03c5c18f5f9d35924809755598a6448c18796294724db726aa4c604d2dafd4c |
|
MD5 | 676800d02911e0547d00f35a4a7bc022 |
|
BLAKE2b-256 | 23bed6ae8f3e1f824a94b26ef6e73c10bd6c82e46ece135442232b2e543986c8 |
Hashes for flatterer-0.8.0-cp36-cp36m-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 75f458bae9c5599f9318a351f0aabdbca13ba43993333995214b955dfb7c15b1 |
|
MD5 | 8a67518a0f0d4ecb5be0a94f2a580344 |
|
BLAKE2b-256 | 6d3e36ed9bf762d48e4e0080c7aeabc5293f71c4aeeea3ccd5f1869048127f3e |
Hashes for flatterer-0.8.0-cp36-cp36m-macosx_10_7_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 233cc6a4de8ef6f11eb152d923cbb0d8558e9c74ce2c7f9589d025455dbeb2cd |
|
MD5 | 9b98e4e221b56c257be28ee445ab50b5 |
|
BLAKE2b-256 | 6ccfc7f862ae840770ee28d46872300172579ebeb66bd1f501d8fd24526e6ad4 |