Opinionated JSON to CSV converter
Project description
Introduction
An opinionated JSON to CSV/XLSX converter which tries to make a useful relational output for data analysis.
It aims to be fast and memory efficient.
Install
pip install flatterer
Flatterer requires Python 3.6 or greater. It is written as a python extension in Rust but has binaries (wheels) for linux (x64), macos (x64 and universal) and windows (x64, x86). On other platforms a rust toolchain will need to be installed.
Example JSON
Say you have a JSON data like this named games.json
:
[
{
"id": 1,
"title": "A Game",
"releaseDate": "2015-01-01",
"platforms": [
{"name":"Xbox"},
{"name":"Playstation"}
],
"rating": {
"code": "E",
"name": "Everyone"
}
},
{
"id": 2,
"title": "B Game",
"releaseDate": "2016-01-01",
"platforms": [
{"name":"PC"}
],
"rating": {
"code": "E",
"name": "Everyone"
}
}
]
Running Flatterer
Run the above file with flatterer.
flatterer games.json games_dir
Output Files
By running the above you will get the following files:
tree games_dir
games_dir/
├── csv
│ ├── games.csv
│ └── platforms.csv
├── datapackage.json
├── fields.csv
└── ...
Main Table
games.csv
contains:
_link | _link_games | id | rating_code | rating_name | releaseDate | title |
---|---|---|---|---|---|---|
1 | 1 | 1 | E | Everyone | 2015-01-01 | A Game |
2 | 2 | 2 | E | Everyone | 2016-01-01 | B Game |
Special column _link
is generated. _link
is the primary key there unique per game.
Also the rating
sub-object is promoted to this table it has a one-to-one relationship with games
.
Sub-object properties are separated by '_'.
One To Many Table
platforms
is an array so is a one-to-many with games therefore needs its own table:
platforms.csv
contains:
_link | _link_games | name |
---|---|---|
1.platforms.0 | 1 | Xbox |
1.platforms.1 | 1 | Playstation |
2.platforms.0 | 2 | PC |
Link Fields
_link
is the primary key for the platforms
table too. Every table except games
table, contains a _link_games
field to easily join to the main games
table.
If there was a sub-array of platforms
then that would have _link
, _link_games
and _link_platforms
fields.
To generalize this the _link__<table_name>
fields joins to the _link
field of <table_name>
i.e the _link__<table_name>
are the foreign keys refrencing <table_name>._link
.
Fields CSV
fields.csv
contains some metadata about the output tables:
table_name | field_name | field_type | count | field_title |
---|---|---|---|---|
platforms | _link | text | 3 | _link |
platforms | _link_games | text | 3 | _link_games |
platforms | name | text | 3 | name |
games | _link | text | 2 | _link |
games | id | number | 2 | id |
games | rating_code | text | 2 | rating_code |
games | rating_name | text | 2 | rating_name |
games | releaseDate | date | 2 | releaseDate |
games | title | text | 2 | title |
The field_type
column contains a type guess useful for inserting into a database. The field_title
is the column heading in the CSV file or XLSX tab, which is initally the same as the field_name.
After editing this file then you can rerun the transform:
flatterer games.json new_games_dir -f myfields.csv --only-fields
This can be useful for renameing columns, rearranging the field order or if you want to remove some fields the --only-fields
flag will only include the fields in the edited file.
datapackage.json
contains metadata in the Tabular Datapackge Spec
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for flatterer-0.12.9-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 31147bc7343cd0cf58b47bba230916f74f86c9cf7e9b1ae54d7b53ed920c6333 |
|
MD5 | 9805653cb0f3118eeebfb31e6c67961c |
|
BLAKE2b-256 | 3ebdcbb58e2b4ae204e281fca862935121a9566a663de5d3ce5d904fa85b18f6 |
Hashes for flatterer-0.12.9-cp310-none-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3df1902ba3fb588c8ba44b17f9f38582f7ace09aa1479e91ad1275db08e01b8d |
|
MD5 | 3affbb6c5a065c65a0e3bceb535b6454 |
|
BLAKE2b-256 | f15ebddc12161097feda1c56e1104b99b8ebedf55f03b29651430f761d4abae1 |
Hashes for flatterer-0.12.9-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 35101b8d94d4e6baf46238c81a5f03099c7a244b0b20715fbdfabc7cf1516656 |
|
MD5 | c0a5337ddbeae4bde44c980b2ead030b |
|
BLAKE2b-256 | cfdebb5d6a63241cf6161f5d800fd7e3168cde612ee10947b45d38721b218641 |
Hashes for flatterer-0.12.9-cp310-cp310-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | aaa23598ab55f227a80d24790e11e5d0b3428857be26b31672046cc55f699769 |
|
MD5 | 79d43ff2f107363c9e0e3cfe3b83cb92 |
|
BLAKE2b-256 | 3e329c173a93b00e6e8fe8843858293ff91ef961673bd39bb12a79a1914da783 |
Hashes for flatterer-0.12.9-cp310-cp310-macosx_10_7_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ede658711c87114faab7e4ebd6a1675e98ef07621f40fdefd56bd88cdb10d88f |
|
MD5 | 6f905df54706e8b16cf64528fd9bacda |
|
BLAKE2b-256 | e9a4f3889761a402d40188179f721eef2293b7501e5411585beb627c21115723 |
Hashes for flatterer-0.12.9-cp39-none-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 69257e1953f9051a8f9166bcaffcdfb94a44fdf81706cee792090e9fe239f140 |
|
MD5 | d77b7a454ed650af0644bc780e659038 |
|
BLAKE2b-256 | 24b9aec634ddf434c55a67c2e03266b6e5e97fb0bc0456609028acfea5913644 |
Hashes for flatterer-0.12.9-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c3790e1caa460d94acc6dac75bef27bae1ab88792b03520e32346feb74bb6d2f |
|
MD5 | c7804e5a03812df0516fda9412201911 |
|
BLAKE2b-256 | d8d9ce985c4b9bb0612a1080808b77a2d9ba0723728fc286dee6c9ff650ff2d3 |
Hashes for flatterer-0.12.9-cp39-cp39-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 83fd9485c1d9623f8e01c7f429e7b50b6722837691f00679771b18b2db07aebd |
|
MD5 | 003aa858f426a9f5bef617dc5f76e28f |
|
BLAKE2b-256 | d83fb16278d54997aebd67325a9631acb9cb3cf3090a497ae90e8778e2f87d6d |
Hashes for flatterer-0.12.9-cp39-cp39-macosx_10_7_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 73f2ed3390bb462017ae22a8f77e5e05beef81ca518b47c430cebd420eb3e360 |
|
MD5 | 9ca492f84d3b2a8825ab1b14ca03ac68 |
|
BLAKE2b-256 | d72bb43e4f895eac7b73a7e739ec9582bb16df4eefd1b1d169e77b481c16dfc4 |
Hashes for flatterer-0.12.9-cp38-none-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a923db10c58010f1aa0a6ab652dc98b995918b3412cebb3e485aa74b0d74f46f |
|
MD5 | fcf1e8888ce7ee7a2c5471d52f8d2771 |
|
BLAKE2b-256 | 6715d71ebda50015a84632b9f5dfac886cdea8133807e69ad54dadad1803cd71 |
Hashes for flatterer-0.12.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 38e7b28c51bbab13d0edc1510fd144ff689ddc58bfd8057bee9da37051ec3d3d |
|
MD5 | 9962f13a8b26192eb2a5d61bc2e2d26a |
|
BLAKE2b-256 | dcf3a4f3025d5f686b30c5b9b8f6ea4913ba6f5ad06714a0df26aea32ac3d4f3 |
Hashes for flatterer-0.12.9-cp38-cp38-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d15a56b863045844f7020adce2fd7cdd47b74a10940e6ea723ce9a6551062745 |
|
MD5 | 7e91fc83b3f649da0903b9a4ecf3f844 |
|
BLAKE2b-256 | caaaaf687cab110d46c7d19429eca2e80cfcde4d914f9464b5778bc2644ab4ff |
Hashes for flatterer-0.12.9-cp38-cp38-macosx_10_7_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ef302934a14dfae8c496594f397561ff106da191ee587ef3ad8fd506c1f609b9 |
|
MD5 | 62deb4e71a1a7ad5c8a0c6142ab57210 |
|
BLAKE2b-256 | 89bf5114b5619b2716960aa9a2ac0bfae75450aaba93288f568ec6b26796ef27 |
Hashes for flatterer-0.12.9-cp37-none-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b9ed6692e9dd0588093055eda82a0a1e5ececd3de093f5d96964eae894e5e79b |
|
MD5 | 3f641b3c9ceb60bb0da7d6b44bfa44d5 |
|
BLAKE2b-256 | 2ceca1b81c0fbf71666acee4967663ad1169a542b21daee8b672252720439fd8 |
Hashes for flatterer-0.12.9-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ace59c88edf420738724456737a037d7bfd0ea293d2fe4c7ca371b79bb8cb907 |
|
MD5 | c1685376af6e3af9937ae9f009191cbf |
|
BLAKE2b-256 | cf89713578ae676aabfda1d452dbec6aa69e29be322d8de9d838bbd953d2cb03 |
Hashes for flatterer-0.12.9-cp37-cp37m-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6c3e1ca77e02c81e8d715f96e56eb34f1b70d238b2e217b5a51b07a073b372b1 |
|
MD5 | c5e3370e4efc280aeaabf2e0693f84e9 |
|
BLAKE2b-256 | 97552ff1fb24e770b947d47c5f682e84cff873d1207817b0669ea0aaada8bc00 |
Hashes for flatterer-0.12.9-cp37-cp37m-macosx_10_7_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 55460a81cd108471878e72dbfa60622cda30fb737b17cf2707efb10daf497272 |
|
MD5 | 20bd3a43d6c8ff2fad10f6f94294ac4d |
|
BLAKE2b-256 | 23f343c4f1627bd6c52c1f758361e083ca8b0b8e5eb3dfbb65d54c34c8c911c6 |
Hashes for flatterer-0.12.9-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 89b1a4bfb174765877f41056288bfcc360a5cf4d50a3fb365dde0d2a08af59a1 |
|
MD5 | a522bf8114a3596f3aa548534042c36d |
|
BLAKE2b-256 | 4ddb78e25b1e92658a72f218db4d674b3b29c6a2ed270e51d03701b3910d0ba1 |
Hashes for flatterer-0.12.9-cp36-cp36m-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 15dc2af0e05ae1db891f336967c59e432a10bfdb7f7613c94e0046fdeecd317e |
|
MD5 | d7e3af7db9c773c7d9cb6c2f6bee8e82 |
|
BLAKE2b-256 | f69d1886784a359a2e6c5a9999b985d529be2c0fabdc84a50d3758b95efcbeb6 |
Hashes for flatterer-0.12.9-cp36-cp36m-macosx_10_7_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2fa4a26092aeae2b6b7c23c8a834cc75716dec8713ce4a14ef10c061e17a6e71 |
|
MD5 | f81dcd087436a3145e440bb90beb7501 |
|
BLAKE2b-256 | 9aa6b2867bd12904aac019721a9577a86ad8b42eb06114d972b1fd51be68dda0 |