A collection of command line tools for crate devs
Project description
A collection of command line tools for Crate. Most of these tools output JSON. In order to pretty-print the output or filter it using jq is recommended.
Target audience are mostly developers of Crate and not users.
Install 💾
Python >= 3.5.1 is required to use the command line tools.
Install them using pip:
pip install cr8
(Users of Python 3.5.0 might run into this AssertionError.)
Usage
The main binary is called cr8 which contains a couple of sub-commands.
Use cr8 -h or cr8 <subcommand> -h to get a more detailed usage description.
The included sub-commands are described in more detail below:
Sub-commands
timeit 🕐
A tool that can be used to measure the runtime of a given SQL statement on a cluster:
>>> echo "select name from sys.cluster" | cr8 timeit localhost:4200
{
"bulk_size": null,
"concurrency": 1,
"ended": ...
"runtime_stats": {
...
},
"started": ...
"statement": "select name from sys.cluster\n",
"version_info": {
"hash": "4713a5498fadf14b2359e13c641e6734f8189dc5",
"number": "0.55.0"
}
}
insert-fake-data
A tool that can be used to fill a table with random data. The script will generate the records using faker.
For example given the table as follows:
create table demo (
name string,
country string
);
The following command can be used to insert 100k records:
>>> cr8 insert-fake-data localhost:4200 demo 1000
Found schema:
{
"country": "string",
"name": "string"
}
Using insert statement:
insert into demo ("country", "name") values (?, ?)
Will make 1 requests with a bulk size of 1000
Generating fake data and executing inserts
<BLANKLINE>
It will automatically read the schema from the table and map the columns to faker providers and insert the give number of records.
(Currently only top-level string columns are supported)
insert-json
insert-json can be used to insert records from a JSON file:
>>> cat tests/demo.json | cr8 insert-json demo --hosts localhost:4200
Executing requests async bulk_size=1000 concurrency=100
{
"max": ...,
"mean": ...,
"min": ...,
"n": 1
}
Or simply print the insert statement generated from a JSON string:
>>> echo '{"name": "Arthur"}' | cr8 insert-json mytable
('insert into mytable ("name") values (?)', ['Arthur'])
...
If a Crate host is provided the insert statement will be executed as well.
insert-blob
A tool to upload a file into a blob table:
>>> cr8 insert-blob localhost:4200 blobtable specs/sample.toml http://localhost:44200/_blobs/blobtable/cd1ebfd02ae167efd65df7fe346c8388ab1b8f91
run-spec
A tool to run benchmarks against a cluster and store the result in another cluster. The benchmark itself is defined in a spec file which defines setup, benchmark and teardown instructions.
The instructions itself are just SQL statements (or files containing SQL statements).
In the specs folder is an example spec file.
Usage:
>>> cr8 run-spec specs/sample.toml localhost:44200 -r localhost:44200
Running setUp
Running benchmark
{'concurrency': 2,
'iterations': 1000,
'statement': 'select count(*) from countries'}
{'bulk_size': None,
'concurrency': 2,
'ended': ...,
'runtime_stats': {...
'started': ...,
'statement': 'select count(*) from countries',
'version_info': {'hash': '4713a5498fadf14b2359e13c641e6734f8189dc5',
'number': '0.55.0'}}
<BLANKLINE>
Running tearDown
<BLANKLINE>
-r is optional and can be used to save the benchmark result into a cluster. The cluster must contain the table specified in sql/benchmarks_table.sql.
Development ☢
Tests are run using python setup.py test.
To get a sandboxed environment with all dependencies installed one can either use venv or buildout:
venv
Create a new virtualenv using venv and active it:
python -m venv .venv source .venv/bin/activate
Install the cr8 package using pip:
python -m pip install -e .
Run cr8:
cr8 -h
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
