Skip to main content

No project description provided

Project description

SQLMesh logo

SQLMesh is a next-generation data transformation and modeling framework that is backwards compatible with dbt. It aims to be easy to use, correct, and efficient.

SQLMesh enables data teams to efficiently run and deploy data transformations written in SQL or Python.

It is more than just a dbt alternative.

Architecture Diagram

Core Features

SQLMesh Plan Mode

Get instant SQL impact analysis of your changes, whether in the CLI or in SQLMesh Plan Mode

Virtual Data Environments
Efficiency and Testing

Running this command will generate a unit test file in the tests/ folder: test_stg_payments.yaml

Runs a live query to generate the expected output of the model

sqlmesh create_test tcloud_demo.stg_payments --query tcloud_demo.seed_raw_payments "select * from tcloud_demo.seed_raw_payments limit 5"

# run the unit test
sqlmesh test
MODEL (
  name tcloud_demo.stg_payments,
  cron '@daily',
  grain payment_id,
  audits (UNIQUE_VALUES(columns = (
      payment_id
  )), NOT_NULL(columns = (
      payment_id
  )))
);

SELECT
    id AS payment_id,
    order_id,
    payment_method,
    amount / 100 AS amount, /* `amount` is currently stored in cents, so we convert it to dollars */
    'new_column' AS new_column, /* non-breaking change example  */
FROM tcloud_demo.seed_raw_payments
test_stg_payments:
model: tcloud_demo.stg_payments
inputs:
    tcloud_demo.seed_raw_payments:
    - id: 66
    order_id: 58
    payment_method: coupon
    amount: 1800
    - id: 27
    order_id: 24
    payment_method: coupon
    amount: 2600
    - id: 30
    order_id: 25
    payment_method: coupon
    amount: 1600
    - id: 109
    order_id: 95
    payment_method: coupon
    amount: 2400
    - id: 3
    order_id: 3
    payment_method: coupon
    amount: 100
outputs:
    query:
    - payment_id: 66
    order_id: 58
    payment_method: coupon
    amount: 18.0
    new_column: new_column
    - payment_id: 27
    order_id: 24
    payment_method: coupon
    amount: 26.0
    new_column: new_column
    - payment_id: 30
    order_id: 25
    payment_method: coupon
    amount: 16.0
    new_column: new_column
    - payment_id: 109
    order_id: 95
    payment_method: coupon
    amount: 24.0
    new_column: new_column
    - payment_id: 3
    order_id: 3
    payment_method: coupon
    amount: 1.0
    new_column: new_column
Take SQL Anywhere Write SQL in any dialect and SQLMesh will transpile it to your target SQL dialect on the fly before sending it to the warehouse. Transpile Example

For more information, check out the website and documentation.

Getting Started

Install SQLMesh through pypi by running:

mkdir sqlmesh-example
cd sqlmesh-example
python -m venv .env
source .env/bin/activate
pip install sqlmesh
sqlmesh init duckdb # get started right away with a local duckdb instance

Follow the quickstart guide to learn how to use SQLMesh. You already have a head start!

Join Our Community

We want to ship better data with you. Connect with us in the following ways:

Contribution

Contributions in the form of issues or pull requests are greatly appreciated. Read more on how to contribute to SQLMesh open source.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

sqlmesh-0.136.1.tar.gz (26.4 MB view details)

Uploaded Source

Built Distribution

sqlmesh-0.136.1-py3-none-any.whl (1.9 MB view details)

Uploaded Python 3

File details

Details for the file sqlmesh-0.136.1.tar.gz.

File metadata

  • Download URL: sqlmesh-0.136.1.tar.gz
  • Upload date:
  • Size: 26.4 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.10.15

File hashes

Hashes for sqlmesh-0.136.1.tar.gz
Algorithm Hash digest
SHA256 ffb90a9281867f9f4c040a55938871c6552ca6a97ff975e12a22439798f2559f
MD5 86de3fe180051080846d0980c874dfbf
BLAKE2b-256 7e99cdc37241ceeaf8e592761ae4bdd7d4b56bbd01e8da07f47cf5f3e73422b8

See more details on using hashes here.

File details

Details for the file sqlmesh-0.136.1-py3-none-any.whl.

File metadata

  • Download URL: sqlmesh-0.136.1-py3-none-any.whl
  • Upload date:
  • Size: 1.9 MB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.10.15

File hashes

Hashes for sqlmesh-0.136.1-py3-none-any.whl
Algorithm Hash digest
SHA256 cde9fa29cad7224d0f75548523cd961118ce2dcce54383651729716fa7db35d2
MD5 c2f5e0b3e6e555a42d51ab7959d93c03
BLAKE2b-256 44e61083379b7e77845e0058371f3267fea6f4edd402792834f0bd9787355e34

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page