Databricks SQL Connector for Python
Project description
Databricks SQL Connector for Python
The Databricks SQL Connector for Python allows you to develop Python applications that connect to Databricks clusters and SQL warehouses. It is a Thrift-based client with no dependencies on ODBC or JDBC. It conforms to the Python DB API 2.0 specification and exposes a SQLAlchemy dialect for use with tools like pandas and alembic which use SQLAlchemy to execute DDL. Use pip install databricks-sql-connector[sqlalchemy] to install with SQLAlchemy's dependencies. pip install databricks-sql-connector[alembic] will install alembic's dependencies.
This connector uses Arrow as the data-exchange format, and supports APIs to directly fetch Arrow tables. Arrow tables are wrapped in the ArrowQueue class to provide a natural API to get several rows at a time.
You are welcome to file an issue here for general use cases. You can also contact Databricks Support here.
Requirements
Python 3.8 or above is required.
Documentation
For the latest documentation, see
Quickstart
Install the library with pip install databricks-sql-connector
Note: Don't hard-code authentication secrets into your Python. Use environment variables
export DATABRICKS_HOST=********.databricks.com
export DATABRICKS_HTTP_PATH=/sql/1.0/endpoints/****************
export DATABRICKS_TOKEN=dapi********************************
Example usage:
import os
from databricks import sql
host = os.getenv("DATABRICKS_HOST")
http_path = os.getenv("DATABRICKS_HTTP_PATH")
access_token = os.getenv("DATABRICKS_TOKEN")
connection = sql.connect(
server_hostname=host,
http_path=http_path,
access_token=access_token)
cursor = connection.cursor()
cursor.execute('SELECT :param `p`, * FROM RANGE(10)', {"param": "foo"})
result = cursor.fetchall()
for row in result:
print(row)
cursor.close()
connection.close()
In the above example:
server-hostnameis the Databricks instance host name.http-pathis the HTTP Path either to a Databricks SQL endpoint (e.g. /sql/1.0/endpoints/1234567890abcdef), or to a Databricks Runtime interactive cluster (e.g. /sql/protocolv1/o/1234567890123456/1234-123456-slid123)personal-access-tokenis the Databricks Personal Access Token for the account that will execute commands and queries
Contributing
See CONTRIBUTING.md
License
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file databricks_sql_connector-3.0.3.tar.gz.
File metadata
- Download URL: databricks_sql_connector-3.0.3.tar.gz
- Upload date:
- Size: 409.3 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.7.1 CPython/3.10.2 Linux/6.2.0-1019-azure
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 961817067989537996b8daa9f8fc8baf1654884acc1b550448a2c684fa23a1d8 |
|
| MD5 | 5127dfebe47727c2bb51728a53909bcf |
|
| BLAKE2b-256 | 116502a8aef006b39228632f44f6d274f83f77309ebb6834cdaa84076889ee4b |
File details
Details for the file databricks_sql_connector-3.0.3-py3-none-any.whl.
File metadata
- Download URL: databricks_sql_connector-3.0.3-py3-none-any.whl
- Upload date:
- Size: 426.1 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.7.1 CPython/3.10.2 Linux/6.2.0-1019-azure
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 8b055cd706b5dbb9d4901d389b3bce42d924c1f6b377ae6396e6c98b78e4dcb2 |
|
| MD5 | df3e3987a3e7e18035dcfb4a08ec231f |
|
| BLAKE2b-256 | 8edc1a543acd93bd92b5a010a3aca464f1ac67b0f7d6f8d5960b2f2a8e9e323d |
