Skip to main content

Microsoft Azure Storage Blob ChangeFeed Client Library for Python

Project description

Azure Storage Blob ChangeFeed client library for Python

This preview package for Python enables users to get blob change feed events. These events can be lazily generated, iterated by page, retrieved for a specific time interval, or iterated from a specific continuation token.

Source code | Package (PyPi) | API reference documentation | Product documentation | Samples

Getting started

Prerequisites

Install the package

Install the Azure Storage Blob ChangeFeed client library for Python with pip:

pip install azure-storage-blob-changefeed --pre

Create a storage account

If you wish to create a new storage account, you can use the Azure Portal, Azure PowerShell, or Azure CLI:

# Create a new resource group to hold the storage account -
# if using an existing resource group, skip this step
az group create --name my-resource-group --location westus2

# Create the storage account
az storage account create -n my-storage-account-name -g my-resource-group --hierarchical-namespace true

Authenticate the client

Interaction with Blob ChangeFeed client starts with an instance of the ChangeFeedClient class. You need an existing storage account, its URL, and a credential to instantiate the client object.

Get credentials

To authenticate the client you have a few options:

  1. Use a SAS token string
  2. Use an account shared access key
  3. Use a token credential from azure.identity

Alternatively, you can authenticate with a storage connection string using the from_connection_string method. See example: Client creation with a connection string.

You can omit the credential if your account URL already has a SAS token.

Create client

Once you have your account URL and credentials ready, you can create the ChangeFeedClient:

from azure.storage.blob.changefeed import ChangeFeedClient

service = ChangeFeedClient(account_url="https://<my-storage-account-name>.dfs.core.windows.net/", credential=credential)

Key concepts

Clients

The Blob ChangeFeed SDK provides one client:

  • ChangeFeedClient: this client allows you to get change feed events by page, get all change feed events, get events in a time range, start listing events with a continuation token.

Examples

The following sections provide several code snippets covering some of the most common Storage Blob ChangeFeed, including:

Client creation with a connection string

Create the ChangeFeedClient using the connection string to your Azure Storage account.

from azure.storage.blob.changefeed import ChangeFeedClient

service = ChangeFeedClient.from_connection_string(conn_str="my_connection_string")

Enumerating Events Within a Time Range

List all events within a time range.

from datetime import datetime
from azure.storage.blob.changefeed import ChangeFeedClient

cf_client = ChangeFeedClient("http://{}.blob.core.windows.net".format("YOUR_ACCOUNT_NAME"),
                             credential="Your_ACCOUNT_KEY")
start_time = datetime(2020, 1, 6)
end_time = datetime(2020, 3, 4)
change_feed = cf_client.list_changes(start_time=start_time, end_time=end_time)

# print range of events
for event in change_feed:
    print(event)

Enumerating All Events

List all events.

from azure.storage.blob.changefeed import ChangeFeedClient

cf_client = ChangeFeedClient("http://{}.blob.core.windows.net".format("YOUR_ACCOUNT_NAME"),
                             credential="Your_ACCOUNT_KEY")
change_feed = cf_client.list_changes()

# print all events
for event in change_feed:
    print(event)

Enumerating Events by Page

List events by page.

from azure.storage.blob.changefeed import ChangeFeedClient

cf_client = ChangeFeedClient("http://{}.blob.core.windows.net".format("YOUR_ACCOUNT_NAME"),
                             credential="Your_ACCOUNT_KEY")

change_feed = cf_client.list_changes().by_page()

# print first page of events
change_feed_page1 = next(change_feed)
for event in change_feed_page1:
    print(event)

Troubleshooting

Logging

This library uses the standard logging library for logging. Basic information about HTTP sessions (URLs, headers, etc.) is logged at INFO level.

Detailed DEBUG level logging, including request/response bodies and unredacted headers, can be enabled on a client with the logging_enable argument:

import sys
import logging
from azure.storage.blob.changefeed import ChangeFeedClient

# Create a logger for the 'azure.storage.blob.changefeed' SDK
logger = logging.getLogger('azure.storage')
logger.setLevel(logging.DEBUG)

# Configure a console output
handler = logging.StreamHandler(stream=sys.stdout)
logger.addHandler(handler)

# This client will log detailed information about its HTTP sessions, at DEBUG level
service_client = ChangeFeedClient.from_connection_string("your_connection_string", logging_enable=True)

Next steps

More sample code

Get started with our Azure Blob ChangeFeed samples.

Several Storage Blob ChangeFeed Python SDK samples are available to you in the SDK's GitHub repository. These samples provide example code for additional scenarios commonly encountered while working with Blob ChangeFeed:

  • change_feed_samples.py - Examples for authenticating and operating on the client:
    • list events by page
    • list all events
    • list events in a time range
    • list events starting from a continuation token

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

azure-storage-blob-changefeed-12.0.0b1.zip (33.6 kB view details)

Uploaded Source

Built Distribution

azure_storage_blob_changefeed-12.0.0b1-py2.py3-none-any.whl (12.0 kB view details)

Uploaded Python 2 Python 3

File details

Details for the file azure-storage-blob-changefeed-12.0.0b1.zip.

File metadata

  • Download URL: azure-storage-blob-changefeed-12.0.0b1.zip
  • Upload date:
  • Size: 33.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/41.2.0 requests-toolbelt/0.9.1 tqdm/4.47.0 CPython/3.8.3

File hashes

Hashes for azure-storage-blob-changefeed-12.0.0b1.zip
Algorithm Hash digest
SHA256 5ab5834f779144c0ccb60d27406cda30a7459cac2ca79d0e38f6f7b49cee7949
MD5 131e2330730a15f683351a89fb5c8e58
BLAKE2b-256 d8154a9e998d2d20167bd83bbcce1f3b76a7049b92e5d3f2cee1d8044b3c04f9

See more details on using hashes here.

File details

Details for the file azure_storage_blob_changefeed-12.0.0b1-py2.py3-none-any.whl.

File metadata

File hashes

Hashes for azure_storage_blob_changefeed-12.0.0b1-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 0d864e2389734750603b02d39c8f9048b5d9b248723f4af43d81d4eb8fb1c225
MD5 2fad899c3f6b323660a6f78957ff3714
BLAKE2b-256 2c7325511221267d1f2d2076c3107a41526a2adb7ccc9ea5d4d45aec1aeefb18

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page