Azure Event Grid client library for Python

Azure Event Grid is a fully-managed intelligent event routing service that allows for uniform event consumption using a publish-subscribe model.

Source code | Package (PyPI) | API reference documentation | Product documentation | Samples | Changelog

Disclaimer

Azure SDK Python packages support for Python 2.7 is ending 01 January 2022. For more information and questions, please refer to https://github.com/Azure/azure-sdk-for-python/issues/20691

Getting started

Prerequisites

• Python 2.7, or 3.6 or later is required to use this package.
• You must have an Azure subscription and an Event Grid Topic resource to use this package. Follow this step-by-step tutorial to register the Event Grid resource provider and create Event Grid topics using the Azure portal. There is a similar tutorial using Azure CLI.

Install the package

Install the Azure Event Grid client library for Python with pip:

pip install azure-eventgrid

• An existing Event Grid topic or domain is required. You can create the resource using Azure Portal or Azure CLI

If you use Azure CLI, replace <resource-group-name> and <resource-name> with your own unique names.

Create an Event Grid Topic

az eventgrid topic --create --location <location> --resource-group <resource-group-name> --name <resource-name>

Create an Event Grid Domain

az eventgrid domain --create --location <location> --resource-group <resource-group-name> --name <resource-name>

Authenticate the client

In order to interact with the Event Grid service, you will need to create an instance of a client. An endpoint and credential are necessary to instantiate the client object.

Using Azure Active Directory (AAD)

Azure Event Grid provides integration with Azure Active Directory (Azure AD) for identity-based authentication of requests. With Azure AD, you can use role-based access control (RBAC) to grant access to your Azure Event Grid resources to users, groups, or applications.

To send events to a topic or domain with a TokenCredential, the authenticated identity should have the "EventGrid Data Sender" role assigned.

With the azure-identity package, you can seamlessly authorize requests in both development and production environments. To learn more about Azure Active Directory, see the azure-identity README.

For example, you can use DefaultAzureCredential to construct a client which will authenticate using Azure Active Directory:

from azure.identity import DefaultAzureCredential
from azure.eventgrid import EventGridPublisherClient, EventGridEvent

event = EventGridEvent(
    data={"team": "azure-sdk"},
    subject="Door1",
    event_type="Azure.Sdk.Demo",
    data_version="2.0"
)

credential = DefaultAzureCredential()
endpoint = os.environ["EG_TOPIC_HOSTNAME"]
client = EventGridPublisherClient(endpoint, credential)

client.send(event)

Looking up the endpoint

You can find the topic endpoint within the Event Grid Topic resource on the Azure portal. This will look like: "https://<event-grid-topic-name>.<topic-location>.eventgrid.azure.net/api/events"

Create the client with AzureKeyCredential

To use an Access key as the credential parameter, pass the key as a string into an instance of AzureKeyCredential.

Note: The Access Key may be found in the azure portal in the "Access Keys" menu of the Event Grid Topic resource. They may also be obtained via the azure CLI, or the azure-mgmt-eventgrid library. A guide for getting access keys can be found here.

from azure.core.credentials import AzureKeyCredential
from azure.eventgrid import EventGridPublisherClient

endpoint = "https://<name>.<region>.eventgrid.azure.net/api/events"
credential = AzureKeyCredential("<access_key>")
eg_publisher_client = EventGridPublisherClient(endpoint, credential)

Note: A client may also be authenticated via SAS signature, using the AzureSasCredential. A sample demonstrating this, is available here (async_version).

Note: The generate_sas method can be used to generate a shared access signature. A sample demonstrating this can be seen here.

Key concepts

Topic

A topic is a channel within the EventGrid service to send events. The event schema that a topic accepts is decided at topic creation time. If events of a schema type are sent to a topic that requires a different schema type, errors will be raised.

Domain

An event domain is a management tool for large numbers of Event Grid topics related to the same application. They allow you to publish events to thousands of topics. Domains also give you authorization and authentication control over each topic. For more information, visit Event domain overview.

When you create an event domain, a publishing endpoint for this domain is made available to you. This process is similar to creating an Event Grid Topic. The only difference is that, when publishing to a domain, you must specify the topic within the domain that you'd like the event to be delivered to.

Event schemas

An event is the smallest amount of information that fully describes something that happened in the system. When a custom topic or domain is created, you must specify the schema that will be used when publishing events.

Event Grid supports multiple schemas for encoding events.

Event Grid schema

While you may configure your topic to use a custom schema, it is more common to use the already-defined Event Grid schema. See the specifications and requirements here.

CloudEvents v1.0 schema

Another option is to use the CloudEvents v1.0 schema. CloudEvents is a Cloud Native Computing Foundation project which produces a specification for describing event data in a common way. The service summary of CloudEvents can be found here.

EventGridPublisherClient

EventGridPublisherClient provides operations to send event data to a topic hostname specified during client initialization.

Regardless of the schema that your topic or domain is configured to use, EventGridPublisherClient will be used to publish events to it. Use the send method publishing events.

The following formats of events are allowed to be sent:

• A list or a single instance of strongly typed EventGridEvents.

• A dict representation of a serialized EventGridEvent object.

• A list or a single instance of strongly typed CloudEvents.

• A dict representation of a serialized CloudEvent object.

• A dict representation of any Custom Schema.

Please have a look at the samples for detailed examples.

Note: It is important to know if your topic supports CloudEvents or EventGridEvents before publishing. If you send to a topic that does not support the schema of the event you are sending, send() will throw an exception.

System Topics

A system topic in Event Grid represents one or more events published by Azure services such as Azure Storage or Azure Event Hubs. For example, a system topic may represent all blob events or only blob creation and blob deletion events published for a specific storage account.

The names of the various event types for the system events published to Azure Event Grid are available in azure.eventgrid.SystemEventNames. For complete list of recognizable system topics, visit System Topics.

For more information about the key concepts on Event Grid, see Concepts in Azure Event Grid.

Event Grid on Kubernetes with Azure Arc

Event Grid on Kubernetes with Azure Arc is an offering that allows you to run Event Grid on your own Kubernetes cluster. This capability is enabled by the use of Azure Arc enabled Kubernetes. Through Azure Arc enabled Kubernetes, a supported Kubernetes cluster connects to Azure. Once connected, you are able to install Event Grid on it. Learn more about it here.

Examples

The following sections provide several code snippets covering some of the most common Event Grid tasks, including:

• Send an Event Grid Event
• Send a Cloud Event
• Send Multiple Events
• Send events as Dictionaries
• Consume a payload from storage queue
• Consume from ServiceBus

Send an Event Grid Event

This example publishes an Event Grid event.

import os
from azure.core.credentials import AzureKeyCredential
from azure.eventgrid import EventGridPublisherClient, EventGridEvent

key = os.environ["EG_ACCESS_KEY"]
endpoint = os.environ["EG_TOPIC_HOSTNAME"]

event = EventGridEvent(
    data={"team": "azure-sdk"},
    subject="Door1",
    event_type="Azure.Sdk.Demo",
    data_version="2.0"
)

credential = AzureKeyCredential(key)
client = EventGridPublisherClient(endpoint, credential)

client.send(event)

Send a Cloud Event

This example publishes a Cloud event.

import os
from azure.core.credentials import AzureKeyCredential
from azure.core.messaging import CloudEvent
from azure.eventgrid import EventGridPublisherClient

key = os.environ["CLOUD_ACCESS_KEY"]
endpoint = os.environ["CLOUD_TOPIC_HOSTNAME"]

event = CloudEvent(
    type="Azure.Sdk.Sample",
    source="https://egsample.dev/sampleevent",
    data={"team": "azure-sdk"}
)

credential = AzureKeyCredential(key)
client = EventGridPublisherClient(endpoint, credential)

client.send(event)

Send Multiple events

It is possible to send events as a batch when sending multiple events to a topic or a domain. This example sends a list of CloudEvents using the send method.

WARNING: When sending a list of multiple events at one time, iterating over and sending each event will not result in optimal performance. For best performance, it is highly recommended to send a list of events.

import os
from azure.core.credentials import AzureKeyCredential
from azure.core.messaging import CloudEvent
from azure.eventgrid import EventGridPublisherClient

key = os.environ["CLOUD_ACCESS_KEY"]
endpoint = os.environ["CLOUD_TOPIC_HOSTNAME"]

event0 = CloudEvent(
    type="Azure.Sdk.Sample",
    source="https://egsample.dev/sampleevent",
    data={"team": "azure-sdk"}
)
event1 = CloudEvent(
    type="Azure.Sdk.Sample",
    source="https://egsample.dev/sampleevent",
    data={"team2": "azure-eventgrid"}
)

events = [event0, event1]

credential = AzureKeyCredential(key)
client = EventGridPublisherClient(endpoint, credential)

client.send(events)

Send events as dictionaries

A dict representation of respective serialized models can also be used to publish CloudEvent(s) or EventGridEvent(s) apart from the strongly typed objects.

Use a dict-like representation to send to a topic with custom schema as shown below.

import os
import uuid
import datetime as dt
from msrest.serialization import UTC
from azure.core.credentials import AzureKeyCredential
from azure.eventgrid import EventGridPublisherClient

key = os.environ["CUSTOM_SCHEMA_ACCESS_KEY"]
endpoint = os.environ["CUSTOM_SCHEMA_TOPIC_HOSTNAME"]

event = custom_schema_event = {
    "customSubject": "sample",
    "customEventType": "sample.event",
    "customDataVersion": "2.0",
    "customId": uuid.uuid4(),
    "customEventTime": dt.datetime.now(UTC()).isoformat(),
    "customData": "sample data"
    }

credential = AzureKeyCredential(key)
client = EventGridPublisherClient(endpoint, credential)

client.send(event)

Consume from storage queue

This example consumes a message received from storage queue and deserializes it to a CloudEvent object.

from azure.core.messaging import CloudEvent
from azure.storage.queue import QueueServiceClient, BinaryBase64DecodePolicy
import os
import json

# all types of CloudEvents below produce same DeserializedEvent
connection_str = os.environ['STORAGE_QUEUE_CONN_STR']
queue_name = os.environ['STORAGE_QUEUE_NAME']

with QueueServiceClient.from_connection_string(connection_str) as qsc:
    payload =  qsc.get_queue_client(
        queue=queue_name,
        message_decode_policy=BinaryBase64DecodePolicy()
        ).peek_messages()

    ## deserialize payload into a list of typed Events
    events = [CloudEvent.from_dict(json.loads(msg.content)) for msg in payload]

Consume from servicebus

This example consumes a payload message received from ServiceBus and deserializes it to an EventGridEvent object.

from azure.eventgrid import EventGridEvent
from azure.servicebus import ServiceBusClient
import os
import json

# all types of EventGridEvents below produce same DeserializedEvent
connection_str = os.environ['SERVICE_BUS_CONN_STR']
queue_name = os.environ['SERVICE_BUS_QUEUE_NAME']

with ServiceBusClient.from_connection_string(connection_str) as sb_client:
    payload =  sb_client.get_queue_receiver(queue_name).receive_messages()

    ## deserialize payload into a list of typed Events
    events = [EventGridEvent.from_dict(json.loads(next(msg.body).decode('utf-8'))) for msg in payload]

Distributed Tracing with EventGrid

You can use opentelemetry for Python as usual with EventGrid since it's compatible with azure-core tracing integration.

Here is an example of using OpenTelemetry to trace sending a CloudEvent.

First, set OpenTelemetry as enabled tracing plugin for EventGrid.

from azure.core.settings import settings
from azure.core.tracing.ext.opentelemetry_span import OpenTelemetrySpan

settings.tracing_implementation = OpenTelemetrySpan

Regular open telemetry usage from here. See OpenTelemetry for details. This example uses a simple console exporter to export the traces. Any exporter can be used here including azure-monitor-opentelemetry-exporter, jaeger, zipkin etc.

from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import ConsoleSpanExporter
from opentelemetry.sdk.trace.export import SimpleSpanProcessor  # this requires opentelemtry >= 1.0.0

# Simple console exporter
exporter = ConsoleSpanExporter()

trace.set_tracer_provider(TracerProvider())
tracer = trace.get_tracer(__name__)
trace.get_tracer_provider().add_span_processor(
    SimpleSpanProcessor(exporter)
)

Once the tracer and exporter are set, please follow the example below to start collecting traces while using the send method from the EventGridPublisherClient to send a CloudEvent object.

import os
from azure.eventgrid import EventGridPublisherClient
from azure.core.messaging import CloudEvent
from azure.core.credentials import AzureKeyCredential

hostname = os.environ['CLOUD_TOPIC_HOSTNAME']
key = AzureKeyCredential(os.environ['CLOUD_ACCESS_KEY'])
cloud_event = CloudEvent(
    source = 'demo',
    type = 'sdk.demo',
    data = {'test': 'hello'},
)
with tracer.start_as_current_span(name="MyApplication"):
    client = EventGridPublisherClient(hostname, key)
    client.send(cloud_event)

Troubleshooting

• Enable azure.eventgrid logger to collect traces from the library.

General

Event Grid client library will raise exceptions defined in Azure Core.

Logging

This library uses the standard logging library for logging. Basic information about HTTP sessions (URLs, headers, etc.) is logged at INFO level.

Optional Configuration

Optional keyword arguments can be passed in at the client and per-operation level. The azure-core reference documentation describes available configurations for retries, logging, transport protocols, and more.

Next steps

The following section provides several code snippets illustrating common patterns used in the Event Grid Python API.

More sample code

These code samples show common champion scenario operations with the Azure Event Grid client library.

• Generate Shared Access Signature: sample_generate_sas.py

• Authenticate the client: sample_authentication.py (async_version)

• Publish events to a topic using SAS: sample_publish_events_to_a_topic_using_sas_credential_async.py (async_version)

• Publish Event Grid Events to a topic: sample_publish_eg_events_to_a_topic.py (async_version)

• Publish EventGrid Events to a domain topic: sample_publish_eg_events_to_a_domain_topic.py (async_version)

• Publish a Cloud Event: sample_publish_events_using_cloud_events_1.0_schema.py (async_version)

• Publish a Custom Schema: sample_publish_custom_schema_to_a_topic.py (async_version)

The following samples cover publishing and consuming dict representations of EventGridEvents and CloudEvents.

• Publish EventGridEvent as dict like representation: sample_publish_eg_event_using_dict.py (async_version)

• Publish CloudEvent as dict like representation: sample_publish_cloud_event_using_dict.py (async_version)

• Consume a Custom Payload of raw cloudevent data: sample_consume_custom_payload.py

More samples can be found here.

• More samples related to the send scenario can be seen here.
• To see more samples related to consuming a payload from different messaging services as a typed object, please visit Consume Samples

Additional documentation

For more extensive documentation on Azure Event Grid, see the Event Grid documentation on docs.microsoft.com.

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit cla.microsoft.com.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

Release History

4.6.0 (2021-10-05)

Features Added

• Added new enum values to SystemEvents.

4.5.0 (2021-08-10)

Features Added

• Added a new enum value Microsoft.ContainerService.NewKubernetesVersionAvailable to SystemEvents.
• Added a from_json method which now accepts storage QueueMessage, eventhub's EventData or ServiceBusMessage or simply json bytes to return an EventGridEvent

4.4.0 (2021-07-19)

• Bumped msrest dependency to 0.6.21 to align with mgmt package.

Features Added

• EventGridPublisherClient now supports Azure Active Directory (AAD) for authentication.

4.3.0 (2021-06-09)

New Features

• Added new event names related to blob inventory to the SystemEventNames enum.

Bug Fixes

• Replaced the ServiceBusDeadletterMessagesAvailableWithNoListenerEventName with the right value.

4.2.0 (2021-05-12)

New Features

• Added new event names to the SystemEventNames enum.

4.1.1 (2021-04-07)

Bug Fixes

• Improved the repr on EventGridEvent to show more meaningful text.

4.1.0 (2021-03-23)

New Features

• Added new SystemEventNames AcsChatThreadParticipantRemovedEventName, AcsChatThreadParticipantAddedEventName and AcsRecordingFileStatusUpdatedEventName.

4.0.0 (2021-03-09)

Note: This is the first stable release of our efforts to create a user-friendly and Pythonic client library for Azure EventGrid. Users migrating from v1.x are advised to view the migration guide.

New Features

• azure-eventgrid package now supports azure.core.messaging.CloudEvent which honors the CNCF CloudEvent spec.
• azure.eventgrid.SystemEventNames can be used to get the event model type mapping for system events.
• Implements the EventGridPublisherClient for the publish flow for EventGrid Events, CloudEvents and Custom schema events.

Breaking Changes

• azure.eventgrid.models namespace along with all the models in it are now removed.:
  • JSON documentation on the events is available here: https://docs.microsoft.com/azure/event-grid/system-topics
  • azure.eventgrid.SystemEventNames provides the list of available events name for easy switching.
• azure.eventgrid.event_grid_client.EventGridClient is now removed in favor of azure.eventgrid.EventGridPublisherClient.
• azure.eventgrid.event_grid_client.EventGridClientConfiguration is now removed.

2.0.0 (2021-03-09)

Disclaimer: v2.0.0 is functionally equivalent to v4.0.0. Users are advised to use v4.0.0 instead of this.

Breaking Changes

• ~azure.eventgrid.CloudEvent is now removed in favor of ~azure.core.messaging.CloudEvent.
• All the SystemEventNames related to Azure Communication Service starting with ACS**** are renamed to Acs*** to honor pascal case.

Features

• Added support for two new SystemEvents - ServiceBusDeadletterMessagesAvailablePeriodicNotificationsEventData and ServiceBusActiveMessagesAvailablePeriodicNotificationsEventData

2.0.0b5 (2021-02-10)

Breaking Changes

• EventGridSharedAccessSignatureCredential is deprecated in favor of AzureSasCredential.
• azure.eventgrid.models namespace along with all the models in it are now removed. azure.eventgrid.SystemEventNames can be used to get the event model type mapping.
• topic_hostname is renamed to endpoint in the EventGridPublisherClient.
• azure.eventgrid.generate_shared_access_signature method is now renamed to generate_sas.
• EventGridConsumeris now removed. Please see the samples to see how events can be deserialized.
• CustomEvent model is removed. Dictionaries must be used to send a custom schema.

Bug Fixes

• EventGridEvent has two additional required positional parameters namely, data and data_version.
• EventGridPublisherClient now appropriately throws a ValueError if an invalid credential is passed during initialization.

2.0.0b4 (2020-11-11)

Bug Fixes

• TypeError is raised when bytes are passed to an EventGridEvent.

2.0.0b3 (2020-10-05)

Feature

• Added support for Keyvault Event Types
• Added distributed tracing support for CloudEvents

2.0.0b2 (2020-09-24)

Features

• Added support for Azure Communication Services event types.

2.0.0b1 (2020-09-08)

Features

• Version (2.0.0b1) is the first preview of our efforts to create a user-friendly and Pythonic client library for Azure EventGrid. For more information about this, and preview releases of other Azure SDK libraries, please visit https://azure.github.io/azure-sdk/releases/latest/python.html.
• Added Support for CloudEvents.
• Implements the EventGridPublisherClient for the publish flow for EventGrid Events, CloudEvents and CustomEvents.
• Implements the EventGridConsumer for the consume flow of the events.

1.3.0 (2019-05-20)

• Event Schemas for new event types from IotHub, Media Services, Container Registry, Maps, and AppConfiguration services.

1.2.0 (2018-08-28)

• Event Schemas for new events (IotHub DeviceConnected and DeviceDisconnected events, Resource events related to actions), and breaking changes to the schema for IotHub DeviceCreated event and IotHub DeviceDeleted event.

1.1.0 (2018-05-24)

• Event Schemas for EventGrid subscription validation event, Azure Media events, and ServiceBus events.

1.0.0 (2018-04-26)

General Breaking changes

This version uses a next-generation code generator that might introduce breaking changes.

• Model signatures now use only keyword-argument syntax. All positional arguments must be re-written as keyword-arguments. To keep auto-completion in most cases, models are now generated for Python 2 and Python 3. Python 3 uses the "*" syntax for keyword-only arguments.
• Enum types now use the "str" mixin (class AzureEnum(str, Enum)) to improve the behavior when unrecognized enum values are encountered. While this is not a breaking change, the distinctions are important, and are documented here: https://docs.python.org/3/library/enum.html#others At a glance:
  • "is" should not be used at all.
  • "format" will return the string value, where "%s" string formatting will return NameOfEnum.stringvalue. Format syntax should be prefered.
• New Long Running Operation:
  • Return type changes from msrestazure.azure_operation.AzureOperationPoller to msrest.polling.LROPoller. External API is the same.
  • Return type is now always a msrest.polling.LROPoller, regardless of the optional parameters used.
  • The behavior has changed when using raw=True. Instead of returning the initial call result as ClientRawResponse, without polling, now this returns an LROPoller. After polling, the final resource will be returned as a ClientRawResponse.
  • New polling parameter. The default behavior is Polling=True which will poll using ARM algorithm. When Polling=False, the response of the initial call will be returned without polling.
  • polling parameter accepts instances of subclasses of msrest.polling.PollingMethod.
  • add_done_callback will no longer raise if called after polling is finished, but will instead execute the callback right away.

Features

• Client class can be used as a context manager to keep the underlying HTTP session open for performance
• Support for consuming Azure Container Registry events and Azure IoT Hub events published to Event Grid.

0.1.0 (2018-01-30)

• Initial Release