Azure Schema Registry client library for Python

Azure Schema Registry is a schema repository service hosted by Azure Event Hubs, providing schema storage, versioning, and management. The registry is leveraged by serializers to reduce payload size while describing payload structure with schema identifiers rather than full schemas.

Source code | Package (PyPi) | API reference documentation | Samples | Changelog

Disclaimer

Azure SDK Python packages support for Python 2.7 is ending 01 January 2022. For more information and questions, please refer to https://github.com/Azure/azure-sdk-for-python/issues/20691

Getting started

Install the package

Install the Azure Schema Registry client library and Azure Identity client library for Python with pip:

pip install azure-schemaregistry azure-identity

Prerequisites:

To use this package, you must have:

• Azure subscription - Create a free account
• Azure Schema Registry
• Python 2.7, 3.6 or later - Install Python

Authenticate the client

Interaction with Schema Registry starts with an instance of SchemaRegistryClient class. You need the fully qualified namespace and AAD credential to instantiate the client object.

Create client using the azure-identity library:

from azure.schemaregistry import SchemaRegistryClient
from azure.identity import DefaultAzureCredential

credential = DefaultAzureCredential()
# Namespace should be similar to: '<your-eventhub-namespace>.servicebus.windows.net/'
fully_qualified_namespace = '<< FULLY QUALIFIED NAMESPACE OF THE SCHEMA REGISTRY >>'
schema_registry_client = SchemaRegistryClient(fully_qualified_namespace, credential)

Key concepts

• Schema: Schema is the organization or structure for data.

• SchemaRegistryClient: SchemaRegistryClient provides the API for storing and retrieving schemas in schema registry.

Examples

The following sections provide several code snippets covering some of the most common Schema Registry tasks, including:

• Register a schema
• Get the schema by id
• Get the id of a schema

Register a schema

Use SchemaRegistryClient.register_schema method to register a schema.

import os

from azure.identity import DefaultAzureCredential
from azure.schemaregistry import SchemaRegistryClient

token_credential = DefaultAzureCredential()
fully_qualified_namespace = os.environ['SCHEMA_REGISTRY_FULLY_QUALIFIED_NAMESPACE']
group_name = "<your-group-name>"
name = "<your-schema-name>"
format = "Avro"
schema_definition = """
{"namespace": "example.avro",
 "type": "record",
 "name": "User",
 "fields": [
     {"name": "name", "type": "string"},
     {"name": "favorite_number",  "type": ["int", "null"]},
     {"name": "favorite_color", "type": ["string", "null"]}
 ]
}
"""

schema_registry_client = SchemaRegistryClient(fully_qualified_namespace=fully_qualified_namespace, credential=token_credential)
with schema_registry_client:
    schema_properties = schema_registry_client.register_schema(group_name, name, schema_definition, format)
    id = schema_properties.id

Get the schema by id

Get the schema content and its properties by schema id.

import os

from azure.identity import DefaultAzureCredential
from azure.schemaregistry import SchemaRegistryClient

token_credential = DefaultAzureCredential()
fully_qualified_namespace = os.environ['SCHEMA_REGISTRY_FULLY_QUALIFIED_NAMESPACE']
id = '<your-schema-id>'

schema_registry_client = SchemaRegistryClient(fully_qualified_namespace=fully_qualified_namespace, credential=token_credential)
with schema_registry_client:
    schema = schema_registry_client.get_schema(id)
    schema_definition = schema.schema_definition

Get the id of a schema

Get the schema id of a schema by schema content and its properties.

import os

from azure.identity import DefaultAzureCredential
from azure.schemaregistry import SchemaRegistryClient

token_credential = DefaultAzureCredential()
fully_qualified_namespace = os.environ['SCHEMA_REGISTRY_FULLY_QUALIFIED_NAMESPACE']
group_name = "<your-group-name>"
name = "<your-schema-name>"
format = "Avro"
schema_definition = """
{"namespace": "example.avro",
 "type": "record",
 "name": "User",
 "fields": [
     {"name": "name", "type": "string"},
     {"name": "favorite_number",  "type": ["int", "null"]},
     {"name": "favorite_color", "type": ["string", "null"]}
 ]
}
"""

schema_registry_client = SchemaRegistryClient(fully_qualified_namespace=fully_qualified_namespace, credential=token_credential)
with schema_registry_client:
    schema_properties = schema_registry_client.register_schema(group_name, name, schema_definition, format)
    id = schema_properties.id

Troubleshooting

General

Schema Registry clients raise exceptions defined in Azure Core.

Logging

This library uses the standard logging library for logging. Basic information about HTTP sessions (URLs, headers, etc.) is logged at INFO level.

Detailed DEBUG level logging, including request/response bodies and unredacted headers, can be enabled on a client with the logging_enable argument:

import sys
import logging
from azure.schemaregistry import SchemaRegistryClient
from azure.identity import DefaultAzureCredential

# Create a logger for the SDK
logger = logging.getLogger('azure.schemaregistry')
logger.setLevel(logging.DEBUG)

# Configure a console output
handler = logging.StreamHandler(stream=sys.stdout)
logger.addHandler(handler)

credential = DefaultAzureCredential()
# This client will log detailed information about its HTTP sessions, at DEBUG level
schema_registry_client = SchemaRegistryClient("your_fully_qualified_namespace", credential, logging_enable=True)

Similarly, logging_enable can enable detailed logging for a single operation, even when it isn't enabled for the client:

schema_registry_client.get_schema(id, logging_enable=True)

Next steps

More sample code

Please take a look at the samples directory for detailed examples of how to use this library to register and retrieve schema to/from Schema Registry.

Event Hubs and Avro Serializer

We provide azure-schemaregistry-avroserializer library as serializer implementation to serialize/deserialize avro data integrated with azure-schemaregistry for automatic schema registration and retrieval. It integrates nicely with the EventHubs SDK. For more information and sample codes, please refer to the Azure Schema Registry Avro Serializer SDK.

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

Release History

1.0.0b3 (2021-10-05)

Breaking Changes

• get_schema_id method on sync and async SchemaRegistryClient has been renamed get_schema_properties.
• schema_id parameter in get_schema method on sync and async SchemaRegistryClient has been renamed id.
• register_schema and get_schema_properties methods on sync and async SchemaRegistryClient now take in the following parameters in the given order:
  • group_name, which has been renamed from schema_group
  • name, which has been renamed from schema_name
  • schema_definition, which has been renamed from schema_content
  • format, which has been renamed from serialization_type
• endpoint parameter in SchemaRegistryClient constructor has been renamed fully_qualified_namespace
• location instance variable in SchemaProperties has been removed.
• Schema and SchemaProperties no longer have positional parameters, as they will not be constructed by the user.

Other Changes

• Updated azure-core dependency to 1.19.0.
• Removed caching support of registered schemas so requests are sent to the service to register schemas, get schema properties, and get schemas.

1.0.0b2 (2021-08-17)

This version and all future versions will require Python 2.7 or Python 3.6+, Python 3.5 is no longer supported.

Features Added

• Support caching of registered schemas and send requests to the service only if the cache does not have the looked-up schema/schema ID.

1.0.0b1 (2020-09-09)

Version 1.0.0b1 is the first preview of our efforts to create a user-friendly and Pythonic client library for Azure Schema Registry.

New features

• SchemaRegistryClient is the top-level client class interacting with the Azure Schema Registry Service. It provides three methods:
  • register_schema: Store schema into the service.
  • get_schema: Get schema content and its properties by schema id.
  • get_schema_id: Get schema id and its properties by schema group, schema name, serialization type and schema content.