Azure Machine Learning Model Monitoring SDK

The azure-ai-mlmonitoring package provides an SDK to enable Model Data Collector (MDC) for custom logging allows customers to collect data at arbitrary points in their data pre-processing pipeline. Customers can leverage SDK in score.py to log data to desired sink before, during, and after any data transformations.

Start by importing the azure-ai-mlmonitoring package in score.py

from azure.ai.mlmonitoring import Collector

def init():
  global inputs_collector, predictions_collector

  # instantiate collectors with appropriate names
  inputs_collector = Collector(name='inputs')                    
  predictions_collector = Collector(name='predictions')

def run(data): 
  input_df = pd.DataFrame(data)  

  # collect input data, store correlation_context
  correlation_context = inputs_collector.collect(data=input_df) 

  # perform scoring
  predictions = model.predict(input_df) 

  # collect predictions data, pass in correlation_context so inputs and predictions data can be correlated later
  predictions_collector.collect(data=predictions, correlation_context=correlation_context)

Create deployment with custom logging enabled and azure-ai-mlmonitoring package.

#source ../configs/model-data-collector/data-storage-basic-OnlineDeployment.YAML
$schema: http://azureml/sdk-2-0/OnlineDeployment.json

endpoint_name: my_endpoint #unchanged
name: blue #unchanged
model: azureml:my-model-m1:1 #azureml:models/<name>:<version> #unchanged
environment: azureml:env-m1:1 #unchanged
data_collector:
  enabled:
      custom: true

Change Log

v0.1.0a1 (2023.1.4)

New Features

• Support model data collection for pandas Dataframe.