The Datadog AWS Lambda Layer
Project description
datadog-lambda-layer-python
Datadog Lambda Layer for Python (2.7, 3.6 and 3.7) enables custom metric submission from AWS Lambda functions, and distributed tracing between serverful and serverless environments.
Installation
Datadog Lambda Layer can be added to a Lambda function via AWS Lambda console, AWS CLI or Serverless Framework using the following ARN.
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-<PYTHON_RUNTIME>:<VERSION>
Replace <AWS_REGION> with the AWS region where your Lambda function is published to. Replace <PYTHON_RUNTIME> with Datadog-Python27 with Datadog-Python36 or Datadog-Python37 that matches your Lambda's Python runtime. Replace <VERSION> with the latest layer version that can be found from CHANGELOG. For example,
arn:aws:lambda:us-east-1:464622532012:layer:Datadog-Python37:1
PyPI
When developing your Lambda function locally where AWS Layer doesn't work, the Datadog Lambda layer can be installed from PyPI by pip install datadog-lambda or adding datadog-lambda to your project's requirements.txt.
The minor version of the datadog-lambda package always match the layer version. E.g., datadog-lambda v0.5.0 matches the content in layer version 5.
Environment Variables
The Datadog API must be defined as an environment variable via AWS CLI or Serverless Framework:
- DD_API_KEY or DD_KMS_API_KEY (if encrypted by KMS)
Set the following Datadog environment variable to datadoghq.eu to send your data to the Datadog EU site.
- DD_SITE
If your Lambda function powers a performance-critical task (e.g., a consumer-facing API). You can avoid the added latencies of metric-submitting API calls, by setting the following Datadog environment variable to True. Custom metrics will be submitted asynchronously through CloudWatch Logs and the Datadog log forwarder.
- DD_FLUSH_TO_LOG
The Serverless Framework
If your Lambda function is deployed using the Serverless Framework, refer to this sample serverless.yml.
provider:
name: aws
runtime: python3.7
tracing:
lambda: true
apiGateway: true
functions:
hello:
handler: handler.hello
events:
- http:
path: hello
method: get
layers:
- arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-<PYTHON_RUNTIME>:<VERSION>
environment:
DD_API_KEY: <DD_API_KEY>
Basic Usage
import requests
from datadog_lambda.wrapper import datadog_lambda_wrapper
from datadog_lambda.metric import lambda_metric
@datadog_lambda_wrapper
def lambda_handler(event, context):
lambda_metric("my_metric", 10, tags=['tag:value'])
requests.get("https://www.datadoghq.com")
Custom Metrics
Custom metrics can be submitted using lambda_metric and the Lambda handler function needs to be decorated with @datadog_lambda_wrapper. The metrics are submitted as distribution metrics.
IMPORTANT NOTE: If you have already been submitting the same custom metric as non-distribution metric (e.g., gauge, count, or histogram) without using the Datadog Lambda Layer, you MUST pick a new metric name to use for lambda_metric. Otherwise that existing metric will be converted to a distribution metric and the historical data prior to the conversion will be no longer queryable.
from datadog_lambda.wrapper import datadog_lambda_wrapper
from datadog_lambda.metric import lambda_metric
@datadog_lambda_wrapper
def lambda_handler(event, context):
lambda_metric(
"coffee_house.order_value", # metric
12.45, # value
tags=['product:latte', 'order:online'] # tags
)
VPC
If your Lambda function is associated with a VPC, you need to ensure it has access to the public internet.
Distributed Tracing
Distributed tracing allows you to propagate a trace context from a service running on a host to a service running on AWS Lambda, and vice versa, so you can see performance end-to-end. Linking is implemented by injecting Datadog trace context into the HTTP request headers.
Distributed tracing headers are language agnostic, e.g., a trace can be propagated between a Java service running on a host to a Lambda function written in Python.
Because the trace context is propagated through HTTP request headers, the Lambda function needs to be triggered by AWS API Gateway or AWS Application Load Balancer.
To enable this feature, you simple need to decorate your Lambda handler function with @datadog_lambda_wrapper.
import requests
from datadog_lambda.wrapper import datadog_lambda_wrapper
@datadog_lambda_wrapper
def lambda_handler(event, context):
requests.get("https://www.datadoghq.com")
Note, the Datadog Lambda Layer is only needed to enable distributed tracing between Lambda and non-Lambda services. For standalone Lambda functions, traces can be found in Datadog APM after configuring the X-Ray integration.
Patching
By default, widely used HTTP client libraries, such as requests, urllib2 and urllib.request are patched automatically to inject Datadog trace context into outgoing requests.
You can also manually retrieve the Datadog trace context (i.e., http headers in a Python dict) and inject it to request headers when needed.
import requests
from datadog_lambda.wrapper import datadog_lambda_wrapper
from datadog_lambda.tracing import get_dd_trace_context
@datadog_lambda_wrapper
def lambda_handler(event, context):
headers = get_dd_trace_context()
requests.get("https://www.datadoghq.com", headers=headers)
Sampling
The traces for your Lambda function are converted by Datadog from AWS X-Ray traces. X-Ray needs to sample the traces that the Datadog tracing agent decides to sample, in order to collect as many complete traces as possible. You can create X-Ray sampling rules to ensure requests with header x-datadog-sampling-priority:1 or x-datadog-sampling-priority:2 via API Gateway always get sampled by X-Ray.
These rules can be created using the following AWS CLI command.
aws xray create-sampling-rule --cli-input-json file://datadog-sampling-priority-1.json
aws xray create-sampling-rule --cli-input-json file://datadog-sampling-priority-2.json
The file content for datadog-sampling-priority-1.json:
{
"SamplingRule": {
"RuleName": "Datadog-Sampling-Priority-1",
"ResourceARN": "*",
"Priority": 9998,
"FixedRate": 1,
"ReservoirSize": 100,
"ServiceName": "*",
"ServiceType": "AWS::APIGateway::Stage",
"Host": "*",
"HTTPMethod": "*",
"URLPath": "*",
"Version": 1,
"Attributes": {
"x-datadog-sampling-priority": "1"
}
}
}
The file content for datadog-sampling-priority-2.json:
{
"SamplingRule": {
"RuleName": "Datadog-Sampling-Priority-2",
"ResourceARN": "*",
"Priority": 9999,
"FixedRate": 1,
"ReservoirSize": 100,
"ServiceName": "*",
"ServiceType": "AWS::APIGateway::Stage",
"Host": "*",
"HTTPMethod": "*",
"URLPath": "*",
"Version": 1,
"Attributes": {
"x-datadog-sampling-priority": "2"
}
}
}
Non-proxy integration
If your Lambda function is triggered by API Gateway via the non-proxy integration, then you have to set up a mapping template, which passes the Datadog trace context from the incoming HTTP request headers to the Lambda function via the event object.
If your Lambda function is deployed by the Serverless Framework, such a mapping template gets created by default.
Opening Issues
If you encounter a bug with this package, we want to hear about it. Before opening a new issue, search the existing issues to avoid duplicates.
When opening an issue, include the Datadog Lambda Layer version, Python version, and stack trace if available. In addition, include the steps to reproduce when appropriate.
You can also open an issue for a feature request.
Contributing
If you find an issue with this package and have a fix, please feel free to open a pull request following the procedures.
License
Unless explicitly stated otherwise all files in this repository are licensed under the Apache License Version 2.0.
This product includes software developed at Datadog (https://www.datadoghq.com/). Copyright 2019 Datadog, Inc.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file datadog_lambda-0.5.0.tar.gz.
File metadata
- Download URL: datadog_lambda-0.5.0.tar.gz
- Upload date:
- Size: 8.4 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/41.0.1 requests-toolbelt/0.9.1 tqdm/4.32.2 CPython/2.7.15
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 096d2412f3000eb7a55ab49ca774621d77b7c85b4657b3bd3fe8bd40b9d62c32 |
|
| MD5 | c69eefefebf9f744455e2e66d21a6a86 |
|
| BLAKE2b-256 | 08a12cebab57af1139d2ee816218ff5a44fbd25113c4302f22bae77b9bba425b |
File details
Details for the file datadog_lambda-0.5.0-py2-none-any.whl.
File metadata
- Download URL: datadog_lambda-0.5.0-py2-none-any.whl
- Upload date:
- Size: 15.3 kB
- Tags: Python 2
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/41.0.1 requests-toolbelt/0.9.1 tqdm/4.32.2 CPython/2.7.15
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 185c10841696ec3e3021dfe48bdc1f06c256075d231b30ff455b340e328568ca |
|
| MD5 | ea76169a1292a58a49ec77cfe6729464 |
|
| BLAKE2b-256 | f3a01d7ead4e6b8d3e5cfece81141ad99b38b89efc7b278a89e965fb6ee14293 |
