Skip to main content

CDK Construct Library to automatically instrument Python and Node Lambda functions with Datadog

Project description

Datadog CDK Constructs

NPM PyPI Slack License

Use this Datadog CDK Construct Library to deploy serverless applications using AWS CDK .

This CDK library automatically configures ingestion of metrics, traces, and logs from your serverless applications by:

  • Installing and configuring the Datadog Lambda library for your Python and Node.js Lambda functions.
  • Enabling the collection of traces and custom metrics from your Lambda functions.
  • Managing subscriptions from the Datadog Forwarder to your Lambda and non-Lambda log groups.

npm Package Installation:

yarn add --dev datadog-cdk-constructs
# or
npm install datadog-cdk-constructs --save-dev

PyPI Package Installation:

pip install datadog-cdk-constructs

Note:

Pay attention to the output from your package manager as the Datadog CDK Construct Library has peer dependencies.

Usage

AWS CDK

If you are new to AWS CDK then check out this workshop.

Add this to your CDK stack:

# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
from datadog_cdk_constructs import Datadog

datadog = Datadog(self, "Datadog",
    node_layer_version=<LAYER_VERSION>,
    python_layer_version=<LAYER_VERSION>,
    add_layers=<BOOLEAN>,
    forwarder_arn="<FORWARDER_ARN>",
    flush_metrics_to_logs=<BOOLEAN>,
    site="<SITE>",
    api_key="{Datadog_API_Key}",
    api_kms_key="{Encrypted_Datadog_API_Key}",
    enable_datadog_tracing=<BOOLEAN>,
    inject_log_context=<BOOLEAN>
)
datadog.add_lambda_functions([<LAMBDA_FUNCTIONS>])
datadog.add_forwarder_to_non_lambda_log_groups([<LOG_GROUPS>])

Configuration

To further configure your Datadog construct, use the following custom parameters:

Note: The Descriptions may use the npm package parameters however they also apply to the PyPI package parameters.

npm package parameter PyPI package parameter Description
addLayers add_layers Whether to add the Lambda Layers or expect the user to bring their own. Defaults to true. When true, the Lambda Library version variables are also required. When false, you must include the Datadog Lambda library in your functions' deployment packages.
pythonLayerVersion python_layer_version Version of the Python Lambda layer to install, such as 21. Required if you are deploying at least one Lambda function written in Python and addLayers is true. Find the latest version number here.
nodeLayerVersion node_layer_version Version of the Node.js Lambda layer to install, such as 29. Required if you are deploying at least one Lambda function written in Node.js and addLayers is true. Find the latest version number from here.
extensionLayerVersion extension_layer_version Version of the Datadog Lambda Extension layer to install, such as 5. When extensionLayerVersion is set, apiKey (or apiKmsKey ) needs to be set as well. When enabled, lambda function log groups will not be subscribed by the forwarder. Learn more about the Lambda extension here.
forwarderArn forwarder_arn When set, the plugin will automatically subscribe the Datadog Forwarder to the functions' log groups. Do not set forwarderArn when extensionLayerVersion is set.
flushMetricsToLogs flush_metrics_to_logs Send custom metrics using CloudWatch logs with the Datadog Forwarder Lambda function (recommended). Defaults to true . If you disable this parameter, it's required to set apiKey (or apiKmsKey ).
site site Set which Datadog site to send data. This is only used when flushMetricsToLogs is false or extensionLayerVersion is set. Possible values are datadoghq.com , datadoghq.eu , us3.datadoghq.com and ddog-gov.com . The default is datadoghq.com .
apiKey api_key Datadog API Key, only needed when flushMetricsToLogs is false or extensionLayerVersion is set. For more information about getting a Datadog API key, see the API key documentation.
apiKmsKey api_kms_key Datadog API Key encrypted using KMS. Use this parameter in place of apiKey when flushMetricsToLogs is false or extensionLayerVersion is set, and you are using KMS encryption.
enableDatadogTracing enable_datadog_tracing Enable Datadog tracing on your Lambda functions. Defaults to true .
injectLogContext inject_log_context When set, the lambda layer will automatically patch console.log with Datadog's tracing ids. Defaults to true .

AWS CDK Configurations

The AWS CDK library offers some additional options that you may include in your lambda functions and application / stack.

Tracing

Enable X-Ray Tracing on your Lambda functions. For more information, see CDK documentation.

# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
import aws_cdk.aws_lambda as lambda_

lambda_function = lambda_.Function(self, "HelloHandler",
    runtime=lambda_.Runtime.NODEJS_10_X,
    code=lambda_.Code.from_asset("lambda"),
    handler="hello.handler",
    tracing=lambda_.Tracing.ACTIVE
)

Nested Stacks

Add the Datadog CDK Construct to each stack you wish to instrument with Datadog. In the example below, we initialize the Datadog CDK Construct and call addLambdaFunctions() in both the RootStack and NestedStack.

# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
from datadog_cdk_constructs import Datadog
import aws_cdk.core as cdk

class RootStack(cdk.Stack):
    def __init__(self, scope, id, *, description=None, env=None, stackName=None, tags=None, synthesizer=None, terminationProtection=None, analyticsReporting=None):
        super().__init__(scope, id, description=description, env=env, stackName=stackName, tags=tags, synthesizer=synthesizer, terminationProtection=terminationProtection, analyticsReporting=analyticsReporting)
        NestedStack(self, "NestedStack")

        datadog = Datadog(self, "Datadog",
            node_layer_version=<LAYER_VERSION>,
            python_layer_version=<LAYER_VERSION>,
            add_layers=<BOOLEAN>,
            forwarder_arn="<FORWARDER_ARN>",
            flush_metrics_to_logs=<BOOLEAN>,
            site="<SITE>",
            api_key="{Datadog_API_Key}",
            api_kms_key="{Encrypted_Datadog_API_Key}",
            enable_datadog_tracing=<BOOLEAN>,
            inject_log_context=<BOOLEAN>
        )
        datadog.add_lambda_functions([<LAMBDA_FUNCTIONS>])

class NestedStack(cdk.NestedStack):
    def __init__(self, scope, id, *, parameters=None, timeout=None, notificationArns=None, removalPolicy=None):
        super().__init__(scope, id, parameters=parameters, timeout=timeout, notificationArns=notificationArns, removalPolicy=removalPolicy)

        datadog = Datadog(self, "Datadog",
            node_layer_version=<LAYER_VERSION>,
            python_layer_version=<LAYER_VERSION>,
            add_layers=<BOOLEAN>,
            forwarder_arn="<FORWARDER_ARN>",
            flush_metrics_to_logs=<BOOLEAN>,
            site="<SITE>",
            api_key="{Datadog_API_Key}",
            api_kms_key="{Encrypted_Datadog_API_Key}",
            enable_datadog_tracing=<BOOLEAN>,
            inject_log_context=<BOOLEAN>
        )
        datadog.add_lambda_functions([<LAMBDA_FUNCTIONS>])

Tags

Add tags to your constructs. We recommend setting an env and service tag to tie Datadog telemetry together. For more information see official AWS documentation and CDK documentation.

How it works

The Datadog CDK construct takes in a list of lambda functions and installs the Datadog Lambda Library by attaching the Lambda Layers for Node.js and Python to your functions. It redirects to a replacement handler that initializes the Lambda Library without any required code changes. Additional configurations added to the Datadog CDK construct will also translate into their respective environment variables under each lambda function (if applicable / required).

While Lambda function based log groups are handled by the addLambdaFunctions method automatically, the construct has an additional function addForwarderToNonLambdaLogGroups which will subscribe the forwarder to any additional log groups of your choosing.

Resources to learn about CDK

Using Projen

This AWS CDK Construct Library uses Projen to maintain project configuration files such as the package.json, .gitignore, .npmignore, etc. Most of the configuration files will be protected by Projen via read-only permissions. In order to change these files, edit the .projenrc.js file then run npx projen to synthesize the new changes. Check out Projen for more details.

Opening Issues

If you encounter a bug with this package, we want to hear about it. Before opening a new issue, search the existing issues to avoid duplicates.

When opening an issue, include the Datadog CDK Construct version, Node version, and stack trace if available. In addition, include the steps to reproduce when appropriate.

You can also open an issue for a feature request.

Contributing

If you find an issue with this package and have a fix, please feel free to open a pull request following the procedures.

Testing

If you contribute to this package you can run the tests using yarn test. This package also includes a sample application for manual testing:

  1. Open a seperate terminal and run yarn watch, this will ensure the Typescript files in the src directory are compiled to Javascript in the lib directory.
  2. Navigate to src/sample, here you can edit index.ts to test your contributions manually.
  3. At the root datadog-cdk-constructs directory run npx cdk --app lib/sample/index.js <CDK Command>, replacing <CDK Command> with common CDK commands like synth, diff, or deploy.
  • Note, if you receive "... is not authorized to perform: ..." you may also need to authorize the commands with your AWS credentials.

Debug Logs

To display the debug logs for this library, set the DD_CONSTRUCT_DEBUG_LOGS env var to true when running cdk synth (use --quiet to suppress generated template output).

Example: Ensure you are at root datadog-cdk-constructs directory

DD_CONSTRUCT_DEBUG_LOGS=true npx cdk --app lib/sample/index.js synth --quiet

Community

For product feedback and questions, join the #serverless channel in the Datadog community on Slack.

License

Unless explicitly stated otherwise all files in this repository are licensed under the Apache License Version 2.0.

This product includes software developed at Datadog (https://www.datadoghq.com/). Copyright 2021 Datadog, Inc.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

datadog-cdk-constructs-0.2.5.tar.gz (125.9 kB view details)

Uploaded Source

Built Distribution

datadog_cdk_constructs-0.2.5-py3-none-any.whl (124.2 kB view details)

Uploaded Python 3

File details

Details for the file datadog-cdk-constructs-0.2.5.tar.gz.

File metadata

  • Download URL: datadog-cdk-constructs-0.2.5.tar.gz
  • Upload date:
  • Size: 125.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/4.5.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.61.1 CPython/3.9.5

File hashes

Hashes for datadog-cdk-constructs-0.2.5.tar.gz
Algorithm Hash digest
SHA256 9c0e7a07527ae1eb8e1ceb4f57cb4e848e5d1ede097326b7e49adc0b63a38460
MD5 fc055ba97429ecd4140da88768609d32
BLAKE2b-256 42c952e1e55666bf9c723ae169cd2f626d10556be8a7a1e47ec085eec28e31ab

See more details on using hashes here.

Provenance

File details

Details for the file datadog_cdk_constructs-0.2.5-py3-none-any.whl.

File metadata

  • Download URL: datadog_cdk_constructs-0.2.5-py3-none-any.whl
  • Upload date:
  • Size: 124.2 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/4.5.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.61.1 CPython/3.9.5

File hashes

Hashes for datadog_cdk_constructs-0.2.5-py3-none-any.whl
Algorithm Hash digest
SHA256 2d6fd88870981821393f4063db1d276681ccc4cfa4f330f7c0d936a64306dcb5
MD5 931b40304ef492566229b18c011fca59
BLAKE2b-256 6fc38114e03279caa1f7174d7ae806bbd89fb30aa7d598de479f4b25c9e5d958

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page