Amazon Lambda Destination

Segment makes it easy to send your data to AWS Lambda (and lots of other destinations). Once you collect your data using Segment’s open source libraries, Segment translates and routes your data to AWS Lambda in the format it can use.

AWS Lambda lets you run code without provisioning or managing servers. You pay only for the compute time you consume - there is no charge when your code is not running.

With Lambda, you can run code for virtually any type of application or backend service - all with zero administration. Just upload your code and Lambda takes care of everything required to run and scale your code with high availability. You can set up your code to automatically trigger from other AWS services or call it directly from any web or mobile app.

Read more about AWS Lambda on the Segment blog.

Getting Started

Before you start, make sure Amazon Lambda supports the source type and connection mode you’ve chosen to implement. You can learn more about connection modes here.

Build a Lambda Function to Process Segment Events

In order to process events from Segment, you will need to provide a Lambda function that can handle your event flow.

We allow you to send each call type (track,identify,etc) to a different Lambda function. The example below shows how to create a new Lambda function from scratch.

Start by browsing to the Lambda service page in your AWS account.

Click the “Create a function” button to create a new function.

Select the “Author from scratch” radio button since we will be providing the source code for the function.

Enter a name for your function and select your preferred runtime.

For the “Role” field, select “Create a new role from AWS policy templates” from the dropdown. Create a “Role name” that makes sense for you, and leave “Policy templates” empty. (This will create a role that can write to Cloud Watch logs. Cloud Watch logs are optional, though we do support them in our Settings).

Click “Create function”.

Copy the “ARN” for the Lambda and copy it into the “Lambda” setting in your Segment Lambda destination settings.

Lambda Function Source Code

Scrolling down slightly on the main page for your new Lambda function, you will see the code editor. You can write code here or use an existing Lambda function. See the Lambda documentation for more details on creating a Lambda.

Set up Segment IAM policy & role for invoking your Lambda

Segment will need to be able to call (“invoke”) your Lambda in order to process events. This requires you to configure an IAM role for your Lambda which allows the Segment account to invoke your function.

There are two options for setting up the IAM policy and role:

1. Use a CloudFormation template. (recommended)
2. Manually create the policy and role.

Use CloudFormation

Using CloudFormation minimizes the set up steps needed, and is Segment’s recommended way to create your Lambda’s policy and role.

Create the CloudFormation Template

Copy or download the SegmentLambdaDestinationCFTemplate from our segment-lambda-recipes GitHub repo. Save the file with whatever name you like, but make sure it doesn’t have a file extension.

Create the CloudFormation stack

Within the AWS Console, navigate to CloudFormation. Navigate to the Stacks page.

Click the “Create Stack” button.

On the “Select Template” page, select “Upload a template to Amazon S3”, then using “Chose File”, select the SegmentLambdaDestinationCFTemplate you created/downloaded in the previous step.

Click “Next”.

Give your stack a meaningful name. The “ExternalId” parameter corresponds to the “External ID” setting in your Segment Lambda destination settings. The “LambdaARN” parameter corresponds to the “Lambda” setting in your Segment Lambda destination settings.

You can leave the next page as is, no changes needed. On the last page, review your template details and click “Create”.

You should now see your new Stack listed in the Stacks page.

Once the status is “CREATE_COMPLETE”, click on the name of your Stack. On the Stack Detail page, under the “Resources” section, you should see a policy and role listed.

Click on the “Physical ID” of the role. You will be redirected to the summary page for the role within the IAM console. Copy the “Role ARN” and copy it into the “Role Address” setting in your Segment Lambda destination settings. Using the examples provided, your Segment Lambda destination settings would now look something like this:

Create Policy and Role Manually

Create an IAM policy.

Sign in to the Identity and Access Management (IAM) console and follow these instructions to Create an IAM policy to allow Segment permission to invoke your Lambda function.

Select the Create Policy from JSON option and use the following template policy in the Policy Document field. Be sure to change the {region}, {account-id} and {function-names} with the applicable values. An example of a Lambda ARN arn:aws:lambda:us-west-2:355207333203:function:my-example-function.

Note: you can put in a placeholder ARN for now, as you will need to come back to this step to update with the ARN of your Lambda once that’s been created.

{
  "Version": "2012-10-17",
  "Statement": [
    {
        "Effect": "Allow",
        "Action": [
            "lambda:InvokeFunction"
        ],
        "Resource": [
            "lambda ARN 1",
            "lambda ARN 2",
            ...
            "lambda ARN n"
        ]
    }
  ]
}

Create an IAM role

Sign in to the Identity and Access Management (IAM) console and follow these instructions to Create an IAM role to allow Segment permission to invoke your Lambda function.

While setting up the new role, add the policy you created in the previous step.

Finish with any other set up items you may want (like tags). Once that’s complete, search for and click on your new roles from the IAM home.

Select the “Trust Relationships” tab, then click the “Edit trust relationship” button.

Copy and paste the following into your trust relationship. You should replace <your-source-id> with either the Source ID of the attached Segment source (the default) or whatever custom external id you set in your AWS Lambda destination settings. arn:aws:iam::595280932656:root refers to Segment’s AWS Account, and is what allows our Destination to access the role to invoke your Lambda.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::595280932656:root"
      },
      "Action": "sts:AssumeRole",
      "Condition": {
        "StringEquals": {
          "sts:ExternalId": "YOUR_SEGMENT_SOURCE_ID"
        }
      }
    }
  ]
}

If you have multiple Source’s using this Role, replace the sts:ExternalId setting above with

    "sts:ExternalId": ["YOUR_SEGMENT_SOURCE_ID", "ANOTHER_SOURCE_ID", "A_THIRD_SOURCE_ID"]

Configure Segment Lambda Destination

Search for “Lambda” in our catalog. Connect the destination to the source you created previously. Now you will be presented with the AWS Lambda settings.

FAQ

What is the Log Type Setting?

This setting controls the Log Type for your Lambda function using Cloud Watch. Select option Tail if you would like to see detailed logs in Cloud Watch.

My Lambda <> Segment connection is timing out, what do I do?

Due to how our event delivery system, Centrifuge, works, your Lambda can’t take more than 5 seconds to run per message. If you’re consistently running into timeout issues, you should consult the AWS Lambda docs, as well as docs for your language of choice, for tips on optimizing performance.

Handling Common Errors You can find delivery logs in Destination > Event Delivery.

Here are some common errors you may come across and how to resolve:

Execution Error - occurs when the lambda throws an error. Check out the code to ensure the lambda will succeed for that event.

Operation timedout - occurs when the lambda takes more than 5s to respond.

Accessdenied - occurs when IAM permissions are not set up correctly. Check the IAM policy and role.

Invalid Credentials - occurs when IAM permissions are not set up correctly. Check the IAM policy and role.

Personas

You can send computed traits and audiences generated through Segment Personas to this destination as a user property. To learn more about Personas, contact us for a demo.

For user-property destinations, Personas sends an Identify call to the destination for each user that is added or removed from an audience. The property name is the snake_cased version of the audience name you provide, with a boolean (true/false) value to indicate if they’re a member of the audience. For example, when a user first completes an order in the last 30 days, Personas sends an identify call with the property order_completed_last_30days: true, and when this user no longer satisfies these criteria (if the user does not complete another order over 30 days) Personas sends another Identify call to set that value to false.

When Personas first creates the audience, it sends an Identify call for every user in the audience. Later syncs only update users which were added or removed since the last sync.

Personas

You can send computed traits and audiences generated using Segment Personas to this destination as a user property. To learn more about Personas, contact us for a demo.

For user-property destinations, an identify call is sent to the destination for each user being added and removed. The property name is the snake_cased version of the audience name, with a true/false value to indicate membership. For example, when a user first completes an order in the last 30 days, Personas sends an Identify call with the property order_completed_last_30days: true. When the user no longer satisfies this condition (for example, it’s been more than 30 days since their last order), Personas sets that value to false.

When you first create an audience, Personas sends an Identify call for every user in that audience. Later audience syncs only send updates for users whose membership has changed since the last sync.

Settings

Segment lets you change these destination settings from the Segment app without having to touch any code.

Client Context

An optional map to pass to the Lambda function. See AWS Lambda documentation for more information.

External ID

This is an optional string Segment will use to assume the role provided to invoke the Lambda function. If this setting is not defined, we’ll use the Source ID. For more information about external IDs while assuming AWS roles, check here.

Lambda

The name of the Lambda function to invoke. These are the supported name formats:

• Function name (my-function) or with alias (my-function:v1).
• Function ARN (arn:aws:lambda:us-west-2:123456789012:function:my-function).
• Partial ARN (123456789012:function:my-function).

You can append a version number or alias to any of the formats.

Log Type

Lambda log type. By default None.

Select Tail if you would like to see detailed logs in Cloud Watch.

Region

AWS Region where the lambda lives. E.G. us-west-2, eu-west-3

Role Address

The address of the AWS role that will be invoking Lambda (ex: arn:aws:iam::874699288871:role/example-role).