Developing and Testing AWS Lambda Functions Locally with SAM CLI

Introduction
Motivation
Prerequisites
Introduction to AWS SAM CLI
- Key Benefits of AWS SAM CLI
Initializing a Local SAM Project
The SAM Template
Inspecting the Lambda Function
Environment Variables
Permissions for Local Testing
Deploying the Function
Best Practices for Lambda Development
Troubleshooting Common Issues
Conclusion

Introduction

I am developing a Lambda Function that sends me a daily Cost Reports by E-Mail. Here I am going to show you how to setup local development and testing for Lambda Functions and how to deploy them using the SAM CLI. For this I am going set up a local sam project “AWS_DailyCostReport” and test my Lambda Function locally before deploying to my AWS Account with the SAM CLI.

AWS Lambda is a powerful tool for building serverless applications. However, developing and testing Lambda functions can be challenging, especially when trying to debug or simulate production environments. Thankfully, the AWS Serverless Application Model (SAM) provides an excellent framework to simplify local development and testing with the sam local invoke command.

In this post, we’ll explore how to develop and test AWS Lambda functions locally using AWS SAM. We’ll cover setup, creating a Lambda function, testing it locally, and best practices for debugging.

AWS Lambda Logo

Amazon Web Services, the AWS Lambda logo, and other AWS service marks are trademarks of Amazon.com, Inc. or its affiliates.

Motivation

Have you ever tried editing or writing a Lambda function directly in the AWS Management Console? When I was working on my language-game for the AWS Game Builder Challenge I started by writing the lambda function in the Console but quickly realized I needed to find another way. The in-browser editor is small, unintuitive, and requires you to redeploy the function after every change just to test it. It gets frustrating very quickly. While it might suffice for quick tweaks, it falls short when developing a new function from scratch. That’s why I prefer a local development setup. With tools like VS Code, I can code, test, and debug efficiently in a familiar environment. Once I’m satisfied with the results, I can easily deploy the function with just one command.

Prerequisites

Before we dive in, ensure the following are in place:

AWS CLI Installed: Install the AWS CLI and configure it with your credentials.
AWS SAM CLI Installed: Download and install the AWS SAM CLI from the official documentation.
Docker Installed: AWS SAM leverages Docker to simulate the Lambda runtime locally.
AWS Account with proper permissions set up

Introduction to AWS SAM CLI

AWS SAM CLI is a command-line tool that provides a simplified way to build, test, and deploy serverless applications. It extends AWS CloudFormation syntax to support serverless-specific resources, making it easier to define and deploy Lambda functions, APIs, DynamoDB tables, and more.

Key Benefits of AWS SAM CLI:

Simplified Testing: Run Lambda functions locally to streamline debugging.
Easy Deployment: Automate packaging and deployment to AWS.
Integration with AWS Tools: Seamlessly works with AWS services and tools like CloudFormation and CodePipeline.
Resource Emulation: Test APIs and DynamoDB interactions locally without deploying resources.

Initializing a local SAM Project

For a simple Lambda Function I like to start with a sam SAM Quick Start Template, it makes it easier to get started and already sets up the directory structure and a basic Lambda Function code.

If you don’t know which SAM Quick Start Template to use, you can just do the guided setup. Since I am only interested in Python Templates I pass the --runtime parameter into the init command. If you leave the --runtime out it will also show node.js and other runtime Templates. Now just follow along the guided setup.

[local-vm ~]$ sam init --runtime python3.9
Which template source would you like to use?
        1 - AWS Quick Start Templates
        2 - Custom Template Location
Choice: 1

Choose an AWS Quick Start application template
        1 - Hello World Example
        2 - Hello World Example with Powertools for AWS Lambda
        3 - Infrastructure event management
        4 - Multi-step workflow
        5 - Lambda EFS example
        6 - Serverless Connector Hello World Example
        7 - Multi-step workflow with Connectors
Template: 1

Based on your selections, the only Package type available is Zip.
We will proceed to selecting the Package type as Zip.

Based on your selections, the only dependency manager available is pip.
We will proceed copying the template using pip.

Would you like to enable X-Ray tracing on the function(s) in your application?  [y/N]: n

Would you like to enable monitoring using CloudWatch Application Insights?
For more info, please view https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch-application-insights.html [y/N]: n

Would you like to set Structured Logging in JSON format on your Lambda functions?  [y/N]: n

Project name [sam-app]: AWS_DailyCostReport


    -----------------------
    Generating application:
    -----------------------
    Name: AWS_DailyCostReport
    Runtime: python3.9
    Architectures: x86_64
    Dependency Manager: pip
    Application Template: hello-world
    Output Directory: .
    Configuration file: AWS_DailyCostReport/samconfig.toml

    Next steps can be found in the README file at AWS_DailyCostReport/README.md


Commands you can use next
=========================
[*] Create pipeline: cd AWS_DailyCostReport && sam pipeline init --bootstrap
[*] Validate SAM template: cd AWS_DailyCostReport && sam validate
[*] Test Function in the Cloud: cd AWS_DailyCostReport && sam sync --stack-name {stack-name} --watch

This initializes a new SAM Project in a new directory with the following directory structure:

[local-vm ~]$ cd AWS_DailyCostReport/
[local-vm AWS_DailyCostReport]$ tree
.
├── events
│   └── event.json
├── hello_world
│   ├── app.py
│   ├── __init__.py
│   └── requirements.txt
├── __init__.py
├── README.md
├── samconfig.toml
├── template.yaml
└── tests
    ├── __init__.py
    ├── integration
    │   ├── __init__.py
    │   └── test_api_gateway.py
    ├── requirements.txt
    └── unit
        ├── __init__.py
        └── test_handler.py

5 directories, 14 files

Don’t worry, that’s a lot of files, but for a standalone Lambda Function we are really only interested in two files:

template.yaml -> SAM Template for provisioning a lambda function
hello_world/app.py -> Lambda Function

The SAM Template

Let us first take a look at the SAM Template to see which AWS resources will be provisioned:

[local-vm AWS_DailyCostReport]$ cat template.yaml
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
  AWS_DailyCostReport

  Sample SAM Template for AWS_DailyCostReport

# More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst
Globals:
  Function:
    Timeout: 3
    MemorySize: 128

Resources:
  HelloWorldFunction:
    Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
    Properties:
      CodeUri: hello_world/
      Handler: app.lambda_handler
      Runtime: python3.9
      Architectures:
        - x86_64
      Events:
        HelloWorld:
          Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
          Properties:
            Path: /hello
            Method: get

Outputs:
  # ServerlessRestApi is an implicit API created out of Events key under Serverless::Function
  # Find out more about other implicit resources you can reference within SAM
  # https://github.com/awslabs/serverless-application-model/blob/master/docs/internals/generated_resources.rst#api
  HelloWorldApi:
    Description: "API Gateway endpoint URL for Prod stage for Hello World function"
    Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/hello/"
  HelloWorldFunction:
    Description: "Hello World Lambda Function ARN"
    Value: !GetAtt HelloWorldFunction.Arn
  HelloWorldFunctionIamRole:
    Description: "Implicit IAM Role created for Hello World function"
    Value: !GetAtt HelloWorldFunctionRole.Arn

As you can see in the resources section there is the definition of the lambda function. The template specifies “CodeUri: hello_world/” which tells sam in which directory the lambda code files are located and “Handler: app.lambda_handler” tells sam where exactly to find the handler. In this case “app.lambda_handler” means file app.py and function lambda_handler. You can rename the function or the file as you wish, but be sure to modify these properties in the sam template. Also note, that the ressource itself is called “HelloWorldFunction”. We need this when we run the command to invoke the function locally.

Another thing to note is that this hello world template implicitly defines a API Gateway. Since SAM Templates are developed for standard serverless setups with minimal effort, it makes it very easy to define and deploy standard serverless setups. In this case, the following section in the function definition creates a REST API listener for the function:

      Events:
        HelloWorld:
          Type: Api 
          Properties:
            Path: /hello
            Method: get

This Code Creates an API Gateway of type REST, with Ressource Path "/hello" and Method GET, which automatically integrates with our lambda function in the backend.

So as you can see SAM makes it very easy and convenient to define AWS Serverless Infrastructure as Code. But since we want a standalone Lambda Function without the API we can go ahead a remove the events property from the SAM Template. Now the Function definition should look like this:

Resources:
  HelloWorldFunction:
    Type: AWS::Serverless::Function 
    Properties:
      CodeUri: hello_world/
      Handler: app.lambda_handler
      Runtime: python3.9
      Architectures:
        - x86_64

Since the API will not be provisioned, we have to also remove this Output directive otherwise we will get an error later:

  HelloWorldApi:
    Description: "API Gateway endpoint URL for Prod stage for Hello World function"
    Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/hello/"

Now the SAM Template looks like this:

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
  AWS_DailyCostReport

  Sample SAM Template for AWS_DailyCostReport

Globals:
  Function:
    Timeout: 3
    MemorySize: 128

Resources:
  HelloWorldFunction:
    Type: AWS::Serverless::Function 
    Properties:
      CodeUri: hello_world/
      Handler: app.lambda_handler
      Runtime: python3.9
      Architectures:
        - x86_64

Outputs:
  HelloWorldFunction:
    Description: "Hello World Lambda Function ARN"
    Value: !GetAtt HelloWorldFunction.Arn
  HelloWorldFunctionIamRole:
    Description: "Implicit IAM Role created for Hello World function"
    Value: !GetAtt HelloWorldFunctionRole.Arn

This SAM Template only provisions a standalone Lambda Function just like we want.

Inspecting the Lambda Function

Let us now look at the generated template for the Lambda Function. For this we look at the hello_world/app.py file. If we remove everything that is commented out it looks like this:

[local-vm AWS_DailyCostReport]$ cat hello_world/app.py
import json

def lambda_handler(event, context):

    return {
        "statusCode": 200,
        "body": json.dumps({
            "message": "hello world",
        }),
    }

As you can see it returns a json containing a statusCode, body and message. This corresponds to a standard JSON API Response.

Now we can just go ahead and run it locally. To do this you must make sure you have docker running, as the sam cli will attempt to run lambda inside a docker container on your machine.

To run the command we must pass in the name of the Lambda Function Ressource as defined in the SAM Template as we have previously seen “HelloWorldFunction”.

The command also expects an event. As you can see in the directory structure we have a folder events which contains a sample event.json. This is an example of an API Event, as is gets passed into the Lambda Function. For the hello world function this is irrelevant, but in reality you probably want to access some of the json elements inside that event, such as query strings, method, URI Path or other data that the API passes into the function.

Now let us just run the function locally:

[local-vm AWS_DailyCostReport]$ sam local invoke "HelloWorldFunction" --event events/event.json
Invoking app.lambda_handler (python3.9)
Local image is out of date and will be updated to the latest runtime. To skip this, pass in the parameter --skip-pull-image
Building image............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
Using local image: public.ecr.aws/lambda/python:3.9-rapid-x86_64.

Mounting /hello_world as /var/task:ro,delegated, inside runtime container
START RequestId: 000c6615-a7ef-4ead-b421-229048000b63 Version: $LATEST
END RequestId: 1a741f62-366c-488a-a9dd-5c059e81a4cc
REPORT RequestId: 1a741f62-366c-488a-a9dd-5c059e81a4cc  Init Duration: 0.08 ms  Duration: 480.01 ms     Billed Duration: 481 ms Memory Size: 128 MB     Max Memory Used: 128 MB
{"statusCode": 200, "body": "{\"message\": \"hello world\"}"}

It worked as expected and returned the hello world JSON.

Environment Variables

If your function relies on Environment Variables you can easily store them in a JSON File and pass it into the function with the cli parameter --env-vars env.json

sam local invoke --env-vars env.json

Permissions for local testing

By default the local invocations of the Function are done using the local IAM context of your aws cli configuration. Make sure that this user has the required permissions to execute the function. If you have multiple profiles configured you can specify a different profile for the local invocation with the --profile flag

sam local invoke --profile <user_profile>

Deploying the Function

Once you are done with your local testing, you can easily deploy the function to your AWS Account as follows:

sam validate --lint
sam build
sam deploy --guided


#validate the template
[local-vm AWS_DailyCostReport]$ sam validate --lint
./template.yaml is a valid SAM Template

#run build
[local-vm AWS_DailyCostReport]$ sam build
Starting Build use cache
Manifest file is changed (new hash: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx) or dependency folder (.aws-sam/deps/xxxxxxxxxxxxxxxxxxxxxxxxxx) is missing for
(HelloWorldFunction), downloading dependencies and copying/building source
Building codeuri: AWS_DailyCostReport/hello_world runtime: python3.9 architecture: x86_64 functions: HelloWorldFunction
 Running PythonPipBuilder:CleanUp
 Running PythonPipBuilder:ResolveDependencies
 Running PythonPipBuilder:CopySource
 Running PythonPipBuilder:CopySource

Build Succeeded

Built Artifacts  : .aws-sam/build
Built Template   : .aws-sam/build/template.yaml

Commands you can use next
=========================
[*] Validate SAM template: sam validate
[*] Invoke Function: sam local invoke
[*] Test Function in the Cloud: sam sync --stack-name {{stack-name}} --watch
[*] Deploy: sam deploy --guided

Now you are ready to deploy your Function

The first time you deploy you must call with the --guided flag. This will ask you some questions and save the answers in a config file. After that you can leave out the --guided flag.

[local-vm] $ sam deploy --guided

Configuring SAM deploy
======================

        Looking for config file [samconfig.toml] :  Found
        Reading default arguments  :  Success

        Setting default arguments for 'sam deploy'
        =========================================
        Stack Name [AWS_DailyCostReport]: AWS-DailyCostReport
        AWS Region [xxxxxxxxxxxxx]:
        #Shows you resources changes to be deployed and require a 'Y' to initiate deploy
        Confirm changes before deploy [Y/n]: y
        #SAM needs permission to be able to create roles to connect to the resources in your template
        Allow SAM CLI IAM role creation [Y/n]: y
        #Preserves the state of previously provisioned resources when an operation fails
        Disable rollback [y/N]: n
        Save arguments to configuration file [Y/n]: y
        SAM configuration file [samconfig.toml]:
        SAM configuration environment [default]:

        Looking for resources needed for deployment:

        Managed S3 bucket: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
        A different default S3 bucket can be set in samconfig.toml and auto resolution of buckets turned off by setting resolve_s3=False

        Saved arguments to config file
        Running 'sam deploy' for future deployments will use the parameters saved above.
        The above parameters can be changed by modifying samconfig.toml
        Learn more about samconfig.toml syntax at
        https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-config.html

        Uploading to AWS-DailyCostReport/xxxxxxxxxxxxxxxxxxxxxx  575224 / 575224  (100.00%)

        Deploying with following values
        ===============================
                Stack name                   : AWS-DailyCostReport
        Region                       : xxxxxxxxxx
        Confirm changeset            : True
        Disable rollback             : False
        Deployment s3 bucket         : xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
        Capabilities                 : ["CAPABILITY_IAM"]
        Parameter overrides          : {}
        Signing Profiles             : {}

Initiating deployment
=====================

Uploading to AWS-DailyCostReport/xxxxxxxxxxxxxxxxxx.template  903 / 903  (100.00%)


Waiting for changeset to be created..

CloudFormation stack changeset
---------------------------------------------------------------------------------------------------------------------------------------------------------
Operation                              LogicalResourceId                      ResourceType                           Replacement
---------------------------------------------------------------------------------------------------------------------------------------------------------
+ Add                                  HelloWorldFunctionRole                 AWS::IAM::Role                         N/A
+ Add                                  HelloWorldFunction                     AWS::Lambda::Function                  N/A
---------------------------------------------------------------------------------------------------------------------------------------------------------


Changeset created successfully. xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx


Previewing CloudFormation changeset before deployment
======================================================
Deploy this changeset? [y/N]: y

2025-01-10 11:36:57 - Waiting for stack create/update to complete

CloudFormation events from stack operations (refresh every 5.0 seconds)
---------------------------------------------------------------------------------------------------------------------------------------------------------
ResourceStatus                         ResourceType                           LogicalResourceId                      ResourceStatusReason
---------------------------------------------------------------------------------------------------------------------------------------------------------
CREATE_IN_PROGRESS                     AWS::CloudFormation::Stack             AWS-DailyCostReport                    User Initiated
CREATE_IN_PROGRESS                     AWS::IAM::Role                         HelloWorldFunctionRole                 -
CREATE_IN_PROGRESS                     AWS::IAM::Role                         HelloWorldFunctionRole                 Resource creation Initiated
CREATE_COMPLETE                        AWS::IAM::Role                         HelloWorldFunctionRole                 -
CREATE_IN_PROGRESS                     AWS::Lambda::Function                  HelloWorldFunction                     -
CREATE_IN_PROGRESS                     AWS::Lambda::Function                  HelloWorldFunction                     Resource creation Initiated
CREATE_IN_PROGRESS -                   AWS::Lambda::Function                  HelloWorldFunction                     Eventual consistency check initiated
CONFIGURATION_COMPLETE
CREATE_IN_PROGRESS -                   AWS::CloudFormation::Stack             AWS-DailyCostReport                    Eventual consistency check initiated
CONFIGURATION_COMPLETE
CREATE_COMPLETE                        AWS::Lambda::Function                  HelloWorldFunction                     -
CREATE_COMPLETE                        AWS::CloudFormation::Stack             AWS-DailyCostReport                    -
---------------------------------------------------------------------------------------------------------------------------------------------------------

CloudFormation outputs from deployed stack
-----------------------------------------------------------------------------------------------------------------------------------------------------------
Outputs
-----------------------------------------------------------------------------------------------------------------------------------------------------------
Key                 HelloWorldFunctionIamRole
Description         Implicit IAM Role created for Hello World function
Value               xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Key                 HelloWorldFunction
Description         Hello World Lambda Function ARN
Value               xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
-----------------------------------------------------------------------------------------------------------------------------------------------------------


Successfully created/updated stack - AWS-DailyCostReport in xxxxxxxxxx

Now your function is deployed, after having tested locally. You can now go to the Lambda console to verify the deployment of your function.

AWS Lambda Console

Best Practices for Lambda Development

Follow the Principle of Least Privilege: Grant minimal permissions to Lambda Execution Roles.
Use Environment Variables: Store configuration data in environment variables to separate code from configuration.
Optimize Cold Start: Use provisioned concurrency or reduce package size by excluding unnecessary dependencies.
Use Sample Events: Create realistic event payloads in the events/ folder.
Log and Monitor: Utilize AWS CloudWatch for logging and X-Ray for tracing.
Unit Tests: Write unit tests to validate your Lambda logic independently from AWS services.

Troubleshooting Common Issues

Issue: “Command Not Found” Error

Ensure that SAM CLI is installed and added to your system’s PATH.
Re-run the installation and verify using sam --version.

Issue: “Access Denied” During Deployment

Check the permissions of your AWS credentials.
Ensure the IAM role has sufficient permissions for the deployment.

Issue: “Resource Not Found” in Local Testing

Verify that required resources (e.g., environment variables, configuration files) are set up locally.
Use sam validate to check your template for issues.

Conclusion

The AWS SAM CLI is a game-changer for serverless development, offering powerful features to streamline testing, deployment, and resource management. In this tutorial, we walked through the entire process of developing and testing a Lambda function locally with SAM CLI. Whether you’re simulating API Gateway, testing event payloads, or debugging in real-time, AWS SAM empowers developers to streamline the serverless development lifecycle.

I hope this guide was insightful and inspires you to enhance your serverless development workflow.

COMING UP NEXT - Learn how to create a Lambda function that sends daily email reports of your AWS account costs.

Feel free to explore my work and connect with me:

GitHub Repo - Explore my repositories and projects.
LinkedIn - Connect with me and grow your professional network.

Stay tuned to DevOpsBug for more updates, tips, and project showcases!

🙂 Happy coding!

Table of Contents#

Introduction#

Motivation#

Prerequisites#

Introduction to AWS SAM CLI#

Key Benefits of AWS SAM CLI:#

Initializing a local SAM Project#

The SAM Template#

Inspecting the Lambda Function#

Environment Variables#

Permissions for local testing#

Deploying the Function#

Best Practices for Lambda Development#

Troubleshooting Common Issues#

Issue: “Command Not Found” Error#

Issue: “Access Denied” During Deployment#

Issue: “Resource Not Found” in Local Testing#

Conclusion#

Table of Contents