This article will walk you through all the steps required to create a Lambda function using only the AWS command line. So no AWS console is allowed here!
Please note that this article won’t go into Lambda layers. Also, it is limited to the Python runtime, so you will need to adapt the commands to another runtime if that’s what you’re using.
Prerequisites
The first thing you’ll need to do is create an IAM user with programmatic access and sufficient permissions to manipulate Lambda functions. The AWS documentation describes how to do this.
The next step is to install the AWS command line interface. Again, the AWS documentation efficiently tells you how to do this for your operating system. You will also need to configure your credentials to be used by the AWS CLI. I strongly recommend that you use an explicit profile, not the default profile. If there is any chance that you have two or more profiles, and you also use the default profile, the likelihood that you will one day enter an AWS command without specifying the –profile option is near 100%, with unintended and probably bad consequences (This happened to me while manipulating CloudFormation stacks, and, needless to say, I learned my lesson.).
So, to configure the AWS CLI, run the following command:
aws configure –profile test
You will obviously need the code for the Lambda function. In this example, the Lambda function will query the GitHub API to retrieve the list of repositories for a given user and will send that list to an SNS topic. Here’s the code in Python (Save it in a file named “list_github_repos.py.”):
import requests
import boto3
import os
def doit(event, context):
username = event['username']
url = f"https://api.github.com/users/{username}/repos"
headers = {"accept": "application/json"}
print(f"Querying GitHub on URL: {url}")
r = requests.get(url, headers=headers)your@email.com
data = r.json()
repos = [i['name'] for i in data]
text = ",".join(repos)
print(f"GitHub responded: {text}")
msg = f"Repositories for {username}: {text}"
sns = boto3.client("sns")
sns.publish(
TopicArn=os.environ['SNS_TOPIC_ARN'],
Message=msg
)
In this case, you need an SNS topic, so go ahead and create it like so:
aws --profile test sns create-topic --name notify-repos
This will return the topic ARN; please make a copy of it, as you will need it later. To really see things in action, subscribe your email address to the SNS topic (Replace “your@email.com” with your real email address):
aws --profile test sns subscribe \
--topic-arn arn:aws:sns:us-east-1:123456789012:notify-repos \
--protocol email --notification-endpoint your@email.com
You will need to go to your email account and confirm the subscription. Please note this will take a few minutes for the subscription to start sending emails.
Finally, you will need a clear understanding of the IAM permissions the Lambda function requires. Here, you’ll need the permissions usually required by a Lambda function (which are related to sending logs to CloudWatch Logs) and also the “Publish” call on an SNS topic.
Create a Role for the Lambda Function
The next step is to create an IAM role for the Lambda function. This role will give permissions to the Lambda function to perform specific actions on certain AWS resources. To create the role, you’ll first need to have an “assume role policy document,” which specifies which entity the role applies to. In this case, it applies to the Lambda service, so the policy document will look like this:
{
“Version”: “2012-10-17”,
“Statement”: [
{
“Effect”: “Allow”,
“Principal”: {
“Service”: “lambda.amazonaws.com”
},
“Action”: “sts:AssumeRole”
}
]
}
Save the above JSON text in a file named “assume-role-policy-document.json.” To create the role, run:
aws --profile test iam create-role --role-name list-github-repos-role \
--assume-role-policy-document file://./assume-role-policy-document.json
The output of this command will show the role ARN. Please save this, as you will need it later.
Now, you have to attach some policies to that role. AWS provides the managed policy AWSLambdaBasicExecutionRole, which just gives the Lambda function permissions to write its logs to CloudWatch Logs. Go ahead and attach that policy to the role:
aws --profile test iam attach-role-policy --role-name list-github-repos-role \
--policy-arn \
arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
In your case, the Lambda function also needs to write to the “notify-repos” SNS topic. The policy document to allow the function to do this is the following (Use the SNS topic ARN you previously saved.):
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "sns:Publish",
"Resource": "arn:aws:sns:us-east-1:123456789012:notify-repos"
}
]
}
Save the above text in a file named “sns-policy-for-lambda.json.” You will now create what is called an “inline policy,” which means that the policy is saved inside the role itself, as opposed to a managed policy, which exists independently. To create this policy inside the role, run:
aws --profile test iam put-role-policy --role-name list-github-repos-role \
--policy-name publish-to-sns \
--policy-document file://./sns-policy-for-lambda.json
Create the Lambda Function
The next step is to create the Lambda function. For this, you need to package it into a zip file, along with any dependencies that are provided by the Lambda runtime engine. Our function imports the following modules: “requests,” “boto3,” and “os.” The last two are provided by the Lambda runtime, but you need to package the “requests” module along with your function code. To do this, create a file named “requirements.txt,” and edit it so it contains only the word “requests,” like so:
echo requests > requirements.txt
Then run the following to install the dependencies in the “pkg” subdirectory:
mkdir pkg
pip3 install --target pkg -r requirements.txt
Now run the following commands to package the function and its dependencies:
cd pkg
zip -r9 ../lambda.zip .
cd ..
rm -rf pkg
zip -g lambda.zip list_github_repos.py
You can now finally create your Lambda function using this command (Use the SNS topic ARN that you saved earlier.):
aws --profile test lambda create-function \
--function-name list_github_repos \
--runtime python3.9 \
--handler list_github_repos.doit \
--zip-file fileb://./lambda.zip \
--timeout 300 \
--environment \
'Variables={SNS_TOPIC_ARN=arn:aws:sns:us-east-1:123456789012:notify-repos}'
--role arn:aws:iam::123456789012:role/list-github-repos-role
A few comments about the above command:
- The “handler” option tells the Lambda runtime which Python function to execute when the Lambda function is called. The format of the argument is “file name”, followed by a dot “.” followed by the name of the function inside that file.
- The “timeout” option tells the Lambda runtime to abort the execution after the given timeout. Here, I specified 5 minutes, which should be more than enough. The maximum you can set is 15 minutes.
- The “environment” option is used to set the environment variable SNS_TOPIC_ARN to the SNS topic you want to send messages to. In this way, you don’t have to hardcode the SNS topic ARN inside the function’s code.
If you need to make changes to the function’s code, you can update the Lambda function by packaging the code and its dependencies like before, and then running the following command:
aws --profile test lambda update-function-code --function-name list_github_repos --zip-file fileb://./lambda.zip
Test the Lambda function
Finally, you can test your Lambda function. You need to pass an event (also called a payload), which must be a base64-encoded JSON object. In this example, the function code just tries to extract the “username” key from the JSON object, which is the GitHub username you want to query. So the way to test your function is:
aws --profile test lambda invoke --function-name list_github_repos --payload $(echo '{"username": "fabricetriboix"}' | base64) /dev/null
The “/dev/null” at the end is the file to which the AWS command line will write the output of the Lambda function. Here, the Lambda function doesn’t return anything, so there will be no output, but the AWS command line still requires this argument. If the invocation fails, it could be useful to set a real file and inspect its content; that will help you diagnose what the problem is.
Once you’ve invoked the Lambda function, and provided everything goes well, you should soon receive an email with a list of the public GitHub repositories for your chosen “username.”
Alternatives to the AWS Command Line
You’ll probably agree with me that creating a Lambda function using purely the AWS command line is quite painful, as this article shows. So let’s explore some alternatives.
AWS Console
The most obvious alternative is to use the AWS console. It provides a nice GUI and sets some things up for you, such as a role for your function. The main inconvenience is that it is suitable only for simple functions without external dependencies and without layers.
Serverless Application Model (SAM)
SAM is essentially an extension to AWS CloudFormation that simplifies the creation and management of Lambda functions. It is very complete and is suitable for whole serverless applications. It does require a learning curve, but that might very well be time well-spent if your entire application is serverless.
CloudFormation
CloudFormation is the infrastructure-as-code tool offered by AWS. It is proprietary and works on AWS only. In practice, writing the code on CloudFormation would require setting up more or less one resource for each manual step you did when invoking the AWS command line. On top of that, you will need to set up an S3 bucket to store the zip file because CloudFormation can’t access the zip on your computer (obviously). So overall, it will be as painful as the plain command line.
Terraform
Terraform is an infrastructure-as-code tool from HashiCorp. Terraform supports many cloud vendors and other environments where resources can be deployed. If using only basic Terraform objects, the only difference with CloudFormation will be the syntax of the code, so just as painful. Terraform does have the advantage of coming with modules, such as this AWS Lambda one, which can simplify your work.
CDK
The Cloud Development Kit is an open-source project initiated by AWS. It is an infrastructure-as-code tool that allows you to write actual code to describe your infrastructure, as opposed to using a declarative approach, as done by both CloudFormation and Terraform. By default, CDK uses CloudFormation as a backend, but many vendors are providing backends for their own platforms. CDK might be the most concise tool for creating a Lambda function, with a single call being required to do so, as detailed in this JavaScript example.
Conclusion
In conclusion, I would say that using the AWS CLI to create and manage Lambda functions is possible, although it is quite painful. Possibly, it might be OK to embed AWS commands in a script, although realistically, if you start going down that road, you will probably want to use a real infrastructure-as-code tool.
For one-offs and experimentations, just using the AWS console probably makes more sense. And for production environments, or if you need traceability and reproducibility, a proper infrastructure-as-code tool such as SAM, CloudFormation, or CDK is definitely a better choice.
