In this blog, we'll explore how to create a serverless application that performs Create, Read, Update, and Delete (CRUD) operations using AWS Lambda and DynamoDB. These four operations are essential for interacting with and manipulating data within the database.
A typical serverless CRUD API consists of three tiers:
- Presentation Tier: API Gateway handles HTTP requests and responses. For an in-depth look at how to set up this tier, read the full article here.
- Logic Tier: Lambda functions process business logic and execute CRUD operations. For detailed information on implementing this tier, read the full article here.
- Data Tier: DynamoDB stores and retrieves data. In this blog, I'll focus on creating and deploying this tier using AWS CloudFormation to automate the process.
By the end of this guide, you'll have a comprehensive understanding of how to build a robust serverless application doing CRUD with dynamoDB.
Prerequisites
- Install Go language
- Install AWS Command Line Interface (CLI)
- We need an AWS account, if you do not have one already then AWS provides a free tier account for a limited time duration, allowing many AWS services to explore for free.
Create DynamoDB table using Cloudformation
In many software landscapes, multiple applications and services share database tables. Deploying and undeploying these tables in the same stack as other resources can potentially disrupt dependent services.
Although it’s not a standard or best practice, I prefer to create a database table in a separate CloudFormation template rather than including it in the main application template, which deploys all other resources like Lambdas, SNS, and SQS.
In the following CloudFormation template, I create a DynamoDB table with a single attribute, id, set as the partition key in the KeySchema. For simplicity, the read and write capacity is set to 5 per second in ProvisionedThroughput, but you can adjust these values as needed.
The template outputs the table’s name (DynamoTableName) and ARN (DynamoARN) so that these values can be used by other templates to create additional resources.
{
"AWSTemplateFormatVersion": "2010-09-09",
"Parameters": {
"pTableName": {
"Type": "String"
}
},
"Resources": {
"myDemoDynamoDBTable": {
"Type": "AWS::DynamoDB::Table",
"Properties": {
"TableName": {
"Ref": "pTableName"
},
"AttributeDefinitions": [
{
"AttributeName": "id",
"AttributeType": "S"
}
],
"KeySchema": [
{
"AttributeName": "id",
"KeyType": "HASH"
}
],
"ProvisionedThroughput": {
"ReadCapacityUnits": "5",
"WriteCapacityUnits": "5"
}
}
}
},
"Outputs": {
"DynamoARN": {
"Value": {
"Fn::GetAtt": [
"myDemoDynamoDBTable",
"Arn"
]
}
},
"DynamoTableName": {
"Description": "Name of DynamoDB table",
"Value": {
"Ref": "myDemoDynamoDBTable"
}
}
}
}I pass the table name (pTableName) as a parameter to cloudformation and deploy it,
aws cloudformation deploy \
--template-file ./deploy/cf-dynamo.json \
--stack-name my-demo-dynamo-stack \
--capabilities CAPABILITY_IAM \
--parameter-overrides \
pTableName="my-demo-dynamo-table" \
--region <your-region-name>Note: replace your-region-name you to your region name, such as eu-west-1 etc.
Lambda Function
Now, we extend the lambda function created in this article to interact with dynamodb. We will trigger lambda function by event in following JSON format,
{
"id": "1",
"title": "Learn AWS with GO language",
"author": "The author"
}to unmarshal above json string we define a Go struct in a new file, mode.go,
package model
type MyBook struct {
ID string `json:"id,omitempty"`
Title string `json:"title,omitempty"`
Author string `json:"author,omitempty"`
}On receiving the event, HandleRequest function creates an instance of DynamoHandler to perform CRUD operations with dynamodb, in following sequence,
- Saves the MyBook object in db table
- Update an attribute value of the saved MyBook in db
- Retrieve the updated item by ID
- Delete the item from db
- Return the updated MyBook object
package main
import (
"context"
"log"
"github.com/aws/aws-lambda-go/lambda"
"github.com/azam-akram/aws-apigateway-lambda-demo-go/dynamo_db"
"github.com/azam-akram/aws-apigateway-lambda-demo-go/model"
)
func HandleRequest(ctx context.Context, book model.MyBook) (*model.MyBook, error) {
log.Println("Received event with Book: ", book)
dynamoHandler := dynamo_db.NewDynamoHandler("my-demo-dynamo-table")
if err := dynamoHandler.Save(&book); err != nil {
log.Fatal("Failed to save item, error: ", err.Error())
}
if err := dynamoHandler.UpdateAttributeByID(book.ID, "author", "Modified Author"); err != nil {
log.Fatal("Failed to update item's value by ID, error: ", err.Error())
}
updatedBook, err := dynamoHandler.GetByID(book.ID)
if err != nil {
log.Fatal("Failed to get item by ID, error: ", err.Error())
}
log.Println("Fetched updated Book from db: ", updatedBook)
err = dynamoHandler.DeleteByID(book.ID)
if err != nil {
log.Fatal("Failed to delete item by ID, error: ", err.Error())
}
log.Println("Item deleted")
return updatedBook, nil
}
func main() {
lambda.Start(HandleRequest)
}
Lambda Function Explaination:
GetDynamoInterface(): creates a new aws session and returns an interface to call dynamodb APIs.
Save(): saves the item in the db table.
UpdateAttributeByID(): updates an attribute value of the object searched by ID.
GetByID(): Retrieves the database item by ID.
DeleteByID(): Delete the item by ID.
convertToDBRecord(): a private function which converts MyBook object to dynamodb item. We need to transform MyBook object to aws dynamodb attribute map, map[string]*dynamodb.AttributeValue{}
DB handling interface
Now we create a layer to work with dynamodb, for that let's define an interface to the implementation dynamo layer,
package dynamo_db
import "github.com/azam-akram/aws-apigateway-lambda-demo-go/model"
type DBHandler interface {
Save(book *model.MyBook) error
Update(book *model.MyBook) error
UpdateAttributeByID(id, key, value string) error
GetByID(id string) (*model.MyBook, error)
DeleteByID(id string) error
}
Following is the implementation of DBHandler interface,
package dynamo_db
import (
"log"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/dynamodb"
"github.com/aws/aws-sdk-go/service/dynamodb/dynamodbattribute"
"github.com/aws/aws-sdk-go/service/dynamodb/dynamodbiface"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/azam-akram/aws-apigateway-lambda-demo-go/model"
)
var handler DBHandler
type DynamoHandler struct {
tableName string
dynamoDBAPI dynamodbiface.DynamoDBAPI
}
func NewDynamoHandler(tName string) DBHandler {
if handler == nil {
handler = &DynamoHandler{
tableName: tName,
dynamoDBAPI: getDynamoInterface(),
}
}
return handler
}
func getDynamoInterface() dynamodbiface.DynamoDBAPI {
dynamoSession := session.Must(session.NewSessionWithOptions(session.Options{
SharedConfigState: session.SharedConfigEnable,
}))
dynamoInstance := dynamodb.New(dynamoSession)
return dynamodbiface.DynamoDBAPI(dynamoInstance)
}
func convertToDBRecord(book *model.MyBook) map[string]*dynamodb.AttributeValue {
item := map[string]*dynamodb.AttributeValue{
"id": {S: &book.ID},
"title": {S: &book.Title},
"author": {S: &book.Author},
}
return item
}
func (h *DynamoHandler) Save(book *model.MyBook) error {
input := &dynamodb.PutItemInput{
Item: convertToDBRecord(book),
TableName: aws.String(h.tableName),
}
savedItem, err := h.dynamoDBAPI.PutItem(input)
if err != nil {
log.Fatal("Failed to save Item: ", err.Error())
return err
}
log.Println("Item saved in db: ", savedItem)
return nil
}
func (h *DynamoHandler) Update(book *model.MyBook) error {
item, err := dynamodbattribute.MarshalMap(book)
if err != nil {
return err
}
input := &dynamodb.PutItemInput{
Item: item,
TableName: aws.String(h.tableName),
}
updatedItem, err := h.dynamoDBAPI.PutItem(input)
if err != nil {
return err
}
log.Println("Item updated in db: ", updatedItem)
return nil
}
func (h *DynamoHandler) UpdateAttributeByID(id, key, value string) error {
input := dynamodb.UpdateItemInput{
ExpressionAttributeValues: map[string]*dynamodb.AttributeValue{
":val": {
S: aws.String(value),
},
},
Key: map[string]*dynamodb.AttributeValue{
"id": {
S: aws.String(id),
},
},
TableName: aws.String(h.tableName),
UpdateExpression: aws.String("set " + key + " = :val"),
}
output, err := h.dynamoDBAPI.UpdateItem(&input)
if err != nil {
return err
}
log.Println("Item updated in db: ", output)
return nil
}
func (h *DynamoHandler) GetByID(id string) (*model.MyBook, error) {
input := &dynamodb.GetItemInput{
TableName: aws.String(h.tableName),
Key: map[string]*dynamodb.AttributeValue{
"id": {S: aws.String(id)},
},
}
item, err := h.dynamoDBAPI.GetItem(input)
if err != nil {
return nil, err
}
if item.Item == nil {
log.Fatal("Can't get item by id = ", id)
return nil, nil
}
var book model.MyBook
err = dynamodbattribute.UnmarshalMap(item.Item, &book)
if err != nil {
return nil, err
}
return &book, nil
}
func (h *DynamoHandler) DeleteByID(id string) error {
input := &dynamodb.DeleteItemInput{
TableName: aws.String(h.tableName),
Key: map[string]*dynamodb.AttributeValue{
"id": {S: aws.String(id)},
},
}
_, err := h.dynamoDBAPI.DeleteItem(input)
if err != nil {
log.Fatal("Can't delete item by id = ", id)
return err
}
return nil
}
Build the module
GOOS=linux go build main.go
Selecting linux ensures that the compiled executable is compatible with the Go runtime, even if you compile it in a non-Linux environment[1].
Use build-lambda-zip to create a deployment archive package (.zip file),
build-lambda-zip.exe -o aws-lambda-dynamo-demo-go.zip main
Upload zip file to aws s3
aws s3 cp aws-lambda-dynamo-demo-go.zip s3://my-demo-s3-bucket/demo-lambda/aws-lambda-dynamo-demo-go.zip
Deploy lambda function Cloudformation stack:
Extending the lambda function created in the previous article, here I will add a new policy in the lambda function role to do CRUD operations to the dynamodb table.
{
"PolicyName": "lambdaDynamoPolicy",
"PolicyDocument":
{
"Version": "2012-10-17",
"Statement":
[
{
"Effect": "Allow",
"Action":
[
"dynamodb:GetItem",
"dynamodb:UpdateItem",
"dynamodb:Query",
"dynamodb:PutItem",
"dynamodb:DeleteItem"
],
"Resource":
[
{
"Ref": "pDynamoARN"
}
]
}
]
}
}
See the full cloudformation template here.
Deploy
aws cloudformation deploy \
--template-file ./deploy/cf.json \
--stack-name my-demo-lambda-dynamo-stack \
--capabilities CAPABILITY_IAM \
--parameter-overrides \
pLambdaCodeBucket=my-demo-s3-bucket \
pLambdaCodeS3KeyPath=demo-lambda/aws-lambda-dynamo-demo-go.zip \
pDynamoARN=arn:aws:dynamodb:eu-west-1:<<account-id>>:table/my-demo-dynamo-table
Cloudformation deploy should be triggered,
After completion of the deployment, a Lambda function should be created with provisions for interacting with DynamoDB.
Test lambda function
Go to Lambda function in AWS Management Console, and Test by following input,
{
"id": "1",
"title": "Learn AWS with GO language",
"author": "The author"
}
HandleRequest function should save, update, retrieve and delete the item in db. We could verify our test in cloudwatch logs (not very pretty logging in this example),
Hope after following my previous and this document, the reader would have some hands-on experience of AWS Lambda function doing some database operations with DynamoDB.
