Implementing CRUD Operations with Go in AWS Lambda and DynamoDB

AWS, GO

Implementing CRUD Operations with Go in AWS Lambda and DynamoDB

By Azam Akram,

January 4, 2023

In this blog, we'll explore how to create a serverless application that performs Create, Read, Update, and Delete (CRUD) operations using AWS Lambda and DynamoDB. These four operations are essential for interacting with and manipulating data within the database.

A typical serverless CRUD API consists of three tiers:

Presentation Tier: API Gateway handles HTTP requests and responses. For an in-depth look at how to set up this tier, read the full article here.
Logic Tier: Lambda functions process business logic and execute CRUD operations. For detailed information on implementing this tier, read the full article here.
Data Tier: DynamoDB stores and retrieves data. In this blog, I'll focus on creating and deploying this tier using AWS CloudFormation to automate the process.

By the end of this guide, you'll have a comprehensive understanding of how to build a robust serverless application doing CRUD with dynamoDB.

Prerequisites

Install Go language
Install AWS Command Line Interface (CLI)
We need an AWS account, if you do not have one already then AWS provides a free tier account for a limited time duration, allowing many AWS services to explore for free.

Create DynamoDB table using Cloudformation

In many software landscapes, multiple applications and services share database tables. Deploying and undeploying these tables in the same stack as other resources can potentially disrupt dependent services.

Although it’s not a standard or best practice, I prefer to create a database table in a separate CloudFormation template rather than including it in the main application template, which deploys all other resources like Lambdas, SNS, and SQS.

In the following CloudFormation template, I create a DynamoDB table with a single attribute, id, set as the partition key in the KeySchema. For simplicity, the read and write capacity is set to 5 per second in ProvisionedThroughput, but you can adjust these values as needed.

The template outputs the table’s name (DynamoTableName) and ARN (DynamoARN) so that these values can be used by other templates to create additional resources.

{
    "AWSTemplateFormatVersion": "2010-09-09",
    "Parameters": {
        "pTableName": {
            "Type": "String"
        }
    },
    "Resources": {
        "myDemoDynamoDBTable": {
            "Type": "AWS::DynamoDB::Table",
            "Properties": {
                "TableName": {
                    "Ref": "pTableName"
                },
                "AttributeDefinitions": [
                    {
                        "AttributeName": "id",
                        "AttributeType": "S"
                    }
                ],
                "KeySchema": [
                    {
                        "AttributeName": "id",
                        "KeyType": "HASH"
                    }
                ],
                "ProvisionedThroughput": {
                    "ReadCapacityUnits": "5",
                    "WriteCapacityUnits": "5"
                }
            }
        }
    },
    "Outputs": {
        "DynamoARN": {
            "Value": {
                "Fn::GetAtt": [
                    "myDemoDynamoDBTable",
                    "Arn"
                ]
            }
        },
        "DynamoTableName": {
            "Description": "Name of DynamoDB table",
            "Value": {
                "Ref": "myDemoDynamoDBTable"
            }
        }
    }
}

I pass the table name (pTableName) as a parameter to cloudformation and deploy it,

aws cloudformation deploy \
--template-file ./deploy/cf-dynamo.json \
--stack-name my-demo-dynamo-stack \
--capabilities CAPABILITY_IAM \
--parameter-overrides \
  pTableName="my-demo-dynamo-table" \
--region <your-region-name>

Note: replace your-region-name you to your region name, such as eu-west-1 etc.

Lambda Function

Now, we extend the lambda function created in this article to interact with dynamodb. We will trigger lambda function by event in following JSON format,

{
  "id": "1",
  "title": "Learn AWS with GO language",
  "author": "The author"
}

to unmarshal above json string we define a Go struct in a new file, mode.go,

package model

type MyBook struct {
	ID     string `json:"id,omitempty"`
	Title  string `json:"title,omitempty"`
	Author string `json:"author,omitempty"`
}

On receiving the event, HandleRequest function creates an instance of DynamoHandler to perform CRUD operations with dynamodb, in following sequence,

Saves the MyBook object in db table
Update an attribute value of the saved MyBook in db
Retrieve the updated item by ID
Delete the item from db
Return the updated MyBook object

package main

import (
	"context"
	"log"

	"github.com/aws/aws-lambda-go/lambda"

	"github.com/azam-akram/aws-apigateway-lambda-demo-go/dynamo_db"
	"github.com/azam-akram/aws-apigateway-lambda-demo-go/model"
)

func HandleRequest(ctx context.Context, book model.MyBook) (*model.MyBook, error) {
	log.Println("Received event with Book: ", book)

	dynamoHandler := dynamo_db.NewDynamoHandler("my-demo-dynamo-table")
	if err := dynamoHandler.Save(&book); err != nil {
		log.Fatal("Failed to save item, error: ", err.Error())
	}

	if err := dynamoHandler.UpdateAttributeByID(book.ID, "author", "Modified Author"); err != nil {
		log.Fatal("Failed to update item's value by ID, error: ", err.Error())
	}

	updatedBook, err := dynamoHandler.GetByID(book.ID)
	if err != nil {
		log.Fatal("Failed to get item by ID, error: ", err.Error())
	}

	log.Println("Fetched updated Book from db: ", updatedBook)

	err = dynamoHandler.DeleteByID(book.ID)
	if err != nil {
		log.Fatal("Failed to delete item by ID, error: ", err.Error())
	}

	log.Println("Item deleted")

	return updatedBook, nil
}

func main() {
	lambda.Start(HandleRequest)
}

Lambda Function Explaination

GetDynamoInterface(): creates a new aws session and returns an interface to call dynamodb APIs.

Save(): saves the item in the db table.

UpdateAttributeByID(): updates an attribute value of the object searched by ID.

GetByID(): Retrieves the database item by ID.

DeleteByID(): Delete the item by ID.

convertToDBRecord(): a private function which converts MyBook object to dynamodb item. We need to transform MyBook object to aws dynamodb attribute map, map[string]*dynamodb.AttributeValue{}

DB handling interface

Now we create a layer to work with dynamodb, for that let's define an interface to the implementation dynamo layer,

package dynamo_db

import "github.com/azam-akram/aws-apigateway-lambda-demo-go/model"

type DBHandler interface {
	Save(book *model.MyBook) error
	Update(book *model.MyBook) error
	UpdateAttributeByID(id, key, value string) error
	GetByID(id string) (*model.MyBook, error)
	DeleteByID(id string) error
}

DB Interface Implementation

Following is the implementation of DBHandler interface,

package dynamo_db

import (
	"log"

	"github.com/aws/aws-sdk-go/aws"
	"github.com/aws/aws-sdk-go/service/dynamodb"
	"github.com/aws/aws-sdk-go/service/dynamodb/dynamodbattribute"
	"github.com/aws/aws-sdk-go/service/dynamodb/dynamodbiface"

	"github.com/aws/aws-sdk-go/aws/session"
	"github.com/azam-akram/aws-apigateway-lambda-demo-go/model"
)

var handler DBHandler

type DynamoHandler struct {
	tableName   string
	dynamoDBAPI dynamodbiface.DynamoDBAPI
}

func NewDynamoHandler(tName string) DBHandler {
	if handler == nil {
		handler = &DynamoHandler{
			tableName:   tName,
			dynamoDBAPI: getDynamoInterface(),
		}
	}
	return handler
}

func getDynamoInterface() dynamodbiface.DynamoDBAPI {
	dynamoSession := session.Must(session.NewSessionWithOptions(session.Options{
		SharedConfigState: session.SharedConfigEnable,
	}))

	dynamoInstance := dynamodb.New(dynamoSession)

	return dynamodbiface.DynamoDBAPI(dynamoInstance)
}

func convertToDBRecord(book *model.MyBook) map[string]*dynamodb.AttributeValue {
	item := map[string]*dynamodb.AttributeValue{
		"id":     {S: &book.ID},
		"title":  {S: &book.Title},
		"author": {S: &book.Author},
	}
	return item
}

Save

func (h *DynamoHandler) Save(book *model.MyBook) error {
	input := &dynamodb.PutItemInput{
		Item:      convertToDBRecord(book),
		TableName: aws.String(h.tableName),
	}

	savedItem, err := h.dynamoDBAPI.PutItem(input)
	if err != nil {
		log.Fatal("Failed to save Item: ", err.Error())
		return err
	}

	log.Println("Item saved in db: ", savedItem)

	return nil
}

Update

func (h *DynamoHandler) Update(book *model.MyBook) error {
	item, err := dynamodbattribute.MarshalMap(book)
	if err != nil {
		return err
	}

	input := &dynamodb.PutItemInput{
		Item:      item,
		TableName: aws.String(h.tableName),
	}

	updatedItem, err := h.dynamoDBAPI.PutItem(input)
	if err != nil {
		return err
	}

	log.Println("Item updated in db: ", updatedItem)
	return nil
}

Update By ID

func (h *DynamoHandler) UpdateAttributeByID(id, key, value string) error {
	input := dynamodb.UpdateItemInput{
		ExpressionAttributeValues: map[string]*dynamodb.AttributeValue{
			":val": {
				S: aws.String(value),
			},
		},
		Key: map[string]*dynamodb.AttributeValue{
			"id": {
				S: aws.String(id),
			},
		},
		TableName:        aws.String(h.tableName),
		UpdateExpression: aws.String("set " + key + " = :val"),
	}

	output, err := h.dynamoDBAPI.UpdateItem(&input)
	if err != nil {
		return err
	}

	log.Println("Item updated in db: ", output)

	return nil
}

Get By ID

func (h *DynamoHandler) GetByID(id string) (*model.MyBook, error) {
	input := &dynamodb.GetItemInput{
		TableName: aws.String(h.tableName),
		Key: map[string]*dynamodb.AttributeValue{
			"id": {S: aws.String(id)},
		},
	}

	item, err := h.dynamoDBAPI.GetItem(input)
	if err != nil {
		return nil, err
	}

	if item.Item == nil {
		log.Fatal("Can't get item by id = ", id)
		return nil, nil
	}

	var book model.MyBook
	err = dynamodbattribute.UnmarshalMap(item.Item, &book)
	if err != nil {
		return nil, err
	}

	return &book, nil
}

Delete

func (h *DynamoHandler) DeleteByID(id string) error {
	input := &dynamodb.DeleteItemInput{
		TableName: aws.String(h.tableName),
		Key: map[string]*dynamodb.AttributeValue{
			"id": {S: aws.String(id)},
		},
	}

	_, err := h.dynamoDBAPI.DeleteItem(input)
	if err != nil {
		log.Fatal("Can't delete item by id = ", id)
		return err
	}

	return nil
}

Build the module

GOOS=linux go build main.go

Selecting linux ensures that the compiled executable is compatible with the Go runtime, even if you compile it in a non-Linux environment[1].

Use build-lambda-zip to create a deployment archive package (.zip file),

build-lambda-zip.exe -o aws-lambda-dynamo-demo-go.zip main

Upload zip file to aws s3

aws s3 cp aws-lambda-dynamo-demo-go.zip s3://my-demo-s3-bucket/demo-lambda/aws-lambda-dynamo-demo-go.zip

Deploy lambda function Cloudformation stack:

Extending the lambda function created in the previous article, here I will add a new policy in the lambda function role to do CRUD operations to the dynamodb table.

{
    "PolicyName": "lambdaDynamoPolicy",
    "PolicyDocument":
    {
        "Version": "2012-10-17",
        "Statement":
        [
            {
                "Effect": "Allow",
                "Action":
                [
                    "dynamodb:GetItem",
                    "dynamodb:UpdateItem",
                    "dynamodb:Query",
                    "dynamodb:PutItem",
                    "dynamodb:DeleteItem"
                ],
                "Resource":
                [
                    {
                        "Ref": "pDynamoARN"
                    }
                ]
            }
        ]
    }
}

See the full cloudformation template here.

Deploy

aws cloudformation deploy \
--template-file ./deploy/cf.json \
--stack-name my-demo-lambda-dynamo-stack \
--capabilities CAPABILITY_IAM \
--parameter-overrides \
    pLambdaCodeBucket=my-demo-s3-bucket \
    pLambdaCodeS3KeyPath=demo-lambda/aws-lambda-dynamo-demo-go.zip \
    pDynamoARN=arn:aws:dynamodb:eu-west-1:<<account-id>>:table/my-demo-dynamo-table

Cloudformation deploy should be triggered,

After completion of the deployment, a Lambda function should be created with provisions for interacting with DynamoDB.

Test lambda function

Go to Lambda function in AWS Management Console, and Test by following input,

{
  "id": "1",
  "title": "Learn AWS with GO language",
  "author": "The author"
}

HandleRequest function should save, update, retrieve and delete the item in db. We could verify our test in cloudwatch logs (not very pretty logging in this example),

Hope after following my previous and this document, the reader would have some hands-on experience of AWS Lambda function doing some database operations with DynamoDB.