aws-lambda-dynamodb-go-v1

In most database systems, developers use CRUD, which stands for Create, Read, Update, and Delete, to manage data through four basic operations. This article will help the developers writing CRUD in AWS Lambda and DynamoDB with Go.

Before we continue, I recommend reading about creating and deploying AWS Lambda functions in Go language to gain a better understanding. This will provide a foundation for the concepts discussed in this article, as I will be building upon them.

Prerequisites:

I will reuse the Lambda function created in this article to perform CRUD operations. Additionally, I will create a DynamoDB table in a separate CloudFormation stack.

Create DynamoDB table using Cloudformation

In many software landscapes, multiple applications and services share database tables. Deploying and undeploying database tables in the same stack as other resources could potentially disrupt some dependent services.

This is not a standard or best practice but I prefer to create a db table in a separate cloudformation template, instead of creating it in the “main” system template, which deploys all other resources like lambdas, SNS, SQS etc.

In the following CF template, I create a dynamodb table, with a single attribute id. We make id a partition key in the KeySchema and for simplicity set read and write capacity as 5 per second in ProvisionedThroughput. You can set these values as per requirements.

I output table’s Name (DynamoTableName) and ARN (DynamoARN), so that these values can be used by other template (to create other resources).

{
"AWSTemplateFormatVersion": "2010-09-09",
"Parameters": {
"pTableName": {
"Type": "String"
}
},
"Resources": {
"myDemoDynamoDBTable": {
"Type": "AWS::DynamoDB::Table",
"Properties": {
"TableName": {
"Ref": "pTableName"
},
"AttributeDefinitions": [
{
"AttributeName": "id",
"AttributeType": "S"
}
],
"KeySchema": [
{
"AttributeName": "id",
"KeyType": "HASH"
}
],
"ProvisionedThroughput": {
"ReadCapacityUnits": "5",
"WriteCapacityUnits": "5"
}
}
}
},
"Outputs": {
"DynamoARN": {
"Value": {
"Fn::GetAtt": [
"myDemoDynamoDBTable",
"Arn"
]
}
},
"DynamoTableName": {
"Description": "Name of DynamoDB table",
"Value": {
"Ref": "myDemoDynamoDBTable"
}
}
}
}

I pass the table name (pTableName) as a parameter to cloudformation and deploy it,

aws cloudformation deploy \
--template-file ./deploy/cf-dynamo.json \
--stack-name my-demo-dynamo-stack \
--capabilities CAPABILITY_IAM \
--parameter-overrides \
pTableName="my-demo-dynamo-table"

Now, extend the lambda function HandleRequest (created in this article) to trigger it with an event containing a JSON object MyBook.

type MyBook struct {
ID string `json:"id,omitempty"`
Title string `json:"title,omitempty"`
Author string `json:"author,omitempty"`
}

Lambda function

HandleRequest function creates an instance of DynamoHandler, which includes the CRUD implementation.

On receiving the request, HandleRequest() perform CRUD operations to dynamodb in following sequence,

  • Saves the MyBook object in db table
  • Update an attribute value of the saved MyBook in db
  • Retrieve the updated item by ID
  • Delete the item from db
  • Return the updated MyBook object
func HandleRequest(ctx context.Context, book MyBook) (*MyBook, error) {
log.Println("Received event with Book: ", book)

dynamoHandler := dynamo_db.NewDynamoHandler()
if err := dynamoHandler.Save(&book); err != nil {
log.Fatal("Failed to save item, error: ", err.Error())
}

if err := dynamoHandler.UpdateAttributeByID(book.ID, "author", "Modified Author"); err != nil {
log.Fatal("Failed to update item's value by ID, error: ", err.Error())
}

updatedBook, err := dynamoHandler.GetByID(book.ID)
if err != nil {
log.Fatal("Failed to get item by ID, error: ", err.Error())
}

log.Println("Fetched updated Book from db: ", updatedBook)

err = dynamoHandler.DeleteByID(book.ID)
if err != nil {
log.Fatal("Failed to delete item by ID, error: ", err.Error())
}

log.Println("Item deleted")

return updatedBook, nil
}

func main() {
lambda.Start(HandleRequest)
}

DB handling interface

type Handler interface {
Save(book *MyBook) error
UpdateAttributeByID(id, key, value string) error
GetByID(id string) (*MyBook, error)
DeleteByID(id string) error
}

DB handling implementation

GetDynamoInterface(): creates a new aws session and returns an interface to call dynamodb APIs.

Save(): saves the item in the db table.

UpdateAttributeByID(): updates an attribute value of the object searched by ID.

GetByID(): Retrieves the database item by ID.

DeleteByID(): Delete the item by ID.

convertToDBRecord(): a private function which converts MyBook object to dynamodb item. We need to transform MyBook object to aws dynamodb attribute map, map[string]*dynamodb.AttributeValue{}

package dynamo_db

import (
"log"

"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/dynamodb"
"github.com/aws/aws-sdk-go/service/dynamodb/dynamodbattribute"
"github.com/aws/aws-sdk-go/service/dynamodb/dynamodbiface"
"github.com/aws/aws-sdk-go/aws/session"
)

type MyBook struct {
ID string `json:"id,omitempty"`
Title string `json:"title,omitempty"`
Author string `json:"author,omitempty"`
}

var handler Handler

type DynamoHandler struct {
TableName string
DynamoDBAPI dynamodbiface.DynamoDBAPI
}

func NewDynamoHandler() Handler {
if handler == nil {
handler = &DynamoHandler{
TableName: "my-demo-dynamo-table",
DynamoDBAPI: GetDynamoInterface(),
}
}
return handler
}

func GetDynamoInterface() dynamodbiface.DynamoDBAPI {
dynamoSession := session.Must(session.NewSessionWithOptions(session.Options{
SharedConfigState: session.SharedConfigEnable,
}))

dynamoInstance := dynamodb.New(dynamoSession)

return dynamodbiface.DynamoDBAPI(dynamoInstance)
}

func convertToDBRecord(book *MyBook) map[string]*dynamodb.AttributeValue {
item := map[string]*dynamodb.AttributeValue{
"id": {S: &book.ID},
"title": {S: &book.Title},
"author": {S: &book.Author},
}
return item
}

func (h *DynamoHandler) Save(book *MyBook) error {
input := &dynamodb.PutItemInput{
Item: convertToDBRecord(book),
TableName: aws.String(h.TableName),
}

savedItem, err := h.DynamoDBAPI.PutItem(input)
if err != nil {
log.Fatal("Failed to save Item: ", err.Error())
return err
}

log.Println("Item saved in db: ", savedItem)

return nil
}

func (h *DynamoHandler) UpdateAttributeByID(id, key, value string) error {
input := dynamodb.UpdateItemInput{
ExpressionAttributeValues: map[string]*dynamodb.AttributeValue{
":val": {
S: aws.String(value),
},
},
Key: map[string]*dynamodb.AttributeValue{
"id": {
S: aws.String(id),
},
},
TableName: aws.String(h.TableName),
UpdateExpression: aws.String("set " + key + " = :val"),
}

output, err := h.DynamoDBAPI.UpdateItem(&input)
if err != nil {
return err
}

log.Println("Item updated in db: ", output)

return nil
}

func (h *DynamoHandler) GetByID(id string) (*MyBook, error) {
input := &dynamodb.GetItemInput{
TableName: aws.String(h.TableName),
Key: map[string]*dynamodb.AttributeValue{
"id": {S: aws.String(id)},
},
}

item, err := h.DynamoDBAPI.GetItem(input)
if err != nil {
return nil, err
}

if item.Item == nil {
log.Fatal("Can't get item by id = ", id)
return nil, nil
}

var book MyBook
err = dynamodbattribute.UnmarshalMap(item.Item, &book)
if err != nil {
return nil, err
}

return &book, nil
}

func (h *DynamoHandler) DeleteByID(id string) error {
input := &dynamodb.DeleteItemInput{
TableName: aws.String(h.TableName),
Key: map[string]*dynamodb.AttributeValue{
"id": {S: aws.String(id)},
},
}

_, err := h.DynamoDBAPI.DeleteItem(input)
if err != nil {
log.Fatal("Can't delete item by id = ", id)
return err
}

return nil
}

Build the module

GOOS=linux go build main.go

Selecting linux ensures that the compiled executable is compatible with the Go runtime, even if you compile it in a non-Linux environment[1].

Use build-lambda-zip to create a deployment archive package (.zip file),

build-lambda-zip.exe -o aws-lambda-dynamo-demo-go.zip main

Upload zip file to aws s3

aws s3 cp aws-lambda-dynamo-demo-go.zip s3://my-demo-s3-bucket/demo-lambda/aws-lambda-dynamo-demo-go.zip

Deploy lambda function Cloudformation stack:

Extending the lambda function created in the previous article, here I will add a new policy in the lambda function role to do CRUD operations to the dynamodb table.

{
"PolicyName": "lambdaDynamoPolicy",
"PolicyDocument":
{
"Version": "2012-10-17",
"Statement":
[
{
"Effect": "Allow",
"Action":
[
"dynamodb:GetItem",
"dynamodb:UpdateItem",
"dynamodb:Query",
"dynamodb:PutItem",
"dynamodb:DeleteItem"
],
"Resource":
[
{
"Ref": "pDynamoARN"
}
]
}
]
}
}
See the full cloudformation template here.

Deploy

aws cloudformation deploy \
--template-file ./deploy/cf.json \
--stack-name my-demo-lambda-dynamo-stack \
--capabilities CAPABILITY_IAM \
--parameter-overrides \
pLambdaCodeBucket=my-demo-s3-bucket \
pLambdaCodeS3KeyPath=demo-lambda/aws-lambda-dynamo-demo-go.zip \
pDynamoARN=arn:aws:dynamodb:eu-west-1:<<account-id>>:table/my-demo-dynamo-table

Cloudformation deploy should be triggered,

After completion of the deployment, a Lambda function should be created with provisions for interacting with DynamoDB.

Test lambda function:

Go to Lambda function in AWS Management Console, and Test by following input,

{
"id": "1",
"title": "Learn AWS with GO language",
"author": "The author"
}

HandleRequest function should save, update, retrieve and delete the item in db. We could verify our test in cloudwatch logs (not very pretty logging in this example),

Hope after following my previous and this document, the reader would have some hands-on experience of AWS Lambda function doing some database operations with DynamoDB.

In most database systems, developers use CRUD, which stands for Create, Read, Update, and Delete, to manage data through four basic operations. This article will help the developers writing CRUD in AWS Lambda and DynamoDB with Go.

Before we continue, I recommend reading about creating and deploying AWS Lambda functions in Go language to gain a better understanding. This will provide a foundation for the concepts discussed in this article, as I will be building upon them.

Prerequisites:

I will reuse the Lambda function created in this article to perform CRUD operations. Additionally, I will create a DynamoDB table in a separate CloudFormation stack.

Create DynamoDB table using Cloudformation

In many software landscapes, multiple applications and services share database tables. Deploying and undeploying database tables in the same stack as other resources could potentially disrupt some dependent services.

This is not a standard or best practice but I prefer to create a db table in a separate cloudformation template, instead of creating it in the “main” system template, which deploys all other resources like lambdas, SNS, SQS etc.

In the following CF template, I create a dynamodb table, with a single attribute id. We make id a partition key in the KeySchema and for simplicity set read and write capacity as 5 per second in ProvisionedThroughput. You can set these values as per requirements.

I output table’s Name (DynamoTableName) and ARN (DynamoARN), so that these values can be used by other template (to create other resources).

{
"AWSTemplateFormatVersion": "2010-09-09",
"Parameters": {
"pTableName": {
"Type": "String"
}
},
"Resources": {
"myDemoDynamoDBTable": {
"Type": "AWS::DynamoDB::Table",
"Properties": {
"TableName": {
"Ref": "pTableName"
},
"AttributeDefinitions": [
{
"AttributeName": "id",
"AttributeType": "S"
}
],
"KeySchema": [
{
"AttributeName": "id",
"KeyType": "HASH"
}
],
"ProvisionedThroughput": {
"ReadCapacityUnits": "5",
"WriteCapacityUnits": "5"
}
}
}
},
"Outputs": {
"DynamoARN": {
"Value": {
"Fn::GetAtt": [
"myDemoDynamoDBTable",
"Arn"
]
}
},
"DynamoTableName": {
"Description": "Name of DynamoDB table",
"Value": {
"Ref": "myDemoDynamoDBTable"
}
}
}
}

I pass the table name (pTableName) as a parameter to cloudformation and deploy it,

aws cloudformation deploy \
--template-file ./deploy/cf-dynamo.json \
--stack-name my-demo-dynamo-stack \
--capabilities CAPABILITY_IAM \
--parameter-overrides \
pTableName="my-demo-dynamo-table"

Now, extend the lambda function HandleRequest (created in this article) to trigger it with an event containing a JSON object MyBook.

type MyBook struct {
ID string `json:"id,omitempty"`
Title string `json:"title,omitempty"`
Author string `json:"author,omitempty"`
}

Lambda function

HandleRequest function creates an instance of DynamoHandler, which includes the CRUD implementation.

On receiving the request, HandleRequest() perform CRUD operations to dynamodb in following sequence,

  • Saves the MyBook object in db table
  • Update an attribute value of the saved MyBook in db
  • Retrieve the updated item by ID
  • Delete the item from db
  • Return the updated MyBook object
func HandleRequest(ctx context.Context, book MyBook) (*MyBook, error) {
log.Println("Received event with Book: ", book)

dynamoHandler := dynamo_db.NewDynamoHandler()
if err := dynamoHandler.Save(&book); err != nil {
log.Fatal("Failed to save item, error: ", err.Error())
}

if err := dynamoHandler.UpdateAttributeByID(book.ID, "author", "Modified Author"); err != nil {
log.Fatal("Failed to update item's value by ID, error: ", err.Error())
}

updatedBook, err := dynamoHandler.GetByID(book.ID)
if err != nil {
log.Fatal("Failed to get item by ID, error: ", err.Error())
}

log.Println("Fetched updated Book from db: ", updatedBook)

err = dynamoHandler.DeleteByID(book.ID)
if err != nil {
log.Fatal("Failed to delete item by ID, error: ", err.Error())
}

log.Println("Item deleted")

return updatedBook, nil
}

func main() {
lambda.Start(HandleRequest)
}

DB handling interface

type Handler interface {
Save(book *MyBook) error
UpdateAttributeByID(id, key, value string) error
GetByID(id string) (*MyBook, error)
DeleteByID(id string) error
}

DB handling implementation

GetDynamoInterface(): creates a new aws session and returns an interface to call dynamodb APIs.

Save(): saves the item in the db table.

UpdateAttributeByID(): updates an attribute value of the object searched by ID.

GetByID(): Retrieves the database item by ID.

DeleteByID(): Delete the item by ID.

convertToDBRecord(): a private function which converts MyBook object to dynamodb item. We need to transform MyBook object to aws dynamodb attribute map, map[string]*dynamodb.AttributeValue{}

package dynamo_db

import (
"log"

"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/dynamodb"
"github.com/aws/aws-sdk-go/service/dynamodb/dynamodbattribute"
"github.com/aws/aws-sdk-go/service/dynamodb/dynamodbiface"
"github.com/aws/aws-sdk-go/aws/session"
)

type MyBook struct {
ID string `json:"id,omitempty"`
Title string `json:"title,omitempty"`
Author string `json:"author,omitempty"`
}

var handler Handler

type DynamoHandler struct {
TableName string
DynamoDBAPI dynamodbiface.DynamoDBAPI
}

func NewDynamoHandler() Handler {
if handler == nil {
handler = &DynamoHandler{
TableName: "my-demo-dynamo-table",
DynamoDBAPI: GetDynamoInterface(),
}
}
return handler
}

func GetDynamoInterface() dynamodbiface.DynamoDBAPI {
dynamoSession := session.Must(session.NewSessionWithOptions(session.Options{
SharedConfigState: session.SharedConfigEnable,
}))

dynamoInstance := dynamodb.New(dynamoSession)

return dynamodbiface.DynamoDBAPI(dynamoInstance)
}

func convertToDBRecord(book *MyBook) map[string]*dynamodb.AttributeValue {
item := map[string]*dynamodb.AttributeValue{
"id": {S: &book.ID},
"title": {S: &book.Title},
"author": {S: &book.Author},
}
return item
}

func (h *DynamoHandler) Save(book *MyBook) error {
input := &dynamodb.PutItemInput{
Item: convertToDBRecord(book),
TableName: aws.String(h.TableName),
}

savedItem, err := h.DynamoDBAPI.PutItem(input)
if err != nil {
log.Fatal("Failed to save Item: ", err.Error())
return err
}

log.Println("Item saved in db: ", savedItem)

return nil
}

func (h *DynamoHandler) UpdateAttributeByID(id, key, value string) error {
input := dynamodb.UpdateItemInput{
ExpressionAttributeValues: map[string]*dynamodb.AttributeValue{
":val": {
S: aws.String(value),
},
},
Key: map[string]*dynamodb.AttributeValue{
"id": {
S: aws.String(id),
},
},
TableName: aws.String(h.TableName),
UpdateExpression: aws.String("set " + key + " = :val"),
}

output, err := h.DynamoDBAPI.UpdateItem(&input)
if err != nil {
return err
}

log.Println("Item updated in db: ", output)

return nil
}

func (h *DynamoHandler) GetByID(id string) (*MyBook, error) {
input := &dynamodb.GetItemInput{
TableName: aws.String(h.TableName),
Key: map[string]*dynamodb.AttributeValue{
"id": {S: aws.String(id)},
},
}

item, err := h.DynamoDBAPI.GetItem(input)
if err != nil {
return nil, err
}

if item.Item == nil {
log.Fatal("Can't get item by id = ", id)
return nil, nil
}

var book MyBook
err = dynamodbattribute.UnmarshalMap(item.Item, &book)
if err != nil {
return nil, err
}

return &book, nil
}

func (h *DynamoHandler) DeleteByID(id string) error {
input := &dynamodb.DeleteItemInput{
TableName: aws.String(h.TableName),
Key: map[string]*dynamodb.AttributeValue{
"id": {S: aws.String(id)},
},
}

_, err := h.DynamoDBAPI.DeleteItem(input)
if err != nil {
log.Fatal("Can't delete item by id = ", id)
return err
}

return nil
}

Build the module

GOOS=linux go build main.go

Selecting linux ensures that the compiled executable is compatible with the Go runtime, even if you compile it in a non-Linux environment[1].

Use build-lambda-zip to create a deployment archive package (.zip file),

build-lambda-zip.exe -o aws-lambda-dynamo-demo-go.zip main

Upload zip file to aws s3

aws s3 cp aws-lambda-dynamo-demo-go.zip s3://my-demo-s3-bucket/demo-lambda/aws-lambda-dynamo-demo-go.zip

Deploy lambda function Cloudformation stack:

Extending the lambda function created in the previous article, here I will add a new policy in the lambda function role to do CRUD operations to the dynamodb table.

{
"PolicyName": "lambdaDynamoPolicy",
"PolicyDocument":
{
"Version": "2012-10-17",
"Statement":
[
{
"Effect": "Allow",
"Action":
[
"dynamodb:GetItem",
"dynamodb:UpdateItem",
"dynamodb:Query",
"dynamodb:PutItem",
"dynamodb:DeleteItem"
],
"Resource":
[
{
"Ref": "pDynamoARN"
}
]
}
]
}
}
See the full cloudformation template here.

Deploy

aws cloudformation deploy \
--template-file ./deploy/cf.json \
--stack-name my-demo-lambda-dynamo-stack \
--capabilities CAPABILITY_IAM \
--parameter-overrides \
pLambdaCodeBucket=my-demo-s3-bucket \
pLambdaCodeS3KeyPath=demo-lambda/aws-lambda-dynamo-demo-go.zip \
pDynamoARN=arn:aws:dynamodb:eu-west-1:<<account-id>>:table/my-demo-dynamo-table

Cloudformation deploy should be triggered,

After completion of the deployment, a Lambda function should be created with provisions for interacting with DynamoDB.

Test lambda function:

Go to Lambda function in AWS Management Console, and Test by following input,

{
"id": "1",
"title": "Learn AWS with GO language",
"author": "The author"
}

HandleRequest function should save, update, retrieve and delete the item in db. We could verify our test in cloudwatch logs (not very pretty logging in this example),

Hope after following my previous and this document, the reader would have some hands-on experience of AWS Lambda function doing some database operations with DynamoDB.

© 2024 Solution Toolkit . All rights reserved.