The most common but powerful JSON-String conversions in GO

As a Go programmer, mastering GO JSON String conversions is a pivotal skill. This article explains how a typical GO application manipulates a JSON (Javascript Object Notation) object in difference scenarios. It compiles the most common but very useful JSON-String conversions at one place, and presents the examples with the links to the running code. 

I assume the readers have some understanding of JSON marshaling, and Go language structures and interfaces. You can learn about how to start with GO language programming here.

https://github.com/azam-akram/utils-go/tree/main/json_utils

Let’s jump straight to the code..

The following interface exposes all the necessary methods for handling various JSON operations. Each method name is reflecting its purpose,

type Handler interface {
	StringToStruct(s string, i interface{}) error
	StructToString(i interface{}) (string, error)
	StringToMap(s string) (map[string]interface{}, error)
	MapToString(m map[string]interface{}) (string, error)
	BytesToString(jsonBytes []byte) string
	StringToBytes(s string) []byte
	StructToBytes(i interface{}) (jsonBytes []byte, err error)
	BytesToStruct(b []byte, d interface{}) error
	ModifyInputJson(s string) (map[string]interface{}, error)
        GetLogger() logger.Logger
	DisplayAllJsonHandlers(str string)
}

We need a GO struct, which implements all of the above methods. Let’s define a singleton object,

var handler Handler

type JsonHandler struct {
	logger logger.Logger
}

func NewJsonHandler() Handler {
	if handler == nil {
		handler = &JsonHandler{
			logger: logger.GetLogger(),
		}
	}
	return handler
}

Declare a nested-structured JSON string, which we will use in various function calls,

var empStr = `{
    "id": "The ID",
    "name": "The User",
    "designation": "CEO",
    "address":
    [
        {
            "doorNumber": 1,
            "street": "The office street 1",
            "city": "The office city 1",
            "country": "The office country 1"
        },
        {
            "doorNumber": 2,
            "street": "The home street 2",
            "city": "The home city 2",
            "country": "The home country 2"
        }
    ]
}`

Now we define a data model to marshal/unmarshal JSON.

type Employee struct {
 ID        string    `json:"id,omitempty"`
 Name      string    `json:"name,omitempty"`
 Addresses []Address `json:"address,omitempty"`
}

type Address struct {
 City    string `json:"city,omitempty"`
 Country string `json:"country,omitempty"`
}

type EmployeeShort struct {
 ID   string `json:"id,omitempty"`
 Name string `json:"name,omitempty"`
}

We are all set to take a look at the core part of this article. We will read each utility function, its use and a running demo link.

1) StringToStruct

Running demo: https://go.dev/play/p/Sx0IxiLNQCt
As the name suggests, StringToStruct() basically converts an input JSON string to any provided GO struct. Second parameter to this function is a reference to a generic interface.

func (jh JsonHandler) StringToStruct(s string, i interface{}) error {
	err := json.Unmarshal([]byte(s), i)
	if err != nil {
		return err
	}

	return nil
}

The reason why this function accepts a generic interface type is, sometimes we want to use the same piece of code to unmarshal JSON string in more than one data structure.

Test:

Test StringToStruct() function, which converts JSON string into struct object, i.e. Employee,

func Test_StringToStruct_Success(t *testing.T) {
	assertThat := assert.New(t)

	jh := NewJsonHandler()

	var emp Employee
	err := jh.StringToStruct(empStr, &emp)
	assertThat.Nil(err)

	assertThat.Equal(emp.ID, "The ID")
	assertThat.Equal(emp.Name, "The User")
}

2) StructToString

Running demo: https://go.dev/play/p/WBmGLjUy0sc
StructToString() is exactly opposite to StringToStruct(), it marshalls a GO struct into a JSON string,

func (jh JsonHandler) StructToString(i interface{}) (string, error) {
	jsonBytes, err := json.Marshal(i)
	if err != nil {
		return "", err
	}

	return string(jsonBytes), nil
}
Test:
func Test_StructToString_Success(t *testing.T) {
	assertThat := assert.New(t)
	employee := &Employee{
		ID:   "The ID",
		Name: "The User",
	}

	jh := NewJsonHandler()
	str, err := jh.StructToString(employee)
	assertThat.Nil(err)

	expectedRes := `{"id":"The ID","name":"The User"}`
	assertThat.Equal(expectedRes, str)
}

3) StringToMap

Running demo: https://go.dev/play/p/jmRzDCvnjFn

func (jh JsonHandler) StringToMap(s string) (map[string]interface{}, error) {
	var m map[string]interface{}
	err := json.Unmarshal([]byte(s), &m)
	if err != nil {
		return nil, err
	}

	return m, nil
}
Test:
func Test_StringToMap_Success(t *testing.T) {
	assertThat := assert.New(t)

	jh := NewJsonHandler()
	jMap, _ := jh.StringToMap(empStr)

	id := jMap["id"].(string)
	user := jMap["name"].(string)

	assertThat.Equal(id, "The ID")
	assertThat.Equal(user, "The User")
}

4) MapToString

Running demo: https://go.dev/play/p/3AQnBrhPGJZ

func (jh JsonHandler) MapToString(m map[string]interface{}) (string, error) {
	jsonBytes, err := json.Marshal(m)
	if err != nil {
		return "", err
	}

	return string(jsonBytes), nil
}
Test:
func Test_MapToString_Success(t *testing.T) {
	assertThat := assert.New(t)

	expectedRes := "{\"id\":\"The ID\",\"name\":\"The User\"}"

	mapData := map[string]interface{}{
		"id":   "The ID",
		"name": "The User",
	}

	jh := NewJsonHandler()
	jsonStr, err := jh.MapToString(mapData)

	assertThat.Nil(err)
	assertThat.Equal(jsonStr, expectedRes)
}

5) BytesToString

Running demo: https://go.dev/play/p/3T6oCnkMsMw

func (jh JsonHandler) BytesToString(jsonBytes []byte) string {
	return string(jsonBytes)
}
Test:
func Test_BytesToString_Success(t *testing.T) {
	assertThat := assert.New(t)
	jh := NewJsonHandler()

	inputBytes := []byte(`{"id": "The ID", "name": "The User"}`)
	outputString := jh.BytesToString(inputBytes)

	actualBytes := []byte(outputString)

	assertThat.Equal(inputBytes, actualBytes)
}

6) StringToBytes

Running demo: https://go.dev/play/p/4l0yp6JXWrq

func (jh JsonHandler) StringToBytes(s string) []byte {
	return []byte(s)
}
Test:
func Test_StringToBytes_Success(t *testing.T) {
	jh := NewJsonHandler()
	jh.StringToBytes(empStr)
	assert.NotNil(t, empStr)
}

7) StructToBytes

Running demo: https://go.dev/play/p/OnNW312kEMH

func (jh JsonHandler) StructToBytes(i interface{}) (jsonBytes []byte, err error) {
	jsonBytes, err = json.Marshal(i)
	if err != nil {
		return nil, err
	}

	return jsonBytes, nil
}
Test:
func Test_StructToBytes_Success(t *testing.T) {
	assertThat := assert.New(t)
	jh := NewJsonHandler()

	employee := &Employee{
		ID:   "The ID",
		Name: "The User",
	}
	actualBytes, err := jh.StructToBytes(employee)

	assertThat.Nil(err)
	assertThat.NotNil(actualBytes)
}

8) BytesToStruct

func (jh JsonHandler) BytesToStruct(b []byte, d interface{}) error {
	err := json.Unmarshal([]byte(b), &d)
	if err != nil {
		return err
	}

	return nil
}
Test:

In this test, we read a file to get the byte data,

func Test_BytesToStruct_Success(t *testing.T) {
	assertThat := assert.New(t)
	jh := NewJsonHandler()

	byteValue, err := ioutil.ReadFile("testdata/employee.json")
	assertThat.Nil(err)

	var emp *Employee
	err = jh.BytesToStruct(byteValue, &emp)

	assertThat.Nil(err)
	assertThat.Equal(emp.ID, "The ID")
}

9) ModifyInputJson

Following function accepts a json string, converts into a map and then we change some fields of that map.

func (jh JsonHandler) ModifyInputJson(s string) (map[string]interface{}, error) {
	var mapToProcess = make(map[string]interface{})
	if err := json.Unmarshal([]byte(s), &mapToProcess); err != nil {
		return nil, errors.New("cannot convert string to map")
	}

	jh.logger.PrintKeyValue("Before modification", "mapToProcess", mapToProcess)
	mapToProcess["degree"] = "phd"
	jh.logger.PrintKeyValue("After adding a new key-value", "mapToProcess", mapToProcess)

	return mapToProcess, nil
}
Test:
func Test_ModifyInputJson_Success(t *testing.T) {
	assertThat := assert.New(t)
	jh := NewJsonHandler()

	modifiedEmpMap, err := jh.ModifyInputJson(empStr)

	assertThat.Nil(err)
	assert.NotNil(t, modifiedEmpMap)
	assertThat.Equal(modifiedEmpMap["degree"], "phd")
	assertThat.Equal(modifiedEmpMap["name"], "The User")
}

How to use:

Once cloned, tidy up the project

go mod tidy
go mod vendor

vet the whole module

go vet ./...

and then run all tests in json_handler_test.go by,

go test ./...

Tests will help to understand how to call each of JSON utility functions, as described above.

References

JSON (JavaScript Object Notation) is a simple data interchange format [RFC8259].
Go language reference website: https://go.dev/
Go JSON encoding package: https://pkg.go.dev/encoding/json

As a Go programmer, mastering GO JSON String conversions is a pivotal skill. This article explains how a typical GO application manipulates a JSON (Javascript Object Notation) object in difference scenarios. It compiles the most common but very useful JSON-String conversions at one place, and presents the examples with the links to the running code. 

I assume the readers have some understanding of JSON marshaling, and Go language structures and interfaces. You can learn about how to start with GO language programming here.

Github link:

https://github.com/azam-akram/utils-go/tree/main/json_utils

Let’s jump straight to the code..

The following interface exposes all the necessary methods for handling various JSON operations. Each method name is reflecting its purpose,

type Handler interface {
	StringToStruct(s string, i interface{}) error
	StructToString(i interface{}) (string, error)
	StringToMap(s string) (map[string]interface{}, error)
	MapToString(m map[string]interface{}) (string, error)
	BytesToString(jsonBytes []byte) string
	StringToBytes(s string) []byte
	StructToBytes(i interface{}) (jsonBytes []byte, err error)
	BytesToStruct(b []byte, d interface{}) error
	ModifyInputJson(s string) (map[string]interface{}, error)
        GetLogger() logger.Logger
	DisplayAllJsonHandlers(str string)
}

We need a GO struct, which implements all of the above methods. Let’s define a singleton object,

var handler Handler

type JsonHandler struct {
	logger logger.Logger
}

func NewJsonHandler() Handler {
	if handler == nil {
		handler = &JsonHandler{
			logger: logger.GetLogger(),
		}
	}
	return handler
}

Declare a nested-structured JSON string, which we will use in various function calls,

var empStr = `{
    "id": "The ID",
    "name": "The User",
    "designation": "CEO",
    "address":
    [
        {
            "doorNumber": 1,
            "street": "The office street 1",
            "city": "The office city 1",
            "country": "The office country 1"
        },
        {
            "doorNumber": 2,
            "street": "The home street 2",
            "city": "The home city 2",
            "country": "The home country 2"
        }
    ]
}`

Now we define a data model to marshal/unmarshal JSON.

type Employee struct {
 ID        string    `json:"id,omitempty"`
 Name      string    `json:"name,omitempty"`
 Addresses []Address `json:"address,omitempty"`
}

type Address struct {
 City    string `json:"city,omitempty"`
 Country string `json:"country,omitempty"`
}

type EmployeeShort struct {
 ID   string `json:"id,omitempty"`
 Name string `json:"name,omitempty"`
}

We are all set to take a look at the core part of this article. We will read each utility function, its use and a running demo link.

1) StringToStruct

Running demo: https://go.dev/play/p/Sx0IxiLNQCt
As the name suggests, StringToStruct() basically converts an input JSON string to any provided GO struct. Second parameter to this function is a reference to a generic interface.

func (jh JsonHandler) StringToStruct(s string, i interface{}) error {
	err := json.Unmarshal([]byte(s), i)
	if err != nil {
		return err
	}

	return nil
}

The reason why this function accepts a generic interface type is, sometimes we want to use the same piece of code to unmarshal JSON string in more than one data structure.

Test:

Test StringToStruct() function, which converts JSON string into struct object, i.e. Employee,

func Test_StringToStruct_Success(t *testing.T) {
	assertThat := assert.New(t)

	jh := NewJsonHandler()

	var emp Employee
	err := jh.StringToStruct(empStr, &emp)
	assertThat.Nil(err)

	assertThat.Equal(emp.ID, "The ID")
	assertThat.Equal(emp.Name, "The User")
}

2) StructToString

Running demo: https://go.dev/play/p/WBmGLjUy0sc
StructToString() is exactly opposite to StringToStruct(), it marshalls a GO struct into a JSON string,

func (jh JsonHandler) StructToString(i interface{}) (string, error) {
	jsonBytes, err := json.Marshal(i)
	if err != nil {
		return "", err
	}

	return string(jsonBytes), nil
}
Test:
func Test_StructToString_Success(t *testing.T) {
	assertThat := assert.New(t)
	employee := &Employee{
		ID:   "The ID",
		Name: "The User",
	}

	jh := NewJsonHandler()
	str, err := jh.StructToString(employee)
	assertThat.Nil(err)

	expectedRes := `{"id":"The ID","name":"The User"}`
	assertThat.Equal(expectedRes, str)
}

3) StringToMap

Running demo: https://go.dev/play/p/jmRzDCvnjFn

func (jh JsonHandler) StringToMap(s string) (map[string]interface{}, error) {
	var m map[string]interface{}
	err := json.Unmarshal([]byte(s), &m)
	if err != nil {
		return nil, err
	}

	return m, nil
}
Test:
func Test_StringToMap_Success(t *testing.T) {
	assertThat := assert.New(t)

	jh := NewJsonHandler()
	jMap, _ := jh.StringToMap(empStr)

	id := jMap["id"].(string)
	user := jMap["name"].(string)

	assertThat.Equal(id, "The ID")
	assertThat.Equal(user, "The User")
}

4) MapToString

Running demo: https://go.dev/play/p/3AQnBrhPGJZ

func (jh JsonHandler) MapToString(m map[string]interface{}) (string, error) {
	jsonBytes, err := json.Marshal(m)
	if err != nil {
		return "", err
	}

	return string(jsonBytes), nil
}
Test:
func Test_MapToString_Success(t *testing.T) {
	assertThat := assert.New(t)

	expectedRes := "{\"id\":\"The ID\",\"name\":\"The User\"}"

	mapData := map[string]interface{}{
		"id":   "The ID",
		"name": "The User",
	}

	jh := NewJsonHandler()
	jsonStr, err := jh.MapToString(mapData)

	assertThat.Nil(err)
	assertThat.Equal(jsonStr, expectedRes)
}

5) BytesToString

Running demo: https://go.dev/play/p/3T6oCnkMsMw

func (jh JsonHandler) BytesToString(jsonBytes []byte) string {
	return string(jsonBytes)
}
Test:
func Test_BytesToString_Success(t *testing.T) {
	assertThat := assert.New(t)
	jh := NewJsonHandler()

	inputBytes := []byte(`{"id": "The ID", "name": "The User"}`)
	outputString := jh.BytesToString(inputBytes)

	actualBytes := []byte(outputString)

	assertThat.Equal(inputBytes, actualBytes)
}

6) StringToBytes

Running demo: https://go.dev/play/p/4l0yp6JXWrq

func (jh JsonHandler) StringToBytes(s string) []byte {
	return []byte(s)
}
Test:
func Test_StringToBytes_Success(t *testing.T) {
	jh := NewJsonHandler()
	jh.StringToBytes(empStr)
	assert.NotNil(t, empStr)
}

7) StructToBytes

Running demo: https://go.dev/play/p/OnNW312kEMH

func (jh JsonHandler) StructToBytes(i interface{}) (jsonBytes []byte, err error) {
	jsonBytes, err = json.Marshal(i)
	if err != nil {
		return nil, err
	}

	return jsonBytes, nil
}
Test:
func Test_StructToBytes_Success(t *testing.T) {
	assertThat := assert.New(t)
	jh := NewJsonHandler()

	employee := &Employee{
		ID:   "The ID",
		Name: "The User",
	}
	actualBytes, err := jh.StructToBytes(employee)

	assertThat.Nil(err)
	assertThat.NotNil(actualBytes)
}

8) BytesToStruct

func (jh JsonHandler) BytesToStruct(b []byte, d interface{}) error {
	err := json.Unmarshal([]byte(b), &d)
	if err != nil {
		return err
	}

	return nil
}
Test:

In this test, we read a file to get the byte data,

func Test_BytesToStruct_Success(t *testing.T) {
	assertThat := assert.New(t)
	jh := NewJsonHandler()

	byteValue, err := ioutil.ReadFile("testdata/employee.json")
	assertThat.Nil(err)

	var emp *Employee
	err = jh.BytesToStruct(byteValue, &emp)

	assertThat.Nil(err)
	assertThat.Equal(emp.ID, "The ID")
}

9) ModifyInputJson

Following function accepts a json string, converts into a map and then we change some fields of that map.

func (jh JsonHandler) ModifyInputJson(s string) (map[string]interface{}, error) {
	var mapToProcess = make(map[string]interface{})
	if err := json.Unmarshal([]byte(s), &mapToProcess); err != nil {
		return nil, errors.New("cannot convert string to map")
	}

	jh.logger.PrintKeyValue("Before modification", "mapToProcess", mapToProcess)
	mapToProcess["degree"] = "phd"
	jh.logger.PrintKeyValue("After adding a new key-value", "mapToProcess", mapToProcess)

	return mapToProcess, nil
}
Test:
func Test_ModifyInputJson_Success(t *testing.T) {
	assertThat := assert.New(t)
	jh := NewJsonHandler()

	modifiedEmpMap, err := jh.ModifyInputJson(empStr)

	assertThat.Nil(err)
	assert.NotNil(t, modifiedEmpMap)
	assertThat.Equal(modifiedEmpMap["degree"], "phd")
	assertThat.Equal(modifiedEmpMap["name"], "The User")
}

How to use:

Once cloned, tidy up the project

go mod tidy
go mod vendor

vet the whole module

go vet ./...

and then run all tests in json_handler_test.go by,

go test ./...

Tests will help to understand how to call each of JSON utility functions, as described above.

References

JSON (JavaScript Object Notation) is a simple data interchange format [RFC8259].
Go language reference website: https://go.dev/
Go JSON encoding package: https://pkg.go.dev/encoding/json

© 2024 Solution Toolkit . All rights reserved.