7.2 JSON

JSON (JavaScript Object Notation) is a lightweight data exchange language which is based on text description. Its advantages include being self-descriptive, easy to understand, etc. Even though it is a subset of JavaScript, JSON uses a different text format, the result being that it can be considered as an independent language. JSON bears similarity to C-family languages.

The biggest difference between JSON and XML is that XML is a complete markup language, whereas JSON is not. JSON is smaller and faster than XML, therefore it’s much easier and quicker to parse in browsers, which is one of the reasons why many open platforms choose to use JSON as their data exchange interface language.

Since JSON is becoming more and more important in web development, let’s take a look at the level of support Go has for JSON. You’ll find that Go’s standard library has very good support for encoding and decoding JSON.

Here we use JSON to represent the example in the previous section:

  1. {"servers":[{"serverName":"Shanghai_VPN","serverIP":"127.0.0.1"},{"serverName":"Beijing_VPN","serverIP":"127.0.0.2"}]}

The rest of this section will use this JSON data to introduce JSON concepts in Go.

Parse JSON

Parse to struct

Suppose we have the JSON in the above example. How can we parse this data and map it to a struct in Go? Go provides the following function for just this purpose:

  1. func Unmarshal(data []byte, v interface{}) error

We can use this function like so:

  1. package main
  3. import (
  4.     "encoding/json"
  5.     "fmt"
  6. )
  8. type Server struct {
  9.     ServerName string
  10.     ServerIP   string
  11. }
  13. type Serverslice struct {
  14.     Servers []Server
  15. }
  17. func main() {
  18.     var s Serverslice
  19.     str := `{"servers":[{"serverName":"Shanghai_VPN","serverIP":"127.0.0.1"},{"serverName":"Beijing_VPN","serverIP":"127.0.0.2"}]}`
  20.     json.Unmarshal([]byte(str), &s)
  21.     fmt.Println(s)
  22. }

In the above example, we defined a corresponding structs in Go for our JSON, using slice for an array of JSON objects and field name as our JSON keys. But how does Go know which JSON object corresponds to which specific struct filed? Suppose we have a key called Foo in JSON. How do we find its corresponding field?

  • First, Go tries to find the (capitalised) exported field whose tag contains Foo.
  • If no match can be found, look for the field whose name is Foo.
  • If there are still not matches look for something like FOO or FoO, ignoring case sensitivity.

You may have noticed that all fields that are going to be assigned should be exported, and Go only assigns fields that can be found, ignoring all others. This can be useful if you need to deal with large chunks of JSON data but you only a specific subset of it; the data you don’t need can easily be discarded.

Parse to interface

When we know what kind of JSON to expect in advance, we can parse it to a specific struct. But what if we don’t know?

We know that an interface{} can be anything in Go, so it is the best container to save our JSON of unknown format. The JSON package uses map[string]interface{} and []interface{} to save all kinds of JSON objects and arrays. Here is a list of JSON mapping relations:

  • bool represents JSON booleans,
  • float64 represents JSON numbers,
  • string represents JSON strings,
  • nil represents JSON null.

Suppose we have the following JSON data:

  1. b := []byte(`{"Name":"Wednesday","Age":6,"Parents":["Gomez","Morticia"]}`)

Now we parse this JSON to an interface{}:

  1. var f interface{}
  2. err := json.Unmarshal(b, &f)

The f stores a map, where keys are strings and values are interface{}’s’.

  1. f = map[string]interface{}{
  2.     "Name": "Wednesday",
  3.     "Age":  6,
  4.     "Parents": []interface{}{
  5.         "Gomez",
  6.         "Morticia",
  7.     },
  8. }

So, how do we access this data? Type assertion.

  1. m := f.(map[string]interface{})

After asserted, you can use the following code to access data:

  1. for k, v := range m {
  2.     switch vv := v.(type) {
  3.     case string:
  4.         fmt.Println(k, "is string", vv)
  5.     case int:
  6.         fmt.Println(k, "is int", vv)
  7.     case float64:
  8.         fmt.Println(k,"is float64",vv)
  9.     case []interface{}:
  10.         fmt.Println(k, "is an array:")
  11.         for i, u := range vv {
  12.             fmt.Println(i, u)
  13.         }
  14.     default:
  15.         fmt.Println(k, "is of a type I don't know how to handle")
  16.     }
  17. }

As you can see, we can now parse JSON of an unknown format through interface{} and type assertion.

The above example is the official solution, but type asserting is not always convenient. So, I recommend an open source project called simplejson, created and maintained by bitly. Here is an example of how to use this project to deal with JSON of an unknown format:

  1. js, err := NewJson([]byte(`{
  2.     "test": {
  3.         "array": [1, "2", 3],
  4.         "int": 10,
  5.         "float": 5.150,
  6.         "bignum": 9223372036854775807,
  7.         "string": "simplejson",
  8.         "bool": true
  9.     }
  10. }`))
  12. arr, _ := js.Get("test").Get("array").Array()
  13. i, _ := js.Get("test").Get("int").Int()
  14. ms := js.Get("test").Get("string").MustString()

It’s not hard to see how convenient this is. Check out the repository to see more information: https://github.com/bitly/go-simplejson.

Producing JSON

In many situations, we need to produce JSON data and respond to clients. In Go, the JSON package has a function called Marshal to do just that:

  1. func Marshal(v interface{}) ([]byte, error)

Suppose we need to produce a server information list. We have following sample:

  1. package main
  3. import (
  4.     "encoding/json"
  5.     "fmt"
  6. )
  8. type Server struct {
  9.     ServerName string
  10.     ServerIP   string
  11. }
  13. type Serverslice struct {
  14.     Servers []Server
  15. }
  17. func main() {
  18.     var s Serverslice
  19.     s.Servers = append(s.Servers, Server{ServerName: "Shanghai_VPN", ServerIP: "127.0.0.1"})
  20.     s.Servers = append(s.Servers, Server{ServerName: "Beijing_VPN", ServerIP: "127.0.0.2"})
  21.     b, err := json.Marshal(s)
  22.     if err != nil {
  23.         fmt.Println("json err:", err)
  24.     }
  25.     fmt.Println(string(b))
  26. }

Output:

  1. {"Servers":[{"ServerName":"Shanghai_VPN","ServerIP":"127.0.0.1"},{"ServerName":"Beijing_VPN","ServerIP":"127.0.0.2"}]}

As you know, all field names are capitalized, but if you want your JSON key names to start with a lower case letter, you should use struct tags. Otherwise, Go will not produce data for internal fields.

  1. type Server struct {
  2.     ServerName string `json:"serverName"`
  3.     ServerIP   string `json:"serverIP"`
  4. }
  6. type Serverslice struct {
  7.     Servers []Server `json:"servers"`
  8. }

After this modification, we can produce the same JSON data as before.

Here are some points you need to keep in mind when trying to produce JSON:

  • Field tags containing "-" will not be outputted.
  • If a tag contains a customized name, Go uses this instead of the field name, like serverName in the above example.
  • If a tag contains omitempty, this field will not be outputted if it is zero-value.
  • If the field type is bool, string, int, int64, etc, and its tag contains ",string", Go converts this field to its corresponding JSON type.

Example:

  1. type Server struct {
  2.     // ID will not be outputed.
  3.     ID int `json:"-"`
  5.     // ServerName2 will be converted to JSON type.
  6.     ServerName  string `json:"serverName"`
  7.     ServerName2 string `json:"serverName2,string"`
  9.     // If ServerIP is empty, it will not be outputted.
  10.     ServerIP   string `json:"serverIP,omitempty"`
  11. }
  13. s := Server {
  14.     ID:         3,
  15.     ServerName:  `Go "1.0" `,
  16.     ServerName2: `Go "1.0" `,
  17.     ServerIP:   ``,
  18. }
  19. b, _ := json.Marshal(s)
  20. os.Stdout.Write(b)

Output:

  1. {"serverName":"Go \"1.0\" ","serverName2":"\"Go \\\"1.0\\\" \""}

The Marshal function only returns data when it has succeeded, so here are some points we need to keep in mind:

  • JSON only supports strings as keys, so if you want to encode a map, its type has to be map[string]T, where T is the type in Go.
  • Types like channel, complex types and functions are not capable of being encoded to JSON.
  • Do not try to encode cyclic data, it leads to an infinite recursion.
  • If the field is a pointer, Go outputs the data that it points to, or else outputs null if it points to nil.

In this section, we introduced how to decode and encode JSON data in Go. We also looked at one third-party project called simplejson which is useful for parsing JSON or unknown format. These are all useful concepts for developing web applications in Go.