Home > Uncategorized > Deriving JSON types in Go

Deriving JSON types in Go

At SmugMug, I am currently writing code in Go to support a proxy to a remote service that formats messages in JSON. A good strategy in Go is to create a type that matches the shape of the expected data. The example below is a trivial example of matching JSON of a known shape to a native type:

package main

import (
        "encoding/json"
        "fmt"
)

type MyJSONType struct {
        A int
        B string
}

func main() {
        s := "{\"A\":123,\"B\":\"hello\"}"
        var mjt MyJSONType
        um_err := json.Unmarshal([]byte(s),&mjt)                                  
        if um_err == nil {
                fmt.Printf("%v\n",mjt)                                            
        }                                                                         
}

Using a technique like this, we can use robust native typing to help us detect when the JSON doesn’t match its expected shape.

But what do we do when the JSON has a shape that varies? Typically in Go we utilize interface {} to match structures of an unknown shape, and let Go automatically create an opaque data structure. But what if we want to derive more useful native types? Consider some variations in JSON that show up in messages seen from our remote service:

{"S":"a string"}

{"N":"123456"}

{"SS":["a string","another string"]}

{"NS":["123","456"]}

Go has powerful value inspection mechanisms for helping us determine how to get these bits into native types, and we can also use some intuition – we know that when we see a key of either SS or NS, we have lists of interface {} values. Using what we know, four functions emerge to help derive native types:

// take an interface{} string and turn it into a real string
func To_S(i interface{}) (string,error) {
        i_str,ok := i.(string)
        if !ok {
                e := fmt.Sprintf("cannot convert %v to string\n",i)
                return "", errors.New(e)
        }
        return i_str,nil
}

// take an interface{} string and turn it into a real string
func To_N(i interface{}) (int64,error) {
        i_int64,ok := i.(int64)
        if !ok {
                e := fmt.Sprintf("cannot convert %v to int64\n",i)
                return 0, errors.New(e)
        }
        return i_int64,nil
}

// take an interface{} string and turn it into a list of real strings.
// also return a list of error elements if there were any
func To_SS(i interface{}) ([]string,[]string,error) {
        i_int,ok := i.([]interface {})
        if !ok {
                e := fmt.Sprintf("cannot convert %v to []interface {}\n",i)
                return nil,nil, errors.New(e)
        }
        var i_str_list []string
        var error_list []string
        for _,v := range i_int {
                i_str,ok := v.(string)
                if ok {
                        i_str_list = append(i_str_list,i_str)
                } else {
                        e_str := fmt.Sprint("%v",v)
                        error_list = append(error_list,string(e_str))
                }
        }
        if len(error_list) > 0 {
                return i_str_list,error_list,errors.New("some conversion errs")
        }
        return i_str_list,error_list,nil
}

// take an interface{} int64 and turn it into a list of real int64s.
// also return a list of error elements if there were any
func To_NS(i interface{}) ([]int64,[]string,error) {
        i_int,ok := i.([]interface {})
        if !ok {
                e := fmt.Sprintf("cannot convert %v to []interface {}\n",i)
                return nil,nil, errors.New(e)
        }
        var i_int64_list []int64
        var error_list []string
        for _,v := range i_int {
                i_int64,ok := v.(int64)
                if ok {
                        i_int64_list = append(i_int64_list,i_int64)
                } else {
                        e_str := fmt.Sprint("%v",v)
                        error_list = append(error_list,string(e_str))
                }
        }
        if len(error_list) > 0 {
                return i_int64_list,error_list,errors.New("some conversion errs")
        }
 return i_int64_list,error_list,nil
}

Now we have native types that preserve the safety of the rest of our system.

submitted by Brad Clawsie

Categories: Uncategorized
  1. April 11, 2012 at 12:21 am

    Some of the stuff here would be easier to grok if you used multiline literals (which exist in Go).

    Line 11 has a typo, I assume it shouldn’t be the comment from the first copied again ;). My worst bugs have been because of copy/pasting and forgetting to modify the or one of the critical values.

    Nice blog and nice post. Cheers.

  2. April 13, 2012 at 11:14 pm

    Thanks for spreading the Good Word about Go! To increase readability, you may want to replace

    s := “{\”A\”:123,\”B\”:\”hello\”}”

    with

    s := `{“A”:123,”B”:”hello”}`
    :-)

  3. May 16, 2012 at 11:06 am

    Have you used this going the other directIon? Serializing native types to the JSON format? If so, how did you go about doing that?

    • bradsmugmugcom
      May 16, 2012 at 11:11 am

      hi Justin. so far, I have been able to get by with Go’s default json.Marshal() for turning native types into JSON…the native types are already “sanitary” as I have already bound them to native structs. -brad

  4. May 16, 2012 at 3:20 pm

    Have you sent data to the remote service? If so, how did you go about mapping the native types to their corresponding json format ie string to {“S”: “blah”} and number to {“N”: 42}.

    • bradsmugmugcom
      May 16, 2012 at 3:32 pm

      hi Justin

      in such a case, I would create a struct type in Go that reflects what the json should be. i.e.

      type Foo struct {
      AString string
      ANum int
      }

      given

      Foo{“hello”,42}

      i can match it to a json format that needs a string and an int

      i recommend going to

      http://play.golang.org/

      and trying out some basic examples of how this works for you

      brad

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: