Go and protocol buffers

A case study of API design

19 February 2013

David Symonds

Software Engineer, Google

Overview

code.google.com/p/protobuf
code.google.com/p/goprotobuf

History

Communicating servers often have different update schedules. 

if (version == 3) {
    ...
} else if (version > 4) {
    if (version == 5) {
        ...
    }
    ...
}

Writing hand-written parsers for changing protocols sucks.

Protocol Buffers

Protocol Buffers - IDL

A foo.proto file defines messages. The protocol compiler (protoc)
parses it and generates language-specific code. 

message Phone {
    enum Type {
        MOBILE = 1;
        HOME = 2;
        WORK = 3;
    }
    optional Type type = 1 [default = HOME];
    required string number = 2;
}

message Person {
    required string name = 1;
    required int32 id = 2;
    optional string email = 3;
    repeated Phone phone = 4;
}

Go API

type Phone struct {
    Type   *Phone_Type `protobuf:"varint,1,opt,name=type,enum=Phone.Type" json:"type,omitempty"`
    Number *string     `protobuf:"bytes,2,req,name=number" json:"number,omitempty"`
}

type Person struct {
    Name  *string  `protobuf:"bytes,1,req,name=name" json:"name,omitempty"`
    Id    *int32   `protobuf:"varint,2,req,name=id" json:"id,omitempty"`
    Email *string  `protobuf:"bytes,3,opt,name=email" json:"email,omitempty"`
    Phone []*Phone `protobuf:"bytes,4,rep,name=phone" json:"phone,omitempty"`
}

Go Use

Composite literals work well. 

p := &pb.Person{
    Name:  proto.String("Andrew"),
    Id:    proto.Int32(1337),
    Phone: []*pb.Phone{
        {
            Type: pb.Phone_MOBILE.Enum(),
            Number: proto.String("..."),
        },
        {
            Number: proto.String("555-0133"),
        },
    },
}

Design: struct tags

Design: distinguishability of setness

Design: enums

type Phone_Type int32

const (
    Phone_MOBILE Phone_Type = 1
    Phone_HOME   Phone_Type = 2
    Phone_WORK   Phone_Type = 3
)

var Phone_Type_name = map[int32]string{ ... }
var Phone_Type_value = map[string]int32{ ... }

func (x Phone_Type) Enum() *Phone_Type { ... }

func (x Phone_Type) String() string { ... }

Design: getters

Default values aren't zero values. 

optional int32 foo = 1 [default = 7];

M{Foo: nil}            // foo's value is 7
M{Foo: proto.Int32(3)} // foo's value is 3

Tedious and error prone to evaluate it manually. 

x := pb.Default_M_Foo
if m.Foo != nil {
    x = *m.Foo
}

Solution: getters (compromise between ease of use and bloat) 

// generated code
func (this *M) GetFoo() int32 {
    if this != nil && this.Foo != nil {
        return *this.Foo
    }
    return Default_M_Foo
}

Flash to the past: May 2009

package mytest

import _io_ "io"

type Request struct {
  Key []int64  "PB(varint,1,rep)";
  XXX_unrecognized *_io_.ByteBuffer;
}

var Default_Request Request
// TODO(r): use a method?
func (this *Request) Reset() *Request {
  *this = Default_Request;
  return this;
}
func (this *Request) Init() *Request {
  return this;
}
func NewRequest() *Request {
  return new(Request).Init();
}

Thank you

David Symonds

Software Engineer, Google

dsymonds@golang.org