Go Serverless

From POC to Prod

27 August 2018

Steven Bogacz

Software Engineer at SendGrid

Our Goals

Let's work toward them with a toy app

2

What are we trying to build?

We want a lightweight, ephemeral storage API

3

We choose the AWS stack, for reasons

4

Time to POC!

In our case, that looks like 

Handler(*events.APIGatewayProxyRequest) (*events.APIGatewayProxyResponse, error)

AWS Lambda Handler Documentation

5

POC Handler Code

Using the events.APIGatewayProxyRequest type directly: 

    switch req.HTTPMethod {
    case "POST":
        key, err = postFile(ctx, req.Body)
    case "GET":
        key, err = extractKey(req)
        if err != nil {
            return nil, err
        }

        data, err = getFile(ctx, key)
    case "DELETE":
        key, err = extractKey(req)
        if err != nil {
            return nil, err
        }

        err = deleteFile(ctx, key)
    }
6

POC POST Code

Some decoupling: 

func postFile(ctx context.Context, data string) (string, error) {
    u, err := uuid.NewV4()
    if err != nil {
        return "", newInternalServerErr(err, "failed to generate key")
    }

Sub-functions don't know about APIGW-specific types

7

POC tests

Using a normal testing approach, we can unit test some helpers 

func TestHelpers(t *testing.T) {
    t.Run("extract key should parse the path correctly", func(t *testing.T) {
        req := &events.APIGatewayProxyRequest{
            Path: "blobs/1234",
        }
        key, err := extractKey(req)
        require.NoError(t, err)
        require.Equal(t, "1234", key)
    })
    // ...
}

This makes us use dummy input, and doesn't give us great coverage

8

Fully testing the POC

We use Terraform (a cloud-agnostic Infrastructure-as-Code tool)

Initial deploy 

terraform apply phase1.plan

33.54s

Subsequent plan and apply 

terraform plan --out=phase1.plan

12.54

terraform apply phase1.plan

19.12

Terraform

9

One approach to lower the dev cycle time

Can take a look at:

Localstack

Supports several locally running versions of AWS Services
- API Gateway
- DynamoDB
- Kinesis
- S3
- etc.

10

One approach to lower the dev cycle time

Configuring our deployment isn't exactly trivial.

Could use terraform, but there's a current issue open to get the AWS fakes to work:

Terraform Provider Open Issue

Can have set up scripts to run before we run our tests, e.g. make test spins up, go test s, spins down.

Downsides:

11

A better approach

Ask how would I do this as idiomatically as possible?

12

A better approach - Our original layout

  tree  
  .
  ├── config.go
  ├── errors.go
  ├── errors_test.go
  ├── main.go
  ├── main_test.go
  ├── toy
  └── toy.zip

  0 directories, 7 files
13

A better approach - Our new layout

  tree  
  .
  ├── cmd 
  │   ├── http
  │   │   ├── main.go
  │   │   └── toy
  │   └── lambda
  │       └── main.go
  ├── internal
  │   ├── httperrs
  │   │   ├── errors.go
  │   │   └── errors_test.go
  │   ├── s3store
  │   │   └── s3_store.go
  │   └── toy
  │       ├── config.go
  │       ├── local_store.go
  │       ├── server.go
  │       ├── server_test.go
  │       └── store.go
  ├── toy
  └── toy.zip

  7 directories, 13 files
14

A better approach - Our new server

Now we have a server struct 

// Server represents all of the config and clients
// needed to run our app
type Server struct {
    store Store
    cfg   *Config

    router *chi.Mux
}

Where Store is 

// Store provides an interface to the blobs we
// store in the toy app
type Store interface {
    Get(context.Context, string) (string, error)
    Set(context.Context, string, string) error
    Del(context.Context, string) error
}
15

A better approach - Server Start

Starting the server looks familiar 

// Start starts the server
func (s *Server) Start() {
    s.router.Route("/blobs", func(r chi.Router) {
        r.Post("/", s.storeBlob)
        r.Route("/{key}", func(r chi.Router) {
            r.Get("/", s.getBlob)
            r.Delete("/", s.deleteBlob)
        })
    })
    if err := h.ListenAndServe(); err != nil {
        if err != http.ErrServerClosed {
            log.Fatal(err)
        }
    }
}
16

A better approach - Two Store implementations

Local 

// LocalStore is an in-memory implementation of our
// store interface
type LocalStore struct {
    store map[string]string
    lock  *sync.RWMutex
}

S3 

// S3Store is an S3 backed implementation of our Store
// interaface
type S3Store struct {
    client *s3.S3
    bucket string
}
17

A better approach - Now with HTTP tests!

With a little setup of our server 

func TestMain(m *testing.M) {
    if err := setupServer(); err != nil {
        log.WithError(err).Fatal("failed to set up server for tests")
    }

    go s.Start()
    time.Sleep(500 * time.Millisecond)
    status := m.Run()
    s.Stop()
    os.Exit(status)
}
18

A better approach - Now with HTTP tests!

Now we can write more thorough tests in a familiar way 

    testBlob := "this is a test blob"
    var key string
    t.Run("create a blob", func(t *testing.T) {
        resp, err := http.Post(addr, "application/text", strings.NewReader(testBlob))
        require.NoError(t, err)
        require.Equal(t, http.StatusOK, resp.StatusCode)
        require.NotNil(t, resp)
        require.NotNil(t, resp.Body)

        defer resp.Body.Close()
        b, err := ioutil.ReadAll(resp.Body)
        require.NoError(t, err)

        key = string(b)
    })
19

A better approach - Lambda

It's great that our code and tests look a little more stdlib-ish, but how does that translate to our actual deployment, i.e. Lambda???

We can translate the events.APIGatewayProxyRequest, either by hand, or with:

AWS Lambda Go API Proxy

You may have noticed two packages under cmd, lambda and http

20

A better approach - Lambda

As long as we expose our Router 

// Router exposes our chi Route externally
func (s *Server) Router() *chi.Mux {
    return s.router
}

we can change our lambda/main.go to set up the proxy adapter 

    chiLambda = chiadapter.New(s.Router())

and call it in the Handler 

// Handler satisfies the AWS Lambda Go interface
func Handler(req events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {

    // If no name is provided in the HTTP request body, throw an error
    return chiLambda.Proxy(req)
}
21

A better approach - Running locally 

./toy --local-store

Can curl locally, and get faster feedback 

curl -XPOST -d @key.json http://127.0.0.1:8080/blobs
43823ea6-6f48-49c2-a798-47eb728e50a7

0.02s

curl http://127.0.0.1:8080/blobs/43823ea6-6f48-49c2-a798-47eb728e50a7
{    "Name": { "S": "banana" }}

0.01s

curl -XDELETE -v http://127.0.0.1:8080/blobs/43823ea6-6f48-49c2-a798-47eb728e50a7
*   Trying 127.0.0.1...
...
< HTTP/1.1 204 No Content

0.02s
22

Can we do even better?

What have we gained?

We're flexible about what we're deployed on now, but what about our backend?

23

Can we do even better? go-cloud

go-cloud project

The go-cloud project aims to provide common interfaces to common services across cloud providers.
BEAR TRAP ALERT This means that they can only support common operations, wouldn't be helpful for more deployment-specific control BEAR TRAP ALERT

Bucket methods cover the functionality we had before 

func NewBucket(b driver.Bucket) *Bucket
func (b *Bucket) Delete(ctx context.Context, key string) error
func (b *Bucket) NewReader(ctx context.Context, key string) (*Reader, error)
func (b *Bucket) NewWriter(ctx context.Context, key string, opt *WriterOptions) (*Writer, error)

We can even delete code!

24

Can we do even better? go-cloud

diff --brief -r third fourth   
Only in third/internal: s3store
Only in third/internal/toy: local_store.go
Files third/internal/toy/server.go and fourth/internal/toy/server.go differ
Only in third/internal/toy: store.go
  
  tree -I vendor
  ├── cmd
  │   ├── http
  │   │   ├── main.go
  │   │   └── toy
  │   └── lambda
  │       └── main.go
  ├── internal 
  │   ├── httperrs
  │   │   ├── errors.go
  │   │   └── errors_test.go
  │   └── toy
  │       ├── config.go
  │       ├── server.go
  │       └── server_test.go
25

Can we do even better? go-cloud

We can use the fileblob package for local tests and running locally 

func setupServer(dir string) error {
    port, err := freeport.GetFreePort()
    if err != nil {
        return errors.Wrap(err, "failed to get free port")
    }

    c := &Config{
        BucketName: "test-bucket",
        Port:       port,
    }
    addr = fmt.Sprintf("http://127.0.0.1:%d/blobs", port)
    log.Infof("starting server at %s", addr)

    store, err := fileblob.NewBucket(dir)
    if err != nil {
        return errors.Wrap(err, "failed to set up local store")
    }
    s = New(c, store)
    return nil
}
26

Can we do even better? go-cloud

Our locally running main can now support both backends... and it wouldn't be much work to support a third (e.g. Google Cloud Storage) 

    if localStore {
        store, cleanup, err = getLocalStore()
    } else {
        store, cleanup, err = getS3Store()
    }
    if err != nil {
        return errors.Wrap(err, "failed to initialize store")
    }
    defer cleanup()
    s = toy.New(config, store)
    go s.Start()
27

Recap

28

Questions?

Talk material in GitHub

29

Thank you

Steven Bogacz

Software Engineer at SendGrid

steven@bogacz.io

https://bogacz.io

@bogaczio

Use the left and right arrow keys or click the left and right edges of the page to navigate between slides.
(Press 'H' or navigate to hide this message.)