Go, as the machine guides

Emmanuel T Odeke

Orijtech, Inc.

Thu 2 Dec 2021

Go, as the machine guides

2

Let the machine guide you

3

About this talk

4

About myself

5

About Go

6

Go as the machine guides

// SanitizeGenesisBalances sorts addresses and coin sets.
func SanitizeGenesisBalances(balances []Balance) []Balance {
    sort.Slice(balances, func(i, j int) bool {
        addr1, _ := sdk.AccAddressFromBech32(balances[i].Address)
        addr2, _ := sdk.AccAddressFromBech32(balances[j].Address)
        return bytes.Compare(addr1.Bytes(), addr2.Bytes()) < 0
    })

    for _, balance := range balances {
        balance.Coins = balance.Coins.Sort()
    }

    return balances
}
7

How do we know?

8

How do we know?

9

Responses

10

CPU and RAM Profiling

11

Back to that seemingly innocent code...

// SanitizeGenesisBalances sorts addresses and coin sets.
func SanitizeGenesisBalances(balances []Balance) []Balance {
    sort.Slice(balances, func(i, j int) bool {
        addr1, _ := sdk.AccAddressFromBech32(balances[i].Address)
        addr2, _ := sdk.AccAddressFromBech32(balances[j].Address)
        return bytes.Compare(addr1.Bytes(), addr2.Bytes()) < 0
    })

    for _, balance := range balances {
        balance.Coins = balance.Coins.Sort()
    }

    return balances
}
12

Profile of the code in context

13

What was up with that code?

14

Remedy

15

Exhibit

16

The state of your HTTP2 servers in Go

17

Debugging HTTP servers (HTTP/2)

package main

func main() {
    ts := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        fmt.Fprintf(w, "Hello, %s", r.Proto)
    }))
    ts.EnableHTTP2 = true
    ts.StartTLS()
    defer ts.Close()

    res, err := ts.Client().Get(ts.URL)
    if err != nil {
        log.Fatal(err)
    }
    greeting, err := ioutil.ReadAll(res.Body)
    res.Body.Close()
    if err != nil { panic(err) }
        println(string(greeting))
}
18

net/http GODEBUG=http2debug=2

19

Wholistic state of your HTTP and gRPC services?

20

Observability

21

Go performance tips

22

Continuous benchmarking

23

map[string(byteslice)] to retrieve values

package main

func main() {
    m := map[string]int{"ten": 10}
    ten := []byte("ten")
    // Expensive way.
    key := string(ten)
    println(m[key])

    // Cheap way.
    println(m[string(ten)])
}
24

map clearing idiom

package main

func clearNonFast(m map[string]int) {
    keys := make([]string, 0, len(m))
    for key := range m {
        keys = append(keys, key)
    }

    // Do something with keys.
    // _ = keys
    for _, key := range keys {
        delete(m, key)
    }
}

func clearFast(m map[string]int) {
    for key := range m {
        delete(m, key)
    }
}
25

map clearing idiom benchmark results

26

Unsafe/reflect

27

Byteslice <-> string conversion

package main

import (
    "fmt"
    "reflect"
    "unsafe"
)

func unsafeByteSliceToStr(b []byte) string {
    return *(*string)(unsafe.Pointer(&b))
}

func unsafeStrToByteSlice(s string) (b []byte) {
    hdr := (*reflect.SliceHeader)(unsafe.Pointer(&b))
    hdr.Cap = len(s)
    hdr.Len = len(s)
    hdr.Data = (*reflect.StringHeader)(unsafe.Pointer(&s)).Data
    return b
}

func main() {
    fmt.Printf("%q\n", unsafeByteSliceToStr([]byte("string")))
    fmt.Printf("% x\n", unsafeStrToByteSlice("string"))
}
28

Unsafe+Reflect results in americanexpress/simplemli

Available at americanexpress/simplemli/pull/4

29

Unintended byteslice->string conversion penalty with fmt.*printf

30

Letting fmt.*printf "%s" handle its conversions

31

Letting fmt.*printf "%s" handle its conversions

32

Scalable and more secure software development guided by the machine?

33

Static analyzers

34

Internal states of the Go runtime?

35

runtime/trace

36

Findings

37

Scheduler and Garbage collector states

38

Examining the generated assembly

39

Go Assembly examination

package main

func main() {
    println("An honor to be at American Express.. Don't live life without it!")
}
40

Hello world Assembly

41

Toolbox assembly

42

Summing it all up

43

Reference materials

44

Thank you

Emmanuel T Odeke

Orijtech, Inc.

Thu 2 Dec 2021

emmanuel@orijtech.com

https://go.orijtech.com

@odeke_et