Vince Yuan +

Profiling memory usage of a Go app

net/http/pprof is a powerful package for profiling. This video introduces how to use it to do memory profiling. Let me write down the steps.

  1. Import the package in your go web project (it doesn't have to be a web app). Just add this line import _ "net/http/pprof" into your code. (If your app is not a web app, see the links at the end of this post.) If you are using Gin web framework, you need to use the pprof wrapper for Gin.
  2. You can view statistic in browser http://localhost:8080/debug/pprof/
  3. Download the heap profile curl -s http://localhost:8080/debug/pprof/heap > ~/Downloads/base.heap
  4. After a while, download the current heap profile curl -s http://localhost:8080/debug/pprof/heap > ~/Downloads/current.heap
  5. Compare two profiles to find the difference.

    go tool pprof --base ~/Downloads/base.heap ~/go/bin/yourapp ~/Downloads/current.heap

    You can show the number of objects by using this option -inuse_objects

    go tool pprof -inuse_objects --base ~/Downloads/base.heap ~/go/bin/yourapp ~/Downloads/current.heap

  6. Now you enter the interactive mode of pprof.

Use top to view the top entries (memory usage).

  1. (pprof) top
  2. 13356 of -19412 total ( -69%)
  3. flat flat% sum% cum cum%
  4. 10923 -56% -56% 10923 -56% runtime.deferproc.func1
  5. 1365 -7% -63% 1365 -7% runtime.malg
  6. 910 -4.7% -68% 910 -4.7% math/big.nat.make
  7. 158 -0.81% -69% 158 -0.81% crypto/tls.(*Conn).readHandshake
  8. 0 -0% -69% 0 -0% bufio.(*Reader).Peek
  9. 0 -0% -69% 0 -0% bufio.(*Reader).Read
  10. 0 -0% -69% 0 -0% bufio.(*Reader).fill
  11. 0 -0% -69% 0 -0% bufio.(*Writer).Write
  12. 0 -0% -69% 0 -0% bufio.(*Writer).flush
  13. 0 -0% -69% 0 -0% bytes.(*Buffer).ReadFrom

Use web to visualize the graph though the web browser. Actually it generates a .svg file. You need to set your browser as the default application to open .svg file first.

You can view the source code! Use list command for functions shown in top. For example, list runtime.deferproc.func1

  1. (pprof) list runtime.deferproc.func1
  2. Total: -19412
  3. ROUTINE ======================== runtime.deferproc.func1 in /usr/local/go/src/runtime/panic.go
  4. 10923 10923 (flat, cum) -56% of Total
  5. . . 67: sp := getcallersp(unsafe.Pointer(&siz))
  6. . . 68: argp := uintptr(unsafe.Pointer(&fn)) + unsafe.Sizeof(fn)
  7. . . 69: callerpc := getcallerpc(unsafe.Pointer(&siz))
  8. . . 70:
  9. . . 71: systemstack(func() {
  10. 10923 10923 72: d := newdefer(siz)
  11. . . 73: if d._panic != nil {
  12. . . 74: throw("deferproc: d.panic != nil after newdefer")
  13. . . 75: }
  14. . . 76: d.fn = fn
  15. . . 77: d.pc = callerpc

More resources:

  1. https://golang.org/pkg/net/http/pprof/
  2. https://blog.golang.org/profiling-go-programs
blog comments powered by Disqus

Blogs

Projects