Test coverage in Go

There are many things that make Go a great programming language. One of those things is the first-class support for testing. After installing Go, you already have everything you need to get started with testing your code:

• The built-in testing package provides a simple but powerful API to write tests.
• The go test command runs the tests and prints a summary of the test results.

That, however, only scratches the surface of what Go can do. Since Go version 1.2, the toolchain has been able to compute and display test coverage statistics. Test coverage helps you find untested parts of your codebase and, when used properly, can be an important indicator of software quality. I was pleased to see this feature being added to Go’s already excellent tooling.

Everybody interested in test coverage and Go should first read Rob Pike’s superb post, The cover story, which is a great introduction to the topic. Be sure to come back, though, as I will show you how to get more out of Go’s test coverage support.

Coverage of multiple packages

To get test coverage statistics, you first have to tell go test to generate a coverage profile (a plaintext file with collected coverage results). Such a profile can then be fed into other programs like go tool cover for further analysis/processing. Here is a quick example demonstrating how to create a beautiful HTML coverage report for a single Go package:

$ cd chef-runner/
$ go test -coverprofile=cover.out ./util
ok  github.com/mlafeldt/chef-runner/util  0.017s  coverage: 78.1% of statements
$ go tool cover -html=cover.out -o coverage.html

Unfortunately, there is a catch. As of Go 1.3.3, go test can’t compute coverage statistics for multiple packages at once. Trying to do so results in an error:

$ go test -coverprofile=cover.out ./...
cannot use test profile flag with multiple packages

This is a known issue which will probably be resolved in the foreseeable future. For chef-runner, one of my pet projects, I still wanted a way to get coverage statistics for all of its packages. I wanted to see at a glance whether I missed testing some key functionality. And I wanted it now.

Fortunately, there is a simple workaround which involves the following steps:

1. Create a coverage profile for each individual package using go list and go test.
2. Concatenate all coverage profiles into a single one.
3. Pass the final profile to go tool cover as usual.

./script/coverage

Since I didn’t want to repeat these three steps over and over again, I wrote a shell script doing it for me. I store this script as script/coverage in every Go project I need it. By default, script/coverage outputs test coverage information for each package and, after that, for each function of a project’s source code. This is what it looks like for chef-runner:

$ cd chef-runner/
$ ./script/coverage
ok  github.com/mlafeldt/chef-runner                         0.012s  coverage: 4.5% of statements
ok  github.com/mlafeldt/chef-runner/bundler                 0.013s  coverage: 100.0% of statements
ok  github.com/mlafeldt/chef-runner/chef/cookbook           0.014s  coverage: 89.7% of statements
ok  github.com/mlafeldt/chef-runner/chef/cookbook/metadata  0.012s  coverage: 90.5% of statements
ok  github.com/mlafeldt/chef-runner/chef/omnibus            0.019s  coverage: 77.3% of statements
ok  github.com/mlafeldt/chef-runner/chef/runlist            0.012s  coverage: 100.0% of statements
ok  github.com/mlafeldt/chef-runner/cli                     0.016s  coverage: 95.2% of statements
ok  github.com/mlafeldt/chef-runner/driver                  0.006s  coverage: 0.0% of statements
ok  github.com/mlafeldt/chef-runner/driver/kitchen          0.013s  coverage: 72.2% of statements
ok  github.com/mlafeldt/chef-runner/driver/ssh              0.012s  coverage: 72.7% of statements
...
github.com/mlafeldt/chef-runner/rsync/rsync.go:47:  Command        100.0%
github.com/mlafeldt/chef-runner/rsync/rsync.go:92:  Copy           0.0%
github.com/mlafeldt/chef-runner/util/util.go:15:    FileExist      100.0%
github.com/mlafeldt/chef-runner/util/util.go:22:    BaseName       100.0%
github.com/mlafeldt/chef-runner/util/util.go:31:    TempDir        100.0%
github.com/mlafeldt/chef-runner/util/util.go:36:    InDir          62.5%
github.com/mlafeldt/chef-runner/util/util.go:55:    InTestDir      80.0%
github.com/mlafeldt/chef-runner/util/util.go:65:    DownloadFile   75.0%
github.com/mlafeldt/chef-runner/version.go:15:      VersionString  100.0%
github.com/mlafeldt/chef-runner/version.go:23:      TargetString   0.0%
total:                                              (statements)   74.2%

If the option --html is given, script/coverage will additionally create a HTML report and open it in a web browser. The report contains annotated source code with different colors highlighting what code is being tested a lot (green) and what code doesn’t have any tests yet (red). Here is a sample HTML report for chef-runner. Isn’t it fantastic? It’s actually my favorite piece of the tooling.

Integration with Coveralls

Shortly after learning how to generate proper coverage statistics, I stumbled upon Coveralls – a web service that provides test coverage history and statistics for projects hosted on GitHub. Coveralls also happens to support Go. Integrating it with Travis CI, another service I’m using to test my open source projects, turned out to be straightforward. Here is a basic Travis CI configuration:

# .travis.yml
language: go

install:
  - go get -t ./...
  - go get code.google.com/p/go.tools/cmd/cover
  - go get github.com/mattn/goveralls

script:
  - PATH="$HOME/gopath/bin:$PATH"
  - script/coverage --coveralls

As you can see, I just added another option (--coveralls) to script/coverage that will cause it to push coverage statistics to Coveralls.

Last but not least, this is what Coveralls does with chef-runner’s coverage data. While I find the provided statistics interesting, I’m fully aware of the fact that they say nothing about how good the tests are. Please keep this in mind.

---

If you, fellow Gopher, find any of this useful, feel free to use my script to add test coverage to your own personal projects as well. Even if you don’t plan to use external services like Coveralls, having coverage statistics at your fingertips can still help you identify bits of untested code.