* Documents the expectations for unit-tests * Updates the tests to the version command so as to provide a clear example of how to test commands. Change-Id: I3fbb509a2e4298b4ae68200764a6034459b61602
5.7 KiB
Testing Guidelines
This document lays out several guidelines to secure high quality and
consistency throughout airshipctl
's test bed.
Testing packages
The airshipctl
project uses the testify library, a thin wrapper around Go's
builtin testing
package. The testify
package provides the following
packages:
assert
: Functions from this package can be used to replace most calls tot.Error
require
: Contains the same functions as above, but these functions should replace calls tot.Fatal
mock
: Contains theMock
mechanism, granting the ability to mock out structs
Test coverage
Tests should cover at least 80% of the codebase. Anything less will cause
the CI gates to fail. A developer should assert that their code meets this
criteria before submitting a patchset. This check can be performed with one of
the following make
targets:
# Runs all unit tests, then computes and reports the coverage
make cover
# Same as above, but in the same dockerized container as the CI gates
make docker-image-unit-tests
Good practice is to assert that the changed packages have not decreased in coverage. The coverage check can be run for a specific package with a command such as the following.
make cover PKG=./pkg/foo
Test directory structure
Test files end in _test.go
, and sit next to the tested file. For example,
airshipctl/pkg/foo/foo.go
should be tested by
airshipctl/pkg/foo/foo_test.go
. A test's package name should also end in
_test
, unless that file intends to test unexported fields and method, at
which point it should be in the package under test.
Go will ignore any files stored in a directory called testdata
, therefore all
non-Go test files (such as expected output or example input) should be stored
there.
Any mocks for a package should be stored in a sub-package ending in mocks
.
Each mocked struct should have its own file, where the filename describes the
struct, i.e. a file containing a mocked Fooer
should be stored at
mocks/fooer.go
. Mocked files can be either handwritten or generated via
mockery. The mockery
tool can generate files in this fashion with the
following command.
mockery -all -case snake
An example file structure might look something like the following.
airshipctl/pkg/foo
├── foo.go
├── foo_test.go
├── mocks
│ └── fooer.go
└── testdata
└── example-input.yaml
Testing guidelines
This section annotates various standards for unit tests in airshipctl
. These
should be thought of as "guidelines" rather than "rules".
- Using table-tests prevents a lot of duplicated code.
- Using subtests allows tests to provide much more fine-grained results.
- Calls to methods from
testify/require
be reserved for situations in which the test should fail immediately (e.g. during test setup). Generally,testify/assert
should be preferred.
How to write unit tests for files listed under the cmd
package
Go files listed under the cmd
package should be relatively slim. Their
purpose is to be a client of the pkg
package. Most of these files will
contain no more than a single function which creates and returns a
cobra.Command
. Nonetheless, these functions need to be tested. To help
alleviate some of the difficulties that come with testing a CLI, airshipctl
provides several helper structs and functions under the testutil
package.
As an example, suppose you have the following function:
func NewVersionCommand() *cobra.Command {
versionCmd := &cobra.Command{
Use: "version",
Short: "Show the version number of airshipctl",
Run: func(cmd *cobra.Command, args []string) {
out := cmd.OutOrStdout()
clientV := version.clientVersion()
w := util.NewTabWriter(out)
defer w.Flush()
fmt.Fprintf(w, "%s:\t%s\n", "airshipctl", clientV)
},
}
return versionCmd
}
Testing this functionality is easy with the use of the pre-built
testutil.CmdTest
:
func TestVersion(t *testing.T) {
versionCmd := cmd.NewVersionCommand()
cmdTests := []*testutil.CmdTest{
{
Name: "version",
CmdLine: "",
Cmd: versionCmd,
Error: nil,
},
{
Name: "version-help",
CmdLine: "--help",
Cmd: versionCmd,
Error: nil,
},
}
for _, tt := range cmdTests {
testutil.RunTest(t, tt)
}
}
The above test uses CmdTest
structs, which are then fed to the RunTest
function. This function provides abstraction around running a command on the
command line and comparing its output to a "golden file" (the pre-determined
expected output). The following describes the fields of the CmdTest
struct.
Name
- The name for this test. This field must be unique, as it will be used while naming the golden fileCmdLine
- The arguments and flags to pass to the commandCmd
- The actual instance of acobra.Command
to run. The above example reuses the command, but more complex tests may require different instances (e.g. to pass in a differentSettings
object)Error
- The expected error for the command to return. This can be omitted if this test doesn't expect an error
Once you've written your test, you can generate the associated golden files by
running make update-golden
, which invokes the "update" mode for
testutil.RunTest
. When the command has completed, you can view the output in
the associated files in the testdata
directory next to your command. Note
that these files are easily discoverable from the output of git status
. When
you're certain that the golden files are correct, you can add them to the repo.