The Go test runner has several quirks and pitfalls that don't match my expectations from other test runners (bad defaults)

go-test-sane-defaults.md

Problems with the Go test runner

go test has some bad defaults that have eaten up my time.

Tests suddenly throwing panic: test timed out after 10m0s

Once you've added enough tests to a package, you will magically start to run into the panic: test timed out after 10m0s error. This is because the test -timeout option has a 10m default and the timeout applies on a test package basis, not a per-test basis as you might expect. So when the combined execution time of all tests in the package exceeds the 10-minute limit, this error arises. Diagnosing this issue can be particularly frustrating because each test run takes at least 10 minutes to reproduce and provide feedback. Additionally, the error message makes it seem like it's a single test being the problem, but it's just that one new test that was the straw that broke the camels back and crossed the 10 minute threshold overall.

The timeout mechanism should ideally operate on a per-test basis. If a package-level timeout is still desired, it should not be set as a default. Or at the very least, the error message should be updated to detect when there are just many tests being run and the -timeout should be bumped vs one test that is stalling and failing everything.

$ go help testflag
...

-timeout d
    If a test binary runs longer than duration d, panic.
    If d is 0, the timeout is disabled.
    The default is 10 minutes (10m).

# ❌
go test -v ./... -run TestFoo

# ✅
go test -v -timeout 60m ./... -run TestFoo

Granted, you might say that if your tests are taking more than 10 minutes, then that is the true problem. Agreed, they are too slow but that's sometimes just the nature of end-to-end tests with multiple servers being spawned and torn down in Docker and communicating over the network. See https://github.com/matrix-org/complement for more details.

Test log output no longer streaming as it runs

As soon as you're running with more than one test package, the logs will stop streaming even if the -run regex only targets a single test in a single package.

In order to workaround the problem and get log streaming again, you can use -p 1 to limit the number of parallel binaries run which is normally split by test package.

$ go help build
...

-p n
    the number of programs, such as build commands or
    test binaries, that can be run in parallel.
    The default is GOMAXPROCS, normally the number of CPUs available.

# ❌
go test -v ./... -run TestFoo

# ✅
go test -v -p 1 ./... -run TestFoo

More context golang/go#49195

Additionally, the -p n option can easily be confused with the -parallel n which is something competely different to control how many tests within a package can be run in parallel (depending on whether they are marked as t.Parallel())

-parallel n explanation

$ go help testflag

...

-parallel n
    Allow parallel execution of test functions that call t.Parallel, and
    fuzz targets that call t.Parallel when running the seed corpus.
    The value of this flag is the maximum number of tests to run
    simultaneously.
    While fuzzing, the value of this flag is the maximum number of
    subprocesses that may call the fuzz function simultaneously, regardless of
    whether T.Parallel is called.
    By default, -parallel is set to the value of GOMAXPROCS.
    Setting -parallel to values higher than GOMAXPROCS may cause degraded
    performance due to CPU contention, especially when fuzzing.
    Note that -parallel only applies within a single test binary.
    The 'go test' command may run tests for different packages
    in parallel as well, according to the setting of the -p flag
    (see 'go help build').