testifylint

Checks usage of github.com/stretchr/testify.

Table of Contents

• Problem statement
• Installation & usage
• Configuring
• Checkers
• Chain of warnings
• testify v2

Problem statement

Tests are also program code and the requirements for them should not differ much from the requirements for the code under tests 🙂

We should try to maintain the consistency of tests, increase their readability, reduce the chance of bugs and speed up the search for a problem.

testify is the most popular Golang testing framework in recent years. But it has a terrible ambiguous API in places, and the purpose of this linter is to protect you from annoying mistakes*.

Most checkers are stylistic, but checkers like error-is-as, require-error, expected-actual, float-compare are really helpful.

* JetBrains "The State of Go Ecosystem" reports 2021 and 2022.

Installation & usage

$ go install github.com/Antonboom/testifylint@latest
$ testifylint -h
$ testifylint ./...

Fixing

$ testifylint --fix ./...

Fixing with golangci-lint is currently unavailable due to golangci/golangci-lint#1779.

Be aware that there may be unused imports after the fix, run go fmt.

Configuring

CLI

# Enable all checkers.
$ testifylint --enable-all ./...

# Enable specific checkers only.
$ testifylint --disable-all --enable=empty,error-is-as ./...

# Disable specific checkers only.
$ testifylint --enable-all --disable=empty,error-is-as ./...

# Checkers configuration.
$ testifylint --bool-compare.ignore-custom-types ./...
$ testifylint --expected-actual.pattern=^wanted$ ./...
$ testifylint --formatter.check-format-string --formatter.require-f-funcs --formatter.require-string-msg ./...
$ testifylint --go-require.ignore-http-handlers ./...
$ testifylint --require-error.fn-pattern="^(Errorf?|NoErrorf?)$" ./...
$ testifylint --suite-extra-assert-call.mode=require ./...

golangci-lint

https://golangci-lint.run/docs/linters/configuration/#testifylint

Checkers

• ✅ – yes
• ❌ – no
• 🤏 – partially

⚠️ Also look at open for contribution checkers

blank-import

❌
import (
    "testing"

    _ "github.com/stretchr/testify"
    _ "github.com/stretchr/testify/assert"
    _ "github.com/stretchr/testify/http"
    _ "github.com/stretchr/testify/mock"
    _ "github.com/stretchr/testify/require"
    _ "github.com/stretchr/testify/suite"
)

✅
import (
    "testing"
)

Autofix: false.

Enabled by default: true.

Reason: testify doesn't do any init() magic, so these imports as _ do nothing and considered useless.

bool-compare

❌
assert.Equal(t, false, result)
assert.EqualValues(t, false, result)
assert.Exactly(t, false, result)
assert.NotEqual(t, true, result)
assert.NotEqualValues(t, true, result)
assert.False(t, !result)
assert.True(t, result == true)
// And other variations...

✅
assert.True(t, result)
assert.False(t, result)

Autofix: true.

Enabled by default: true.

Reason: Code simplification.

Also bool-compare supports user defined types like

type Bool bool

And fixes assertions via casting variable to builtin bool:

var predicate Bool
❌ assert.Equal(t, false, predicate)
✅ assert.False(t, bool(predicate))

To turn off this behavior use the --bool-compare.ignore-custom-types flag.

compares

❌
assert.True(t, a == b)
assert.True(t, a != b)
assert.True(t, a > b)
assert.True(t, a >= b)
assert.True(t, a < b)
assert.True(t, a <= b)
assert.False(t, a == b)
// And so on...

✅
assert.Equal(t, a, b)
assert.NotEqual(t, a, b)
assert.Greater(t, a, b)
assert.GreaterOrEqual(t, a, b)
assert.Less(t, a, b)
assert.LessOrEqual(t, a, b)

Autofix: true.

Enabled by default: true.

Reason: More appropriate testify API with clearer failure message.

If a and b are pointers then assert.Same/NotSame is required instead, due to the inappropriate recursive nature of assert.Equal (based on reflect.DeepEqual).

contains

❌
assert.True(t, strings.Contains(a, "abc123"))
assert.False(t, !strings.Contains(a, "abc123"))

assert.False(t, strings.Contains(a, "abc123"))
assert.True(t, !strings.Contains(a, "abc123"))

✅
assert.Contains(t, a, "abc123")
assert.NotContains(t, a, "abc123")

Autofix: true.

Enabled by default: true.

Reason: Code simplification and more appropriate testify API with clearer failure message.

empty

❌
assert.Len(t, arr, 0)
assert.Zero(t, str)
assert.Zero(t, len(arr))
assert.Equal(t, 0, len(arr))
assert.EqualValues(t, 0, len(arr))
assert.Exactly(t, 0, len(arr))
assert.LessOrEqual(t, len(arr), 0)
assert.GreaterOrEqual(t, 0, len(arr))
assert.Less(t, len(arr), 1)
assert.Greater(t, 1, len(arr))
assert.Equal(t, "", str)
assert.EqualValues(t, "", str)
assert.Exactly(t, "", str)
assert.Equal(t, ``, str)
assert.EqualValues(t, ``, str)
assert.Exactly(t, ``, str)

assert.Positive(t, len(arr))
assert.NotZero(t, str)
assert.NotZero(t, len(arr))
assert.NotEqual(t, 0, len(arr))
assert.NotEqualValues(t, 0, len(arr))
assert.Greater(t, len(arr), 0)
assert.Less(t, 0, len(arr))
assert.NotEqual(t, "", str)
assert.NotEqualValues(t, "", str)
assert.NotEqual(t, ``, str)
assert.NotEqualValues(t, ``, str)

✅
assert.Empty(t, arr)
assert.NotEmpty(t, arr)

Autofix: true.

Enabled by default: true.

Reason: More appropriate testify API with clearer failure message.

Also empty removes extra len call in *Emtpy assertions:

❌
assert.Empty(t, len(arr))
assert.NotEmpty(t, len(arr))

✅
assert.Empty(t, arr)
assert.NotEmpty(t, arr)

P.S. empty does not remove the string conversion and keeps as is:

• string(bytes) – to keep assert failure message readability;
• string(str) – in favor of unconvert.

encoded-compare

❌
assert.Equal(t, `{"foo": "bar"}`, body)
assert.EqualValues(t, `{"foo": "bar"}`, body)
assert.Exactly(t, `{"foo": "bar"}`, body)
assert.Equal(t, expectedJSON, resultJSON)
assert.Equal(t, expBodyConst, w.Body.String())
assert.Equal(t, fmt.Sprintf(`{"value":"%s"}`, hexString), result)
assert.Equal(t, "{}", json.RawMessage(resp))
assert.Equal(t, expJSON, strings.Trim(string(resultJSONBytes), "\n")) // + Replace, ReplaceAll, TrimSpace

assert.Equal(t, expectedYML, conf)

✅
assert.JSONEq(t, `{"foo": "bar"}`, body)
assert.YAMLEq(t, expectedYML, conf)

Autofix: true.

Enabled by default: true.

Reason: Protection from bugs and more appropriate testify API with clearer failure message.

encoded-compare detects JSON-style string constants (usable in fmt.Sprintf also) and JSON-style/YAML-style named variables. If variable is converted to json.RawMessage, then it is considered JSON unconditionally.

When fixing, encoded-compare removes unnecessary conversions to []byte, string, json.RawMessage and calls of strings.Replace, strings.ReplaceAll, strings.Trim, strings.TrimSpace, and adds a conversion to string when needed.

equal-values

❌
assert.EqualValues(t, 42, result.IntField)
assert.NotEqualValues(t, 42, result.IntField)
// And other variations with similar types (strings, numerics, structs, etc.)...

✅
assert.Equal(t, 42, result.IntField)
assert.NotEqual(t, 42, result.IntField)

Autofix: true.

Enabled by default: true.

Reason: Using more appropriate and more error-proof testify API.

In additional:

1. Overflow-underflow issues are covered by testify itself.
2. Nil comparisons are covered by nil-compare.

error-is-as

❌
assert.Error(t, err, errSentinel) // Typo, errSentinel hits `msgAndArgs`.
assert.NoError(t, err, errSentinel)
assert.IsType(t, err, errSentinel)
assert.IsType(t, (*http.MaxBytesError)(nil), err)
assert.IsNotType(t, err, errSentinel)
assert.IsNotType(t, store.NotFoundError{}, err)
assert.True(t, errors.Is(err, errSentinel))
assert.False(t, errors.Is(err, errSentinel))
assert.True(t, errors.As(err, &target))
assert.False(t, errors.As(err, &target))

✅
assert.ErrorIs(t, err, errSentinel)
assert.NotErrorIs(t, err, errSentinel)
assert.ErrorAs(t, err, &target)
assert.NotErrorAs(t, err, &target)

Autofix: partially.

Enabled by default: true.

Reason: In the first two cases, a common mistake that leads to hiding the incorrect wrapping of sentinel errors. In the rest cases – more appropriate testify API with clearer failure message.

Also error-is-as repeats go vet's errorsas check logic, but without autofix.

assert.ErrorAs cheatsheet:

assert.ErrorAs(t, err, new(*http.MaxBytesError))
assert.ErrorAs(t, err, store.NotFoundError{})

mbErr := new(http.MaxBytesError)
require.ErrorAs(t, err, &mbErr)
assert.Equal(t, 100, mbErr.Limit)

if mbErr := new(http.MaxBytesError); assert.ErrorAs(t, err, &mbErr) {
    assert.Equal(t, 100, mbErr.Limit)
}

error-nil

❌
assert.Nil(t, err)
assert.Empty(t, err)
assert.Zero(t, err)
assert.Equal(t, nil, err)
assert.EqualValues(t, nil, err)
assert.Exactly(t, nil, err)
assert.ErrorIs(t, err, nil)
assert.IsType(t, err, nil)

assert.NotNil(t, err)
assert.NotEmpty(t, err)
assert.NotZero(t, err)
assert.NotEqual(t, nil, err)
assert.NotEqualValues(t, nil, err)
assert.NotErrorIs(t, err, nil)
assert.IsNotType(t, err, nil)

✅
assert.NoError(t, err)
assert.Error(t, err)

Autofix: true.

Enabled by default: true.

Reason: More appropriate testify API with clearer failure message.

expected-actual

``go ❌ assert.Equal(t, result, expected) assert.Equal(t, result, len(expected)) assert.Equal(t, len(resultFields), len(expectedFields)) assert.EqualExportedValues(t, resultObj, User{Name: "Rob"}) assert.EqualValues(t, result, 42) assert.Exactly(t, result, int64(42)) assert.JSONEq(t, result,{"version": 3}`) assert.InDelta(t, result, 42.42, 1.0) assert.InDeltaMapValues(t, result, map[string]float64{"score": 0.99}, 1.0) assert.InDe