Zig Test

Zig Test

Code written within one or more test declarations can be used to ensure behavior meets expectations:

introducing_zig_test.zig

const std = @import("std");
test "expect addOne adds one to 41" {
    // The Standard Library contains useful functions to help create tests.
    // `expect` is a function that verifies its argument is true.
    // It will return an error if its argument is false to indicate a failure.
    // `try` is used to return an error to the test runner to notify it that the test failed.
    try std.testing.expect(addOne(41) == 42);
}
/// The function `addOne` adds one to the number given as its argument.
fn addOne(number: i32) i32 {
    return number + 1;
}

Shell

$ zig test introducing_zig_test.zig
1/1 test.expect addOne adds one to 41... OK
All 1 tests passed.

The introducing_zig_test.zig code sample tests the function addOne to ensure that it returns 42 given the input 41. From this test’s perspective, the addOne function is said to be code under test.

zig test is a tool that creates and runs a test build. By default, it builds and runs an executable program using the default test runner provided by the Zig Standard Library as its main entry point. During the build, test declarations found while resolving the given Zig source file are included for the default test runner to run and report on.

This documentation discusses the features of the default test runner as provided by the Zig Standard Library. Its source code is located in lib/test_runner.zig.

The shell output shown above displays two lines after the zig test command. These lines are printed to standard error by the default test runner:

Test [1/1] test “expect addOne adds one to 41”…

Lines like this indicate which test, out of the total number of tests, is being run. In this case, [1/1] indicates that the first test, out of a total of one test, is being run. Note that, when the test runner program’s standard error is output to the terminal, these lines are cleared when a test succeeds.

All 1 tests passed.

This line indicates the total number of tests that have passed.

Test Declarations

Test declarations contain the keyword test, followed by an optional name written as a string literal, followed by a block containing any valid Zig code that is allowed in a function.

By convention, non-named tests should only be used to make other tests run. Non-named tests cannot be filtered.

Test declarations are similar to Functions: they have a return type and a block of code. The implicit return type of test is the Error Union Type anyerror!void, and it cannot be changed. When a Zig source file is not built using the zig test tool, the test declarations are omitted from the build.

Test declarations can be written in the same file, where code under test is written, or in a separate Zig source file. Since test declarations are top-level declarations, they are order-independent and can be written before or after the code under test.

Nested Container Tests

When the zig test tool is building a test runner, only resolved test declarations are included in the build. Initially, only the given Zig source file’s top-level declarations are resolved. Unless nested containers are referenced from a top-level test declaration, nested container tests will not be resolved.

The code sample below uses the std.testing.refAllDecls(@This()) function call to reference all of the containers that are in the file including the imported Zig source file. The code sample also shows an alternative way to reference containers using the _ = C; syntax. This syntax tells the compiler to ignore the result of the expression on the right side of the assignment operator.

testdecl_container_top_level.zig

const std = @import("std");
const expect = std.testing.expect;
// Imported source file tests will run when referenced from a top-level test declaration.
// The next line alone does not cause "introducing_zig_test.zig" tests to run.
const imported_file = @import("introducing_zig_test.zig");
test {
    // To run nested container tests, either, call `refAllDecls` which will
    // reference all declarations located in the given argument.
    // `@This()` is a builtin function that returns the innermost container it is called from.
    // In this example, the innermost container is this file (implicitly a struct).
    std.testing.refAllDecls(@This());
    // or, reference each container individually from a top-level test declaration.
    // The `_ = C;` syntax is a no-op reference to the identifier `C`.
    _ = S;
    _ = U;
    _ = @import("introducing_zig_test.zig");
}
const S = struct {
    test "S demo test" {
        try expect(true);
    }
    const SE = enum {
        V,
        // This test won't run because its container (SE) is not referenced.
        test "This Test Won't Run" {
            try expect(false);
        }
    };
};
const U = union { // U is referenced by the file's top-level test declaration
    s: US,        // and US is referenced here; therefore, "U.Us demo test" will run
    const US = struct {
        test "U.US demo test" {
            // This test is a top-level test declaration for the struct.
            // The struct is nested (declared) inside of a union.
            try expect(true);
        }
    };
    test "U demo test" {
        try expect(true);
    }
};

Shell

$ zig test testdecl_container_top_level.zig
1/4 test_0... OK
2/4 test.S demo test... OK
3/4 test.U demo test... OK
4/4 test.expect addOne adds one to 41... OK
All 4 tests passed.

Test Failure

The default test runner checks for an error returned from a test. When a test returns an error, the test is considered a failure and its error return trace is output to standard error. The total number of failures will be reported after all tests have run.

test.zig

const std = @import("std");
test "expect this to fail" {
    try std.testing.expect(false);
}
test "expect this to succeed" {
    try std.testing.expect(true);
}

Shell

$ zig test test.zig
1/2 test.expect this to fail... FAIL (TestUnexpectedResult)
/home/ci/release-0.10.1/out/zig-x86_64-linux-musl-baseline/lib/zig/std/testing.zig:347:14: 0x2115b7 in expect (test)
    if (!ok) return error.TestUnexpectedResult;
             ^
docgen_tmp/test.zig:4:5: 0x2116d5 in test.expect this to fail (test)
    try std.testing.expect(false);
    ^
2/2 test.expect this to succeed... OK
1 passed; 0 skipped; 1 failed.
error: the following test command failed with exit code 1:
/home/ci/release-0.10.1/out/zig-local-cache/o/886dbde2c2a21074c6c6d3ff9b83336b/test

Skip Tests

One way to skip tests is to filter them out by using the zig test command line parameter --test-filter [text]. This makes the test build only include tests whose name contains the supplied filter text. Note that non-named tests are run even when using the --test-filter [text] command line parameter.

To programmatically skip a test, make a test return the error error.SkipZigTest and the default test runner will consider the test as being skipped. The total number of skipped tests will be reported after all tests have run.

test.zig

test "this will be skipped" {
    return error.SkipZigTest;
}

Shell

$ zig test test.zig
1/1 test.this will be skipped... SKIP
0 passed; 1 skipped; 0 failed.

The default test runner skips tests containing a suspend point while the test is running using the default, blocking IO mode. (The evented IO mode is enabled using the --test-evented-io command line parameter.)

async_skip.zig

const std = @import("std");
test "async skip test" {
    var frame = async func();
    const result = await frame;
    try std.testing.expect(result == 1);
}
fn func() i32 {
    suspend {
        resume @frame();
    }
    return 1;
}

Shell

$ zig test async_skip.zig -fstage1
1/1 test "async skip test"... SKIP (async test)
0 passed; 1 skipped; 0 failed.

In the code sample above, the test would not be skipped in blocking IO mode if the nosuspend keyword was used (see Async and Await).

Report Memory Leaks

When code allocates Memory using the Zig Standard Library‘s testing allocator, std.testing.allocator, the default test runner will report any leaks that are found from using the testing allocator:

test.zig

const std = @import("std");
test "detect leak" {
    var list = std.ArrayList(u21).init(std.testing.allocator);
    // missing `defer list.deinit();`
    try list.append('☔');
    try std.testing.expect(list.items.len == 1);
}

Shell

$ zig test test.zig
1/1 test.detect leak... OK
[gpa] (err): memory address 0x7f5ea8dc6000 leaked:
/home/ci/release-0.10.1/out/zig-x86_64-linux-musl-baseline/lib/zig/std/array_list.zig:361:89: 0x226845 in ensureTotalCapacityPrecise (test)
                const new_memory = try self.allocator.reallocAtLeast(self.allocatedSlice(), new_capacity);
                                                                                        ^
/home/ci/release-0.10.1/out/zig-x86_64-linux-musl-baseline/lib/zig/std/array_list.zig:346:55: 0x21c46e in ensureTotalCapacity (test)
                return self.ensureTotalCapacityPrecise(better_capacity);
                                                      ^
/home/ci/release-0.10.1/out/zig-x86_64-linux-musl-baseline/lib/zig/std/array_list.zig:385:41: 0x219887 in addOne (test)
            try self.ensureTotalCapacity(newlen);
                                        ^
/home/ci/release-0.10.1/out/zig-x86_64-linux-musl-baseline/lib/zig/std/array_list.zig:170:49: 0x2136cd in append (test)
            const new_item_ptr = try self.addOne();
                                                ^
docgen_tmp/test.zig:6:20: 0x2135f2 in test.detect leak (test)
    try list.append('☔');
                   ^
/home/ci/release-0.10.1/out/zig-x86_64-linux-musl-baseline/lib/zig/test_runner.zig:63:28: 0x21a773 in main (test)
        } else test_fn.func();
                           ^
/home/ci/release-0.10.1/out/zig-x86_64-linux-musl-baseline/lib/zig/std/start.zig:604:22: 0x21402c in posixCallMainAndExit (test)
            root.main();
                     ^
/home/ci/release-0.10.1/out/zig-x86_64-linux-musl-baseline/lib/zig/std/start.zig:376:5: 0x213b31 in _start (test)
    @call(.{ .modifier = .never_inline }, posixCallMainAndExit, .{});
    ^
All 1 tests passed.
1 errors were logged.
1 tests leaked memory.
error: the following test command failed with exit code 1:
/home/ci/release-0.10.1/out/zig-local-cache/o/886dbde2c2a21074c6c6d3ff9b83336b/test

Detecting Test Build

Use the compile variable @import("builtin").is_test to detect a test build:

detect_test.zig

const std = @import("std");
const builtin = @import("builtin");
const expect = std.testing.expect;
test "builtin.is_test" {
    try expect(isATest());
}
fn isATest() bool {
    return builtin.is_test;
}

Shell

$ zig test detect_test.zig
1/1 test.builtin.is_test... OK
All 1 tests passed.

Test Output and Logging

The default test runner and the Zig Standard Library’s testing namespace output messages to standard error.

The Testing Namespace

The Zig Standard Library’s testing namespace contains useful functions to help you create tests. In addition to the expect function, this document uses a couple of more functions as exemplified here:

testing_functions.zig

const std = @import("std");
test "expectEqual demo" {
    const expected: i32 = 42;
    const actual = 42;
    // The first argument to `expectEqual` is the known, expected, result.
    // The second argument is the result of some expression.
    // The actual's type is casted to the type of expected.
    try std.testing.expectEqual(expected, actual);
}
test "expectError demo" {
    const expected_error = error.DemoError;
    const actual_error_union: anyerror!void = error.DemoError;
    // `expectError` will fail when the actual error is different than
    // the expected error.
    try std.testing.expectError(expected_error, actual_error_union);
}

Shell

$ zig test testing_functions.zig
1/2 test.expectEqual demo... OK
2/2 test.expectError demo... OK
All 2 tests passed.

The Zig Standard Library also contains functions to compare Slices, strings, and more. See the rest of the std.testing namespace in the Zig Standard Library for more available functions.

Test Tool Documentation

zig test has a few command line parameters which affect the compilation. See zig test —help for a full list.