Source Edit

Author:Zahary Karadjov

This module implements boilerplate to make unit testing easy.

The test status and name is printed after any output or traceback.

Tests can be nested, however failure of a nested test will not mark the parent test as failed. Setup and teardown are inherited. Setup can be overridden locally.

Compiled test files as well as nim c -r <testfile.nim> exit with 0 for success (no failed tests) or 1 for failure.

Testament

Instead of unittest, please consider using the Testament tool which offers process isolation for your tests.

Alternatively using when isMainModule: doAssert conditionHere is usually a much simpler solution for testing purposes.

Running a single test

Specify the test name as a command line argument.

  1. nim c -r test "my test name" "another test"

Multiple arguments can be used.

Running a single test suite

Specify the suite name delimited by “::”.

  1. nim c -r test "my test name::"

Selecting tests by pattern

A single “*“ can be used for globbing.

Delimit the end of a suite name with “::”.

Tests matching any of the arguments are executed.

  1. nim c -r test fast_suite::mytest1 fast_suite::mytest2
  2. nim c -r test "fast_suite::mytest*"
  3. nim c -r test "auth*::" "crypto::hashing*"
  4. # Run suites starting with 'bug #' and standalone tests starting with '#'
  5. nim c -r test 'bug #*::' '::#*'

Examples

  1. suite "description for this stuff":
  2. echo "suite setup: run once before the tests"
  3. setup:
  4. echo "run before each test"
  5. teardown:
  6. echo "run after each test"
  7. test "essential truths":
  8. # give up and stop if this fails
  9. require(true)
  10. test "slightly less obvious stuff":
  11. # print a nasty message and move on, skipping
  12. # the remainder of this block
  13. check(1 != 1)
  14. check("asd"[2] == 'd')
  15. test "out of bounds error is thrown on bad access":
  16. let v = @[1, 2, 3] # you can do initialization here
  17. expect(IndexDefect):
  18. discard v[4]
  19. echo "suite teardown: run once after the tests"

Limitations/Bugs

Since check will rewrite some expressions for supporting checkpoints (namely assigns expressions to variables), some type conversions are not supported. For example check 4.0 == 2 + 2 won’t work. But doAssert 4.0 == 2 + 2 works. Make sure both sides of the operator (such as \==, >= and so on) have the same type.

Imports

since, exitprocs, macros, strutils, streams, times, sets, sequtils, os, terminal

Types

  1. ConsoleOutputFormatter = ref object of OutputFormatter
  2. ## Have test results printed in color.
  3. ## Default is `auto` depending on `isatty(stdout)`, or override it with
  4. ## `-d:nimUnittestColor:auto|on|off`.
  5. ##
  6. ## Deprecated: Setting the environment variable `NIMTEST_COLOR` to `always`
  7. ## or `never` changes the default for the non-js target to true or false respectively.
  8. ## Deprecated: the environment variable `NIMTEST_NO_COLOR`, when set, changes the
  9. ## default to true, if `NIMTEST_COLOR` is undefined.
  10. ## Set the verbosity of test results.
  11. ## Default is `PRINT_ALL`, or override with:
  12. ## `-d:nimUnittestOutputLevel:PRINT_ALL|PRINT_FAILURES|PRINT_NONE`.
  13. ##
  14. ## Deprecated: the `NIMTEST_OUTPUT_LVL` environment variable is set for the non-js target.

Source Edit

  1. JUnitOutputFormatter = ref object of OutputFormatter

Source Edit

  1. OutputFormatter = ref object of RootObj

Source Edit

  1. OutputLevel = enum
  2. PRINT_ALL, ## Print as much as possible.
  3. PRINT_FAILURES, ## Print only the failed tests.
  4. PRINT_NONE ## Print nothing.

The output verbosity of the tests. Source Edit

  1. TestResult = object
  2. suiteName*: string ## Name of the test suite that contains this test case.
  3. ## Can be ``nil`` if the test case is not in a suite.
  4. testName*: string ## Name of the test case
  5. status*: TestStatus

Source Edit

  1. TestStatus = enum
  2. OK, FAILED, SKIPPED

The status of a test when it is done. Source Edit

Vars

  1. abortOnError {.threadvar.}: bool

Set to true in order to quit immediately on fail. Default is false, or override with -d:nimUnittestAbortOnError:on|off.

Deprecated: can also override depending on whether NIMTEST_ABORT_ON_ERROR environment variable is set.

Source Edit

Procs

  1. proc addOutputFormatter(formatter: OutputFormatter) {....raises: [], tags: [],
  2. forbids: [].}

Source Edit

  1. proc checkpoint(msg: string) {....raises: [], tags: [], forbids: [].}

Set a checkpoint identified by msg. Upon test failure all checkpoints encountered so far are printed out. Example:

  1. checkpoint("Checkpoint A")
  2. check((42, "the Answer to life and everything") == (1, "a"))
  3. checkpoint("Checkpoint B")

outputs “Checkpoint A” once it fails.

Source Edit

  1. proc close(formatter: JUnitOutputFormatter) {....raises: [IOError, OSError],
  2. tags: [WriteIOEffect], forbids: [].}

Completes the report and closes the underlying stream. Source Edit

  1. proc defaultConsoleFormatter(): ConsoleOutputFormatter {....raises: [ValueError],
  2. tags: [ReadEnvEffect], forbids: [].}

Source Edit

  1. proc delOutputFormatter(formatter: OutputFormatter) {....raises: [], tags: [],
  2. forbids: [].}

Source Edit

  1. proc disableParamFiltering() {....raises: [], tags: [], forbids: [].}

disables filtering tests with the command line params Source Edit

  1. proc newConsoleOutputFormatter(outputLevel: OutputLevel = outputLevelDefault;
  2. colorOutput = true): ConsoleOutputFormatter {.
  3. ...raises: [], tags: [], forbids: [].}

Source Edit

  1. proc newJUnitOutputFormatter(stream: Stream): JUnitOutputFormatter {.
  2. ...raises: [IOError, OSError], tags: [WriteIOEffect], forbids: [].}

Creates a formatter that writes report to the specified stream in JUnit format. The stream is NOT closed automatically when the test are finished, because the formatter has no way to know when all tests are finished. You should invoke formatter.close() to finalize the report. Source Edit

  1. proc resetOutputFormatters() {....raises: [], tags: [], forbids: [].}

Source Edit

Methods

  1. method failureOccurred(formatter: ConsoleOutputFormatter;
  2. checkpoints: seq[string]; stackTrace: string) {.
  3. ...raises: [], tags: [], forbids: [].}

Source Edit

  1. method failureOccurred(formatter: JUnitOutputFormatter;
  2. checkpoints: seq[string]; stackTrace: string) {.
  3. ...raises: [], tags: [], forbids: [].}

stackTrace is provided only if the failure occurred due to an exception. checkpoints is never nil. Source Edit

  1. method failureOccurred(formatter: OutputFormatter; checkpoints: seq[string];
  2. stackTrace: string) {.base, ...gcsafe, raises: [], tags: [],
  3. forbids: [].}

stackTrace is provided only if the failure occurred due to an exception. checkpoints is never nil. Source Edit

  1. method suiteEnded(formatter: ConsoleOutputFormatter) {....raises: [], tags: [],
  2. forbids: [].}

Source Edit

  1. method suiteEnded(formatter: JUnitOutputFormatter) {....raises: [IOError, OSError],
  2. tags: [WriteIOEffect], forbids: [].}

Source Edit

  1. method suiteEnded(formatter: OutputFormatter) {.base, ...gcsafe, raises: [],
  2. tags: [], forbids: [].}

Source Edit

  1. method suiteStarted(formatter: ConsoleOutputFormatter; suiteName: string) {.
  2. ...raises: [IOError], tags: [RootEffect, WriteIOEffect], forbids: [].}

Source Edit

  1. method suiteStarted(formatter: JUnitOutputFormatter; suiteName: string) {.
  2. ...raises: [IOError, OSError, ValueError], tags: [WriteIOEffect], forbids: [].}

Source Edit

  1. method suiteStarted(formatter: OutputFormatter; suiteName: string) {.base,
  2. ...gcsafe, raises: [], tags: [], forbids: [].}

Source Edit

  1. method testEnded(formatter: ConsoleOutputFormatter; testResult: TestResult) {.
  2. ...raises: [IOError], tags: [RootEffect, WriteIOEffect], forbids: [].}

Source Edit

  1. method testEnded(formatter: JUnitOutputFormatter; testResult: TestResult) {.
  2. ...raises: [IOError, OSError, ValueError], tags: [TimeEffect, WriteIOEffect],
  3. forbids: [].}

Source Edit

  1. method testEnded(formatter: OutputFormatter; testResult: TestResult) {.base,
  2. ...gcsafe, raises: [], tags: [], forbids: [].}

Source Edit

  1. method testStarted(formatter: ConsoleOutputFormatter; testName: string) {.
  2. ...raises: [], tags: [], forbids: [].}

Source Edit

  1. method testStarted(formatter: JUnitOutputFormatter; testName: string) {.
  2. ...raises: [], tags: [TimeEffect], forbids: [].}

Source Edit

  1. method testStarted(formatter: OutputFormatter; testName: string) {.base, ...gcsafe,
  2. raises: [], tags: [], forbids: [].}

Source Edit

Macros

  1. macro check(conditions: untyped): untyped

Verify if a statement or a list of statements is true. A helpful error message and set checkpoints are printed out on failure (if outputLevel is not PRINT_NONE).

Example:

  1. import std/strutils
  2. check("AKB48".toLowerAscii() == "akb48")
  3. let teams = {'A', 'K', 'B', '4', '8'}
  4. check:
  5. "AKB48".toLowerAscii() == "akb48"
  6. 'C' notin teams

Source Edit

  1. macro expect(exceptions: varargs[typed]; body: untyped): untyped

Test if body raises an exception found in the passed exceptions. The test passes if the raised exception is part of the acceptable exceptions. Otherwise, it fails.

Example:

  1. import std/[math, random, strutils]
  2. proc defectiveRobot() =
  3. randomize()
  4. case rand(1..4)
  5. of 1: raise newException(OSError, "CANNOT COMPUTE!")
  6. of 2: discard parseInt("Hello World!")
  7. of 3: raise newException(IOError, "I can't do that Dave.")
  8. else: assert 2 + 2 == 5
  9. expect IOError, OSError, ValueError, AssertionDefect:
  10. defectiveRobot()

Source Edit

Templates

  1. template fail()

Print out the checkpoints encountered so far and quit if abortOnError is true. Otherwise, erase the checkpoints and indicate the test has failed (change exit code and test status). This template is useful for debugging, but is otherwise mostly used internally. Example:

  1. checkpoint("Checkpoint A")
  2. complicatedProcInThread()
  3. fail()

outputs “Checkpoint A” before quitting.

Source Edit

  1. template require(conditions: untyped)

Same as check except any failed test causes the program to quit immediately. Any teardown statements are not executed and the failed test output is not generated. Source Edit

  1. template skip()

Mark the test as skipped. Should be used directly in case when it is not possible to perform test for reasons depending on outer environment, or certain application logic conditions or configurations. The test code is still executed.

  1. if not isGLContextCreated():
  2. skip()

Source Edit

  1. template suite(name, body) {.dirty.}

Declare a test suite identified by name with optional setup and/or teardown section.

A test suite is a series of one or more related tests sharing a common fixture (setup, teardown). The fixture is executed for EACH test.

  1. suite "test suite for addition":
  2. setup:
  3. let result = 4
  4. test "2 + 2 = 4":
  5. check(2+2 == result)
  6. test "(2 + -2) != 4":
  7. check(2 + -2 != result)
  8. # No teardown needed

The suite will run the individual test cases in the order in which they were listed. With default global settings the above code prints:

  1. [Suite] test suite for addition
  2. [OK] 2 + 2 = 4
  3. [OK] (2 + -2) != 4

Source Edit

  1. template test(name, body) {.dirty.}

Define a single test case identified by name.

  1. test "roses are red":
  2. let roses = "red"
  3. check(roses == "red")

The above code outputs:

  1. [OK] roses are red

Source Edit