单元测试
测试 Julia Base 库
Julia is under rapid development and has an extensive test suite to verify functionality across multiple platforms. If you build Julia from source, you can run this test suite with make test
. In a binary install, you can run the test suite using Base.runtests()
.
Base.runtests
— Function
Base.runtests(tests=["all"]; ncores=ceil(Int, Sys.CPU_THREADS / 2),
exit_on_error=false, [seed])
Run the Julia unit tests listed in tests
, which can be either a string or an array of strings, using ncores
processors. If exit_on_error
is false
, when one test fails, all remaining tests in other files will still be run; they are otherwise discarded, when exit_on_error == true
. If a seed is provided via the keyword argument, it is used to seed the global RNG in the context where the tests are run; otherwise the seed is chosen randomly.
基本的单元测试
The Test
module provides simple unit testing functionality. Unit testing is a way to see if your code is correct by checking that the results are what you expect. It can be helpful to ensure your code still works after you make changes, and can be used when developing as a way of specifying the behaviors your code should have when complete.
Simple unit testing can be performed with the @test
and @test_throws
macros:
Test.@test
— Macro
@test ex
@test f(args...) key=val ...
Tests that the expression ex
evaluates to true
. Returns a Pass
Result
if it does, a Fail
Result
if it is false
, and an Error
Result
if it could not be evaluated.
Examples
julia> @test true
Test Passed
julia> @test [1, 2] + [2, 1] == [3, 3]
Test Passed
The @test f(args...) key=val...
form is equivalent to writing @test f(args..., key=val...)
which can be useful when the expression is a call using infix syntax such as approximate comparisons:
julia> @test π ≈ 3.14 atol=0.01
Test Passed
This is equivalent to the uglier test @test ≈(π, 3.14, atol=0.01)
. It is an error to supply more than one expression unless the first is a call expression and the rest are assignments (k=v
).
Test.@test_throws
— Macro
@test_throws exception expr
Tests that the expression expr
throws exception
. The exception may specify either a type, or a value (which will be tested for equality by comparing fields). Note that @test_throws
does not support a trailing keyword form.
Examples
julia> @test_throws BoundsError [1, 2, 3][4]
Test Passed
Thrown: BoundsError
julia> @test_throws DimensionMismatch [1, 2, 3] + [1, 2]
Test Passed
Thrown: DimensionMismatch
For example, suppose we want to check our new function foo(x)
works as expected:
julia> using Test
julia> foo(x) = length(x)^2
foo (generic function with 1 method)
如果条件为真,则返回 Pass
:
julia> @test foo("bar") == 9
Test Passed
julia> @test foo("fizz") >= 10
Test Passed
如果条件为假,则返回 Fail
并抛出异常。
julia> @test foo("f") == 20
Test Failed at none:1
Expression: foo("f") == 20
Evaluated: 1 == 20
ERROR: There was an error during testing
If the condition could not be evaluated because an exception was thrown, which occurs in this case because length
is not defined for symbols, an Error
object is returned and an exception is thrown:
julia> @test foo(:cat) == 1
Error During Test
Test threw an exception of type MethodError
Expression: foo(:cat) == 1
MethodError: no method matching length(::Symbol)
Closest candidates are:
length(::SimpleVector) at essentials.jl:256
length(::Base.MethodList) at reflection.jl:521
length(::MethodTable) at reflection.jl:597
...
Stacktrace:
[...]
ERROR: There was an error during testing
If we expect that evaluating an expression should throw an exception, then we can use @test_throws
to check that this occurs:
julia> @test_throws MethodError foo(:cat)
Test Passed
Thrown: MethodError
Working with Test Sets
Typically a large number of tests are used to make sure functions work correctly over a range of inputs. In the event a test fails, the default behavior is to throw an exception immediately. However, it is normally preferable to run the rest of the tests first to get a better picture of how many errors there are in the code being tested.
The @testset
macro can be used to group tests into sets. All the tests in a test set will be run, and at the end of the test set a summary will be printed. If any of the tests failed, or could not be evaluated due to an error, the test set will then throw a TestSetException
.
Test.@testset
— Macro
@testset [CustomTestSet] [option=val ...] ["description"] begin ... end
@testset [CustomTestSet] [option=val ...] ["description $v"] for v in (...) ... end
@testset [CustomTestSet] [option=val ...] ["description $v, $w"] for v in (...), w in (...) ... end
Starts a new test set, or multiple test sets if a for
loop is provided.
If no custom testset type is given it defaults to creating a DefaultTestSet
. DefaultTestSet
records all the results and, if there are any Fail
s or Error
s, throws an exception at the end of the top-level (non-nested) test set, along with a summary of the test results.
Any custom testset type (subtype of AbstractTestSet
) can be given and it will also be used for any nested @testset
invocations. The given options are only applied to the test set where they are given. The default test set type does not take any options.
The description string accepts interpolation from the loop indices. If no description is provided, one is constructed based on the variables.
By default the @testset
macro will return the testset object itself, though this behavior can be customized in other testset types. If a for
loop is used then the macro collects and returns a list of the return values of the finish
method, which by default will return a list of the testset objects used in each iteration.
Before the execution of the body of a @testset
, there is an implicit call to Random.seed!(seed)
where seed
is the current seed of the global RNG. Moreover, after the execution of the body, the state of the global RNG is restored to what it was before the @testset
. This is meant to ease reproducibility in case of failure, and to allow seamless re-arrangements of @testset
s regardless of their side-effect on the global RNG state.
Examples
julia> @testset "trigonometric identities" begin
θ = 2/3*π
@test sin(-θ) ≈ -sin(θ)
@test cos(-θ) ≈ cos(θ)
@test sin(2θ) ≈ 2*sin(θ)*cos(θ)
@test cos(2θ) ≈ cos(θ)^2 - sin(θ)^2
end;
Test Summary: | Pass Total
trigonometric identities | 4 4
We can put our tests for the foo(x)
function in a test set:
julia> @testset "Foo Tests" begin
@test foo("a") == 1
@test foo("ab") == 4
@test foo("abc") == 9
end;
Test Summary: | Pass Total
Foo Tests | 3 3
测试集可以嵌套:
julia> @testset "Foo Tests" begin
@testset "Animals" begin
@test foo("cat") == 9
@test foo("dog") == foo("cat")
end
@testset "Arrays $i" for i in 1:3
@test foo(zeros(i)) == i^2
@test foo(fill(1.0, i)) == i^2
end
end;
Test Summary: | Pass Total
Foo Tests | 8 8
In the event that a nested test set has no failures, as happened here, it will be hidden in the summary. If we do have a test failure, only the details for the failed test sets will be shown:
julia> @testset "Foo Tests" begin
@testset "Animals" begin
@testset "Felines" begin
@test foo("cat") == 9
end
@testset "Canines" begin
@test foo("dog") == 9
end
end
@testset "Arrays" begin
@test foo(zeros(2)) == 4
@test foo(fill(1.0, 4)) == 15
end
end
Arrays: Test Failed
Expression: foo(fill(1.0, 4)) == 15
Evaluated: 16 == 15
[...]
Test Summary: | Pass Fail Total
Foo Tests | 3 1 4
Animals | 2 2
Arrays | 1 1 2
ERROR: Some tests did not pass: 3 passed, 1 failed, 0 errored, 0 broken.
其他测试宏
As calculations on floating-point values can be imprecise, you can perform approximate equality checks using either @test a ≈ b
(where ≈
, typed via tab completion of \approx
, is the isapprox
function) or use isapprox
directly.
julia> @test 1 ≈ 0.999999999
Test Passed
julia> @test 1 ≈ 0.999999
Test Failed at none:1
Expression: 1 ≈ 0.999999
Evaluated: 1 ≈ 0.999999
ERROR: There was an error during testing
Test.@inferred
— Macro
@inferred [AllowedType] f(x)
Tests that the call expression f(x)
returns a value of the same type inferred by the compiler. It is useful to check for type stability.
f(x)
can be any call expression. Returns the result of f(x)
if the types match, and an Error
Result
if it finds different types.
Optionally, AllowedType
relaxes the test, by making it pass when either the type of f(x)
matches the inferred type modulo AllowedType
, or when the return type is a subtype of AllowedType
. This is useful when testing type stability of functions returning a small union such as Union{Nothing, T}
or Union{Missing, T}
.
julia> f(a) = a > 1 ? 1 : 1.0
f (generic function with 1 method)
julia> typeof(f(2))
Int64
julia> @code_warntype f(2)
Variables
#self#::Core.Compiler.Const(f, false)
a::Int64
Body::UNION{FLOAT64, INT64}
1 ─ %1 = (a > 1)::Bool
└── goto #3 if not %1
2 ─ return 1
3 ─ return 1.0
julia> @inferred f(2)
ERROR: return type Int64 does not match inferred return type Union{Float64, Int64}
[...]
julia> @inferred max(1, 2)
2
julia> g(a) = a < 10 ? missing : 1.0
g (generic function with 1 method)
julia> @inferred g(20)
ERROR: return type Float64 does not match inferred return type Union{Missing, Float64}
[...]
julia> @inferred Missing g(20)
1.0
julia> h(a) = a < 10 ? missing : f(a)
h (generic function with 1 method)
julia> @inferred Missing h(20)
ERROR: return type Int64 does not match inferred return type Union{Missing, Float64, Int64}
[...]
Test.@test_logs
— Macro
@test_logs [log_patterns...] [keywords] expression
Collect a list of log records generated by expression
using collect_test_logs
, check that they match the sequence log_patterns
, and return the value of expression
. The keywords
provide some simple filtering of log records: the min_level
keyword controls the minimum log level which will be collected for the test, the match_mode
keyword defines how matching will be performed (the default :all
checks that all logs and patterns match pairwise; use :any
to check that the pattern matches at least once somewhere in the sequence.)
The most useful log pattern is a simple tuple of the form (level,message)
. A different number of tuple elements may be used to match other log metadata, corresponding to the arguments to passed to AbstractLogger
via the handle_message
function: (level,message,module,group,id,file,line)
. Elements which are present will be matched pairwise with the log record fields using ==
by default, with the special cases that Symbol
s may be used for the standard log levels, and Regex
s in the pattern will match string or Symbol fields using occursin
.
Examples
Consider a function which logs a warning, and several debug messages:
function foo(n)
@info "Doing foo with n=$n"
for i=1:n
@debug "Iteration $i"
end
42
end
We can test the info message using
@test_logs (:info,"Doing foo with n=2") foo(2)
If we also wanted to test the debug messages, these need to be enabled with the min_level
keyword:
@test_logs (:info,"Doing foo with n=2") (:debug,"Iteration 1") (:debug,"Iteration 2") min_level=Debug foo(2)
The macro may be chained with @test
to also test the returned value:
@test (@test_logs (:info,"Doing foo with n=2") foo(2)) == 42
Test.@test_deprecated
— Macro
@test_deprecated [pattern] expression
When --depwarn=yes
, test that expression
emits a deprecation warning and return the value of expression
. The log message string will be matched against pattern
which defaults to r"deprecated"i
.
When --depwarn=no
, simply return the result of executing expression
. When --depwarn=error
, check that an ErrorException is thrown.
Examples
# Deprecated in julia 0.7
@test_deprecated num2hex(1)
# The returned value can be tested by chaining with @test:
@test (@test_deprecated num2hex(1)) == "0000000000000001"
Test.@test_warn
— Macro
@test_warn msg expr
Test whether evaluating expr
results in stderr
output that contains the msg
string or matches the msg
regular expression. If msg
is a boolean function, tests whether msg(output)
returns true
. If msg
is a tuple or array, checks that the error output contains/matches each item in msg
. Returns the result of evaluating expr
.
See also @test_nowarn
to check for the absence of error output.
Test.@test_nowarn
— Macro
@test_nowarn expr
Test whether evaluating expr
results in empty stderr
output (no warnings or other messages). Returns the result of evaluating expr
.
损坏的测试
If a test fails consistently it can be changed to use the @test_broken
macro. This will denote the test as Broken
if the test continues to fail and alerts the user via an Error
if the test succeeds.
Test.@test_broken
— Macro
@test_broken ex
@test_broken f(args...) key=val ...
Indicates a test that should pass but currently consistently fails. Tests that the expression ex
evaluates to false
or causes an exception. Returns a Broken
Result
if it does, or an Error
Result
if the expression evaluates to true
.
The @test_broken f(args...) key=val...
form works as for the @test
macro.
Examples
julia> @test_broken 1 == 2
Test Broken
Expression: 1 == 2
julia> @test_broken 1 == 2 atol=0.1
Test Broken
Expression: ==(1, 2, atol=0.1)
@test_skip
is also available to skip a test without evaluation, but counting the skipped test in the test set reporting. The test will not run but gives a Broken
Result
.
Test.@test_skip
— Macro
@test_skip ex
@test_skip f(args...) key=val ...
Marks a test that should not be executed but should be included in test summary reporting as Broken
. This can be useful for tests that intermittently fail, or tests of not-yet-implemented functionality.
The @test_skip f(args...) key=val...
form works as for the @test
macro.
Examples
julia> @test_skip 1 == 2
Test Broken
Skipped: 1 == 2
julia> @test_skip 1 == 2 atol=0.1
Test Broken
Skipped: ==(1, 2, atol=0.1)
自定义 AbstractTestSet 类型
Packages can create their own AbstractTestSet
subtypes by implementing the record
and finish
methods. The subtype should have a one-argument constructor taking a description string, with any options passed in as keyword arguments.
Test.record
— Function
record(ts::AbstractTestSet, res::Result)
Record a result to a testset. This function is called by the @testset
infrastructure each time a contained @test
macro completes, and is given the test result (which could be an Error
). This will also be called with an Error
if an exception is thrown inside the test block but outside of a @test
context.
Test.finish
— Function
finish(ts::AbstractTestSet)
Do any final processing necessary for the given testset. This is called by the @testset
infrastructure after a test block executes. One common use for this function is to record the testset to the parent’s results list, using get_testset
.
Test
takes responsibility for maintaining a stack of nested testsets as they are executed, but any result accumulation is the responsibility of the AbstractTestSet
subtype. You can access this stack with the get_testset
and get_testset_depth
methods. Note that these functions are not exported.
Test.get_testset
— Function
get_testset()
Retrieve the active test set from the task’s local storage. If no test set is active, use the fallback default test set.
Test.get_testset_depth
— Function
get_testset_depth()
Returns the number of active test sets, not including the default test set
Test
also makes sure that nested @testset
invocations use the same AbstractTestSet
subtype as their parent unless it is set explicitly. It does not propagate any properties of the testset. Option inheritance behavior can be implemented by packages using the stack infrastructure that Test
provides.
定义一个基本的 AbstractTestSet
子类:
import Test: record, finish
using Test: AbstractTestSet, Result, Pass, Fail, Error
using Test: get_testset_depth, get_testset
struct CustomTestSet <: Test.AbstractTestSet
description::AbstractString
foo::Int
results::Vector
# constructor takes a description string and options keyword arguments
CustomTestSet(desc; foo=1) = new(desc, foo, [])
end
record(ts::CustomTestSet, child::AbstractTestSet) = push!(ts.results, child)
record(ts::CustomTestSet, res::Result) = push!(ts.results, res)
function finish(ts::CustomTestSet)
# just record if we're not the top-level parent
if get_testset_depth() > 0
record(get_testset(), ts)
end
ts
end
使用测试集:
@testset CustomTestSet foo=4 "custom testset inner 2" begin
# this testset should inherit the type, but not the argument.
@testset "custom testset inner" begin
@test true
end
end