Testing Crystal Code

Crystal comes with a fully-featured spec library in the Spec module. It provides a structure for writing executable examples of how your code should behave.

Inspired by Rspec, it includes a domain specific language (DSL) that allows you to write examples in a way similar to plain english.

A basic spec looks something like this:

  1. require "spec"
  3. describe Array do
  4.   describe "#size" do
  5.     it "correctly reports the number of elements in the Array" do
  6.       [1, 2, 3].size.should eq 3
  7.     end
  8.   end
  10.   describe "#empty?" do
  11.     it "is true when no elements are in the array" do
  12.       ([] of Int32).empty?.should be_true
  13.     end
  15.     it "is false if there are elements in the array" do
  16.       [1].empty?.should be_false
  17.     end
  18.   end
  19. end

Anatomy of a spec file

To use the spec module and DSL, you need to add require "spec" to your spec files. Many projects use a custom spec helper which organizes these includes.

Concrete test cases are defined in it blocks. An optional (but strongly recommended) descriptive string states it’s purpose and a block contains the main logic performing the test.

Test cases that have been defined or outlined but are not yet expected to work can be defined using pending instead of it. They will not be run but show up in the spec report as pending.

An it block contains an example that should invoke the code to be tested and define what is expected of it. Each example can contain multiple expectations, but it should test only one specific behaviour.

When spec is included, every object has the instance methods #should and #should_not. These methods are invoked on the value being tested with an expectation as argument. If the expectation is met, code execution continues. Otherwise the example has failed and other code in this block will not be executed.

In test files, specs are structured by example groups which are defined by describe and context sections. Typically a top level describe defines the outer unit (such as a class) to be tested by the spec. Further describe sections can be nested within the outer unit to specify smaller units under test (such as individual methods).

For unit tests, it is recommended to follow the conventions for method names: Outer describe is the name of the class, inner describe targets methods. Instance methods are prefixed with #, class methods with ..

To establish certain contexts - think empty array versus array with elements - the context method may be used to communicate this to the reader. It has a different name, but behaves exactly like describe.

describe and context take a description as argument (which should usually be a string) and a block containing the individual specs or nested groupings.

Expectations

Expectations define if the value being tested (actual) matches a certain value or specific criteria.

Equivalence, Identity and Type

There are methods to create expectations which test for equivalence (eq), identity (be), type (be_a), and nil (be_nil). Note that the identity expectation uses .same? which tests if #object_id are identical. This is only true if the expected value points to the same object instead of an equivalent one. This is only possible for reference types and won’t work for value types like structs or numbers.

  1. actual.should eq(expected)   # passes if actual == expected
  2. actual.should be(expected)   # passes if actual.same?(expected)
  3. actual.should be_a(expected) # passes if actual.is_a?(expected)
  4. actual.should be_nil         # passes if actual.nil?

Truthiness

  1. actual.should be_true   # passes if actual == true
  2. actual.should be_false  # passes if actual == false
  3. actual.should be_truthy # passes if actual is truthy (neither nil nor false nor Pointer.null)
  4. actual.should be_falsey # passes if actual is falsey (nil, false or Pointer.null)

Comparisons

  1. actual.should be < expected  # passes if actual <  expected
  2. actual.should be <= expected # passes if actual <= expected
  3. actual.should be > expected  # passes if actual >  expected
  4. actual.should be >= expected # passes if actual >= expected

Other matchers

  1. actual.should be_close(expected, delta) # passes if actual is within delta of expected:
  2. #                                         (actual - expected).abs <= delta
  3. actual.should contain(expected) # passes if actual.includes?(expected)
  4. actual.should match(expected)   # passes if actual =~ expected

Expecting errors

These matchers run a block and pass if it raises a certain exception.

  1. expect_raises(MyError) do
  2.   # Passes if this block raises an exception of type MyError.
  3. end
  5. expect_raises(MyError, "error message") do
  6.   # Passes if this block raises an exception of type MyError
  7.   # and the error message contains "error message".
  8. end
  10. expect_raises(MyError, /error \w{7}/) do
  11.   # Passes if this block raises an exception of type MyError
  12.   # and the error message matches the regular expression.
  13. end

expect_raises returns the rescued exception so it can be used for further expectations, for example to verify specific properties of the exception.

  1. ex = expect_raises(MyError) do
  2.   # Passes if this block raises an exception of type MyError.
  3. end
  4. ex.my_error_value.should eq "foo"

Focusing on a group of specs

describe, context and it blocks can be marked with focus: true, like this:

  1. it "adds", focus: true do
  2.   (2 + 2).should_not eq(5)
  3. end

If any such thing is marked with focus: true then only those examples will run.

Tagging specs

Tags can be used to group specs, allowing to only run a subset of specs when providing a --tag argument to the spec runner (see Using the compiler).

describe, context and it blocks can be tagged, like this:

  1. it "is slow", tags: "slow" do
  2.   sleep 60
  3.   true.should be_true
  4. end
  6. it "is fast", tags: "fast" do
  7.   true.should be_true
  8. end

Tagging an example group (describe or context) extends to all of the contained examples.

Multiple tags can be specified by giving an Enumerable, such as Array or Set.

Running specs

The Crystal compiler has a spec command with tools to constrain which examples get run and tailor the output. All specs of a project are compiled and executed through the command crystal spec.

By convention, specs live in the spec/ directory of a project. Spec files must end with _spec.cr to be recognizable as such by the compiler command.

You can compile and run specs from folder trees, individual files, or specific lines in a file. If the specified line is the beginning of a describe or context section, all specs inside that group are run.

The default formatter outputs the file and line style command for failing specs which makes it easy to rerun just this individual spec.

You can turn off colors with the switch --no-color.

Randomizing order of specs

Specs, by default, run in the order defined, but can be run in a random order by passing --order random to crystal spec.

Specs run in random order will display a seed value upon completion. This seed value can be used to rerun the specs in that same order by passing the seed value to --order.

Examples

  1. # Run all specs in files matching spec/**/*_spec.cr
  2. crystal spec
  4. # Run  all specs in files matching spec/**/*_spec.cr without colors
  5. crystal spec --no-color
  7. # Run all specs in files matching spec/my/test/**/*_spec.cr
  8. crystal spec spec/my/test/
  10. # Run all specs in spec/my/test/file_spec.cr
  11. crystal spec spec/my/test/file_spec.cr
  13. # Run the spec or group defined in line 14 of spec/my/test/file_spec.cr
  14. crystal spec spec/my/test/file_spec.cr:14
  16. # Run all specs tagged with "fast"
  17. crystal spec --tag 'fast'
  19. # Run all specs not tagged with "slow"
  20. crystal spec --tag '~slow'

Spec helper

Many projects use a custom spec helper file, usually named spec/spec_helper.cr.

This file is used to require spec and other includes like code from the project needed for every spec file. This is also a good place to define global helper methods that make writing specs easier and avoid code duplication.

spec/spec_helper.cr

  1. require "spec"
  2. require "../src/my_project.cr"
  4. def create_test_object(name)
  5.   project = MyProject.new(option: false)
  6.   object = project.create_object(name)
  7.   object
  8. end

spec/my_project_spec.cr

  1. require "./spec_helper"
  3. describe "MyProject::Object" do
  4.   it "is created" do
  5.     object = create_test_object(name)
  6.     object.should_not be_nil
  7.   end
  8. end