2.15. Parameterized Tests

Parameterized tests make it possible to run a test multiple times with different arguments. They are declared just like regular @Test methods but use the [@ParameterizedTest](https://junit.org/junit5/docs/current/api/org.junit.jupiter.params/org/junit/jupiter/params/ParameterizedTest.html) annotation instead. In addition, you must declare at least one source that will provide the arguments for each invocation and then consume the arguments in the test method.

The following example demonstrates a parameterized test that uses the @ValueSource annotation to specify a String array as the source of arguments.

@ParameterizedTest
@ValueSource(strings = { "racecar", "radar", "able was I ere I saw elba" })
void palindromes(String candidate) {
    assertTrue(StringUtils.isPalindrome(candidate));
}

When executing the above parameterized test method, each invocation will be reported separately. For instance, the ConsoleLauncher will print output similar to the following.

palindromes(String) ✔
├─ [1] candidate=racecar ✔
├─ [2] candidate=radar ✔
└─ [3] candidate=able was I ere I saw elba ✔

2.15.1. Required Setup

In order to use parameterized tests you need to add a dependency on the junit-jupiter-params artifact. Please refer to Dependency Metadata for details.

2.15.2. Consuming Arguments

Parameterized test methods typically consume arguments directly from the configured source (see Sources of Arguments) following a one-to-one correlation between argument source index and method parameter index (see examples in @CsvSource). However, a parameterized test method may also choose to aggregate arguments from the source into a single object passed to the method (see Argument Aggregation). Additional arguments may also be provided by a ParameterResolver (e.g., to obtain an instance of TestInfo, TestReporter, etc.). Specifically, a parameterized test method must declare formal parameters according to the following rules.

• Zero or more indexed arguments must be declared first.

• Zero or more aggregators must be declared next.

• Zero or more arguments supplied by a ParameterResolver must be declared last.

In this context, an indexed argument is an argument for a given index in the Arguments provided by an ArgumentsProvider that is passed as an argument to the parameterized method at the same index in the method’s formal parameter list. An aggregator is any parameter of type ArgumentsAccessor or any parameter annotated with @AggregateWith.

2.15.3. Sources of Arguments

Out of the box, JUnit Jupiter provides quite a few source annotations. Each of the following subsections provides a brief overview and an example for each of them. Please refer to the Javadoc in the [org.junit.jupiter.params.provider](https://junit.org/junit5/docs/current/api/org.junit.jupiter.params/org/junit/jupiter/params/provider/package-summary.html) package for additional information.

@ValueSource

@ValueSource is one of the simplest possible sources. It lets you specify a single array of literal values and can only be used for providing a single argument per parameterized test invocation.

The following types of literal values are supported by @ValueSource.

• short

• byte

• int

• long

• float

• double

• char

• boolean

• java.lang.String

• java.lang.Class

For example, the following @ParameterizedTest method will be invoked three times, with the values 1, 2, and 3 respectively.

@ParameterizedTest
@ValueSource(ints = { 1, 2, 3 })
void testWithValueSource(int argument) {
    assertTrue(argument > 0 && argument < 4);
}

Null and Empty Sources

In order to check corner cases and verify proper behavior of our software when it is supplied bad input, it can be useful to have null and empty values supplied to our parameterized tests. The following annotations serve as sources of null and empty values for parameterized tests that accept a single argument.

• [@NullSource](https://junit.org/junit5/docs/current/api/org.junit.jupiter.params/org/junit/jupiter/params/provider/NullSource.html): provides a single null argument to the annotated @ParameterizedTest method.

  • @NullSource cannot be used for a parameter that has a primitive type.

• [@EmptySource](https://junit.org/junit5/docs/current/api/org.junit.jupiter.params/org/junit/jupiter/params/provider/EmptySource.html): provides a single empty argument to the annotated @ParameterizedTest method for parameters of the following types: java.lang.String, java.util.List, java.util.Set, java.util.Map, primitive arrays (e.g., int[], char[][], etc.), object arrays (e.g.,String[], Integer[][], etc.).

  • Subtypes of the supported types are not supported.

• [@NullAndEmptySource](https://junit.org/junit5/docs/current/api/org.junit.jupiter.params/org/junit/jupiter/params/provider/NullAndEmptySource.html): a composed annotation that combines the functionality of @NullSource and @EmptySource.

If you need to supply multiple varying types of blank strings to a parameterized test, you can achieve that using @ValueSource — for example, @ValueSource(strings = {" ", " ", "\t", "\n"}).

You can also combine @NullSource, @EmptySource, and @ValueSource to test a wider range of null, empty, and blank input. The following example demonstrates how to achieve this for strings.

@ParameterizedTest
@NullSource
@EmptySource
@ValueSource(strings = { " ", "   ", "\t", "\n" })
void nullEmptyAndBlankStrings(String text) {
    assertTrue(text == null || text.trim().isEmpty());
}

Making use of the composed @NullAndEmptySource annotation simplifies the above as follows.

@ParameterizedTest
@NullAndEmptySource
@ValueSource(strings = { " ", "   ", "\t", "\n" })
void nullEmptyAndBlankStrings(String text) {
    assertTrue(text == null || text.trim().isEmpty());
}

@EnumSource

@EnumSource provides a convenient way to use Enum constants.

@ParameterizedTest
@EnumSource(ChronoUnit.class)
void testWithEnumSource(TemporalUnit unit) {
    assertNotNull(unit);
}

The annotation’s value attribute is optional. When omitted, the declared type of the first method parameter is used. The test will fail if it does not reference an enum type. Thus, the value attribute is required in the above example because the method parameter is declared as TemporalUnit, i.e. the interface implemented by ChronoUnit, which isn’t an enum type. Changing the method parameter type to ChronoUnit allows you to omit the explicit enum type from the annotation as follows.

@ParameterizedTest
@EnumSource
void testWithEnumSourceWithAutoDetection(ChronoUnit unit) {
    assertNotNull(unit);
}

The annotation provides an optional names attribute that lets you specify which constants shall be used, like in the following example. If omitted, all constants will be used.

@ParameterizedTest
@EnumSource(names = { "DAYS", "HOURS" })
void testWithEnumSourceInclude(ChronoUnit unit) {
    assertTrue(EnumSet.of(ChronoUnit.DAYS, ChronoUnit.HOURS).contains(unit));
}

The @EnumSource annotation also provides an optional mode attribute that enables fine-grained control over which constants are passed to the test method. For example, you can exclude names from the enum constant pool or specify regular expressions as in the following examples.

@ParameterizedTest
@EnumSource(mode = EXCLUDE, names = { "ERAS", "FOREVER" })
void testWithEnumSourceExclude(ChronoUnit unit) {
    assertFalse(EnumSet.of(ChronoUnit.ERAS, ChronoUnit.FOREVER).contains(unit));
}

@ParameterizedTest
@EnumSource(mode = MATCH_ALL, names = "^.*DAYS$")
void testWithEnumSourceRegex(ChronoUnit unit) {
    assertTrue(unit.name().endsWith("DAYS"));
}

@MethodSource

[@MethodSource](https://junit.org/junit5/docs/current/api/org.junit.jupiter.params/org/junit/jupiter/params/provider/MethodSource.html) allows you to refer to one or more factory methods of the test class or external classes.

Factory methods within the test class must be static unless the test class is annotated with @TestInstance(Lifecycle.PER_CLASS); whereas, factory methods in external classes must always be static. In addition, such factory methods must not accept any arguments.

Each factory method must generate a stream of arguments, and each set of arguments within the stream will be provided as the physical arguments for individual invocations of the annotated @ParameterizedTest method. Generally speaking this translates to a Stream of Arguments (i.e., Stream<Arguments>); however, the actual concrete return type can take on many forms. In this context, a “stream” is anything that JUnit can reliably convert into a Stream, such as Stream, DoubleStream, LongStream, IntStream, Collection, Iterator, Iterable, an array of objects, or an array of primitives. The “arguments” within the stream can be supplied as an instance of Arguments, an array of objects (e.g., Object[]), or a single value if the parameterized test method accepts a single argument.

If you only need a single parameter, you can return a Stream of instances of the parameter type as demonstrated in the following example.

@ParameterizedTest
@MethodSource("stringProvider")
void testWithExplicitLocalMethodSource(String argument) {
    assertNotNull(argument);
}
static Stream<String> stringProvider() {
    return Stream.of("apple", "banana");
}

If you do not explicitly provide a factory method name via @MethodSource, JUnit Jupiter will search for a factory method that has the same name as the current @ParameterizedTest method by convention. This is demonstrated in the following example.

@ParameterizedTest
@MethodSource
void testWithDefaultLocalMethodSource(String argument) {
    assertNotNull(argument);
}
static Stream<String> testWithDefaultLocalMethodSource() {
    return Stream.of("apple", "banana");
}

Streams for primitive types (DoubleStream, IntStream, and LongStream) are also supported as demonstrated by the following example.

@ParameterizedTest
@MethodSource("range")
void testWithRangeMethodSource(int argument) {
    assertNotEquals(9, argument);
}
static IntStream range() {
    return IntStream.range(0, 20).skip(10);
}

If a parameterized test method declares multiple parameters, you need to return a collection, stream, or array of Arguments instances or object arrays as shown below (see the Javadoc for [@MethodSource](https://junit.org/junit5/docs/current/api/org.junit.jupiter.params/org/junit/jupiter/params/provider/MethodSource.html) for further details on supported return types). Note that arguments(Object…​) is a static factory method defined in the Arguments interface. In addition, Arguments.of(Object…​) may be used as an alternative to arguments(Object…​).

@ParameterizedTest
@MethodSource("stringIntAndListProvider")
void testWithMultiArgMethodSource(String str, int num, List<String> list) {
    assertEquals(5, str.length());
    assertTrue(num >=1 && num <=2);
    assertEquals(2, list.size());
}
static Stream<Arguments> stringIntAndListProvider() {
    return Stream.of(
        arguments("apple", 1, Arrays.asList("a", "b")),
        arguments("lemon", 2, Arrays.asList("x", "y"))
    );
}

An external, static factory method can be referenced by providing its fully qualified method name as demonstrated in the following example.

package example;
import java.util.stream.Stream;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.MethodSource;
class ExternalMethodSourceDemo {
    @ParameterizedTest
    @MethodSource("example.StringsProviders#tinyStrings")
    void testWithExternalMethodSource(String tinyString) {
        // test with tiny string
    }
}
class StringsProviders {
    static Stream<String> tinyStrings() {
        return Stream.of(".", "oo", "OOO");
    }
}

@CsvSource

@CsvSource allows you to express argument lists as comma-separated values (i.e., String literals).

@ParameterizedTest
@CsvSource({
    "apple,         1",
    "banana,        2",
    "'lemon, lime', 0xF1"
})
void testWithCsvSource(String fruit, int rank) {
    assertNotNull(fruit);
    assertNotEquals(0, rank);
}

The default delimiter is a comma (,), but you can use another character by setting the delimiter attribute. Alternatively, the delimiterString attribute allows you to use a String delimiter instead of a single character. However, both delimiter attributes cannot be set simultaneously.

@CsvSource uses a single quote ' as its quote character. See the 'lemon, lime' value in the example above and in the table below. An empty, quoted value '' results in an empty String unless the emptyValue attribute is set; whereas, an entirely empty value is interpreted as a null reference. By specifying one or more nullValues, a custom value can be interpreted as a null reference (see the NIL example in the table below). An ArgumentConversionException is thrown if the target type of a null reference is a primitive type.

@CsvFileSource

@CsvFileSource lets you use CSV files from the classpath or the local file system. Each line from a CSV file results in one invocation of the parameterized test.

The default delimiter is a comma (,), but you can use another character by setting the delimiter attribute. Alternatively, the delimiterString attribute allows you to use a String delimiter instead of a single character. However, both delimiter attributes cannot be set simultaneously.

@ParameterizedTest
@CsvFileSource(resources = "/two-column.csv", numLinesToSkip = 1)
void testWithCsvFileSourceFromClasspath(String country, int reference) {
    assertNotNull(country);
    assertNotEquals(0, reference);
}
@ParameterizedTest
@CsvFileSource(files = "src/test/resources/two-column.csv", numLinesToSkip = 1)
void testWithCsvFileSourceFromFile(String country, int reference) {
    assertNotNull(country);
    assertNotEquals(0, reference);
}

two-column.csv

Country, reference
Sweden, 1
Poland, 2
"United States of America", 3

In contrast to the syntax used in @CsvSource, @CsvFileSource uses a double quote " as the quote character. See the "United States of America" value in the example above. An empty, quoted value "" results in an empty String unless the emptyValue attribute is set; whereas, an entirely empty value is interpreted as a null reference. By specifying one or more nullValues, a custom value can be interpreted as a null reference. An ArgumentConversionException is thrown if the target type of a null reference is a primitive type.

@ArgumentsSource

@ArgumentsSource can be used to specify a custom, reusable ArgumentsProvider. Note that an implementation of ArgumentsProvider must be declared as either a top-level class or as a static nested class.

@ParameterizedTest
@ArgumentsSource(MyArgumentsProvider.class)
void testWithArgumentsSource(String argument) {
    assertNotNull(argument);
}

public class MyArgumentsProvider implements ArgumentsProvider {
    @Override
    public Stream<? extends Arguments> provideArguments(ExtensionContext context) {
        return Stream.of("apple", "banana").map(Arguments::of);
    }
}

2.15.4. Argument Conversion

Widening Conversion

JUnit Jupiter supports Widening Primitive Conversion for arguments supplied to a @ParameterizedTest. For example, a parameterized test annotated with @ValueSource(ints = { 1, 2, 3 }) can be declared to accept not only an argument of type int but also an argument of type long, float, or double.

Implicit Conversion

To support use cases like @CsvSource, JUnit Jupiter provides a number of built-in implicit type converters. The conversion process depends on the declared type of each method parameter.

For example, if a @ParameterizedTest declares a parameter of type TimeUnit and the actual type supplied by the declared source is a String, the string will be automatically converted into the corresponding TimeUnit enum constant.

@ParameterizedTest
@ValueSource(strings = "SECONDS")
void testWithImplicitArgumentConversion(ChronoUnit argument) {
    assertNotNull(argument.name());
}

String instances are implicitly converted to the following target types.

Fallback String-to-Object Conversion

In addition to implicit conversion from strings to the target types listed in the above table, JUnit Jupiter also provides a fallback mechanism for automatic conversion from a String to a given target type if the target type declares exactly one suitable factory method or a factory constructor as defined below.

• factory method: a non-private, static method declared in the target type that accepts a single String argument and returns an instance of the target type. The name of the method can be arbitrary and need not follow any particular convention.

• factory constructor: a non-private constructor in the target type that accepts a single String argument. Note that the target type must be declared as either a top-level class or as a static nested class.

For example, in the following @ParameterizedTest method, the Book argument will be created by invoking the Book.fromTitle(String) factory method and passing "42 Cats" as the title of the book.

@ParameterizedTest
@ValueSource(strings = "42 Cats")
void testWithImplicitFallbackArgumentConversion(Book book) {
    assertEquals("42 Cats", book.getTitle());
}

public class Book {
    private final String title;
    private Book(String title) {
        this.title = title;
    }
    public static Book fromTitle(String title) {
        return new Book(title);
    }
    public String getTitle() {
        return this.title;
    }
}

Explicit Conversion

Instead of relying on implicit argument conversion you may explicitly specify an ArgumentConverter to use for a certain parameter using the @ConvertWith annotation like in the following example. Note that an implementation of ArgumentConverter must be declared as either a top-level class or as a static nested class.

@ParameterizedTest
@EnumSource(ChronoUnit.class)
void testWithExplicitArgumentConversion(
        @ConvertWith(ToStringArgumentConverter.class) String argument) {
    assertNotNull(ChronoUnit.valueOf(argument));
}

public class ToStringArgumentConverter extends SimpleArgumentConverter {
    @Override
    protected Object convert(Object source, Class<?> targetType) {
        assertEquals(String.class, targetType, "Can only convert to String");
        if (source instanceof Enum<?>) {
            return ((Enum<?>) source).name();
        }
        return String.valueOf(source);
    }
}

If the converter is only meant to convert one type to another, you can extend TypedArgumentConverter to avoid boilerplate type checks.

public class ToLengthArgumentConverter extends TypedArgumentConverter<String, Integer> {
    protected ToLengthArgumentConverter() {
        super(String.class, Integer.class);
    }
    @Override
    protected Integer convert(String source) {
        return source.length();
    }
}

Explicit argument converters are meant to be implemented by test and extension authors. Thus, junit-jupiter-params only provides a single explicit argument converter that may also serve as a reference implementation: JavaTimeArgumentConverter. It is used via the composed annotation JavaTimeConversionPattern.

@ParameterizedTest
@ValueSource(strings = { "01.01.2017", "31.12.2017" })
void testWithExplicitJavaTimeConverter(
        @JavaTimeConversionPattern("dd.MM.yyyy") LocalDate argument) {
    assertEquals(2017, argument.getYear());
}

2.15.5. Argument Aggregation

By default, each argument provided to a @ParameterizedTest method corresponds to a single method parameter. Consequently, argument sources which are expected to supply a large number of arguments can lead to large method signatures.

In such cases, an [ArgumentsAccessor](https://junit.org/junit5/docs/current/api/org.junit.jupiter.params/org/junit/jupiter/params/aggregator/ArgumentsAccessor.html) can be used instead of multiple parameters. Using this API, you can access the provided arguments through a single argument passed to your test method. In addition, type conversion is supported as discussed in Implicit Conversion.

@ParameterizedTest
@CsvSource({
    "Jane, Doe, F, 1990-05-20",
    "John, Doe, M, 1990-10-22"
})
void testWithArgumentsAccessor(ArgumentsAccessor arguments) {
    Person person = new Person(arguments.getString(0),
                               arguments.getString(1),
                               arguments.get(2, Gender.class),
                               arguments.get(3, LocalDate.class));
    if (person.getFirstName().equals("Jane")) {
        assertEquals(Gender.F, person.getGender());
    }
    else {
        assertEquals(Gender.M, person.getGender());
    }
    assertEquals("Doe", person.getLastName());
    assertEquals(1990, person.getDateOfBirth().getYear());
}

An instance of ArgumentsAccessor is automatically injected into any parameter of type ArgumentsAccessor.

Custom Aggregators

Apart from direct access to a @ParameterizedTest method’s arguments using an ArgumentsAccessor, JUnit Jupiter also supports the usage of custom, reusable aggregators.

To use a custom aggregator, implement the [ArgumentsAggregator](https://junit.org/junit5/docs/current/api/org.junit.jupiter.params/org/junit/jupiter/params/aggregator/ArgumentsAggregator.html) interface and register it via the @AggregateWith annotation on a compatible parameter in the @ParameterizedTest method. The result of the aggregation will then be provided as an argument for the corresponding parameter when the parameterized test is invoked. Note that an implementation of ArgumentsAggregator must be declared as either a top-level class or as a static nested class.

@ParameterizedTest
@CsvSource({
    "Jane, Doe, F, 1990-05-20",
    "John, Doe, M, 1990-10-22"
})
void testWithArgumentsAggregator(@AggregateWith(PersonAggregator.class) Person person) {
    // perform assertions against person
}

public class PersonAggregator implements ArgumentsAggregator {
    @Override
    public Person aggregateArguments(ArgumentsAccessor arguments, ParameterContext context) {
        return new Person(arguments.getString(0),
                          arguments.getString(1),
                          arguments.get(2, Gender.class),
                          arguments.get(3, LocalDate.class));
    }
}

If you find yourself repeatedly declaring @AggregateWith(MyTypeAggregator.class) for multiple parameterized test methods across your codebase, you may wish to create a custom composed annotation such as @CsvToMyType that is meta-annotated with @AggregateWith(MyTypeAggregator.class). The following example demonstrates this in action with a custom @CsvToPerson annotation.

@ParameterizedTest
@CsvSource({
    "Jane, Doe, F, 1990-05-20",
    "John, Doe, M, 1990-10-22"
})
void testWithCustomAggregatorAnnotation(@CsvToPerson Person person) {
    // perform assertions against person
}

@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.PARAMETER)
@AggregateWith(PersonAggregator.class)
public @interface CsvToPerson {
}

2.15.6. Customizing Display Names

By default, the display name of a parameterized test invocation contains the invocation index and the String representation of all arguments for that specific invocation. Each of them is preceded by the parameter name (unless the argument is only available via an ArgumentsAccessor or ArgumentAggregator), if present in the bytecode (for Java, test code must be compiled with the -parameters compiler flag).

However, you can customize invocation display names via the name attribute of the @ParameterizedTest annotation like in the following example.

@DisplayName("Display name of container")
@ParameterizedTest(name = "{index} ==> the rank of ''{0}'' is {1}")
@CsvSource({ "apple, 1", "banana, 2", "'lemon, lime', 3" })
void testWithCustomDisplayNames(String fruit, int rank) {
}

When executing the above method using the ConsoleLauncher you will see output similar to the following.

Display name of container ✔
├─ 1 ==> the rank of 'apple' is 1 ✔
├─ 2 ==> the rank of 'banana' is 2 ✔
└─ 3 ==> the rank of 'lemon, lime' is 3 ✔

Please note that name is a MessageFormat pattern. Thus, a single quote (') needs to be represented as a doubled single quote ('') in order to be displayed.

The following placeholders are supported within custom display names.

2.15.7. Lifecycle and Interoperability

Each invocation of a parameterized test has the same lifecycle as a regular @Test method. For example, @BeforeEach methods will be executed before each invocation. Similar to Dynamic Tests, invocations will appear one by one in the test tree of an IDE. You may at will mix regular @Test methods and @ParameterizedTest methods within the same test class.

You may use ParameterResolver extensions with @ParameterizedTest methods. However, method parameters that are resolved by argument sources need to come first in the argument list. Since a test class may contain regular tests as well as parameterized tests with different parameter lists, values from argument sources are not resolved for lifecycle methods (e.g. @BeforeEach) and test class constructors.

@BeforeEach
void beforeEach(TestInfo testInfo) {
    // ...
}
@ParameterizedTest
@ValueSource(strings = "apple")
void testWithRegularParameterResolver(String argument, TestReporter testReporter) {
    testReporter.publishEntry("argument", argument);
}
@AfterEach
void afterEach(TestInfo testInfo) {
    // ...
}