JUnit 5 Release Notes (original) (raw)

This document contains the change log for all JUnit 5 releases since 5.6 GA.

Please refer to the User Guide for comprehensive reference documentation for programmers writing tests, extension authors, and engine authors as well as build tool and IDE vendors.

5.7.1

Date of Release: February 4, 2021

Scope: Bug fixes since 5.7.0

For a complete list of all closed issues and pull requests for this release, consult the5.7.1 milestone page in the JUnit repository on GitHub.

General bug fixes

• For compatibility with JarInputStream, all JARs now contain META-INF/ andMETA-INF/MANIFEST.MF as their first entries again.

JUnit Platform

Bug Fixes

• StringUtils.nullSafeToString() now returns "null" if the invocation of toString()on the supplied object returns null. Although this is an internal utility, the effect of this change may be witnessed by end users and test engine or extension authors.
• Method scanForClassesInPackage(String) in ClasspathScanner now returns a valid list of class names when the package name is equal to the name of a module on the module path.
• The legacy XML report now always includes container-level failures (e.g. from@BeforeAll Jupiter lifecycle methods).

JUnit Jupiter

Bug Fixes

• If the toString() implementation of an argument passed to a @ParameterizedTestmethod returns null, the display name formatter for the parameterized test now treats the name of the corresponding argument as "null" instead of throwing an exception. This fixes a regression introduced in JUnit Jupiter 5.7.0.
• Creator functions passed to ExtensionContext.Store.getOrComputeIfAbsent() are now only called once even if they throw an exception.

New Features and Improvements

• The user guide now explains Nested Tests in more detail.
• New utility method in TestInstancePreDestroyCallback helps to ensure all test instances are processed when used in conjunction with @Nested tests.
• JAVA_17 has been added to the JRE enum for use with JRE-based execution conditions.

JUnit Vintage

Bug Fixes

• The legacy reporting name of top-level test classes is now always their fully-qualified class name. Previously, a textual description was returned for JUnit 3 suites.

5.7.0

Date of Release: September 13, 2020

Scope:

• Promotion of experimental features in JUnit Platform and Jupiter
• Java Flight Recorder support
• TestKit improvements
• @Isolated tests
• Configurable default method orderer
• Custom disabled reasons for all @Enabled*/@Disabled* annotations
• Improvements to assertTimeoutPreemptively()
• Improvements to @CsvFileSource and @CsvSource

For a complete list of all closed issues and pull requests for this release, consult the5.7 M1,5.7 RC1, and5.7 GA milestone pages in the JUnit repository on GitHub.

Overall Improvements

• Javadoc JARs now contain package-list in addition to element-list for compatibility with tools like NetBeans 11.

JUnit Platform

Promoted Features

The following APIs have been promoted from "experimental" status:

• LauncherConstants is now stable
• LauncherConfig (except for methods introduced in 5.7) is now stable
• LegacyXmlReportGeneratingListener is now stable
• org.junit.platform.testkit.engine is now maintained

Bug Fixes

• In XML reports generated by the ConsoleLauncher orLegacyXmlReportGeneratingListener, characters in exception messages and other user-supplied values that are not allowed in XML are now replaced with their character reference – for example, \0 becomes �.
• The Launcher now throws an exception when a previously executed TestPlan is attempted to be executed again.

Deprecations and Breaking Changes

• In the EngineTestKit API, the all(), containers(), and tests() methods inEngineExecutionResults that were deprecated in JUnit Platform 1.6.0 have been removed in favor of allEvents(), containerEvents(), and testEvents(), respectively.
• The following methods in EngineTestKit are now deprecated with replacements:
  • execute(String, EngineDiscoveryRequest) → execute(String, LauncherDiscoveryRequest)
  • execute(TestEngine, EngineDiscoveryRequest) → execute(TestEngine, LauncherDiscoveryRequest)
  • Builder.filters(DiscoveryFilter…​) → Builder.filters(Filter…​)
• EngineTestKit no longer takes into account implicit configuration parameters (i.e. system properties and the junit-platform.properties classpath resource) by default. This change makes tests executed via EngineTestKit independent of the environment they are executed in.

New Features and Improvements

• The number of containers and tests excluded by post discovery filters based on their tags is now logged, along with the exclusion reasons.
• New junit.platform.execution.listeners.deactivate configuration parameter that allows one to specify a comma-separated list of patterns for deactivatingTestExecutionListener implementations registered via the ServiceLoader mechanism.
• The @Testable annotation may now be applied directly to fields.
• New Node.DynamicTestExecutor#execute(TestDescriptor, EngineExecutionListener) method for engines that wish to provide a custom EngineExecutionListener and cancel or wait for the execution of a submitted test via the returned Future.
• New EngineExecutionListener.NOOP EngineExecutionListener implementation.
• All declared methods in the EngineExecutionListener API now have empty defaultimplementations.
• The EngineTestKit now reuses the same test discovery and execution logic as theLauncher. Thus, it’s now possible to test an engine’s behavior in the presence of post-discovery filters (e.g. tag filters) and with regard to pruning.
• The EngineTestKit now supports matching conditions with events loosely, i.e. an incomplete match with or without a fixed order.
• The @Testable annotation may now target any element type, including fields, methods, classes, packages, and modules.
• When using the ConsoleLauncher, classes selected explicitly via --select-class and--select-method are now always executed regardless of class name patterns provided via --include-classname or the default class name pattern.
• The ConsoleLauncher now honors the --disable-ansi-colors option when printing usage help.
• New FilePosition support in FileSelector and ClasspathResourceSelector.
• New getJavaClass() and getJavaMethod() methods inorg.junit.platform.engine.support.descriptor.MethodSource.
• Custom PostDiscoveryFilter implementations can now be registered via Java’sServiceLoader mechanism.
• New org.junit.platform.jfr module. When running on Java 11 or later, it provides and registers a TestExecutionListener that generates Java Flight Recorder events.
• The ExecutionRecorder in junit-platform-testkit is now public for fine-grained testing of classes that use EngineExecutionListener.
• ForkJoinPoolHierarchicalTestExecutorService can now be constructed by supplying aParallelExecutionConfiguration.
• HierarchicalTestEngine now supports a global resource lock.

JUnit Jupiter

Promoted Features

The following APIs have been promoted from experimental to stable:

• junit-jupiter-migrationsupport
• junit-jupiter-params (i.e. @ParameterizedTest etc.)
• @TestMethodOrder, MethodOrderer, and its pre-5.7 implementations
• @DisplayNameGeneration, DisplayNameGenerator, and its pre-5.7 implementations
• @Timeout
• TestInstanceFactory
• TestInstancePreDestroyCallback
• TestInstances and corresponding ExtensionContext methods
• TestWatcher
• Kotlin-specific assertions that were introduced in 5.1

Bug Fixes

• @TempDir is now able to clean up files in read-only directories.
• The Jupiter engine now ignores MethodSelectors for methods in non-Jupiter test classes instead of failing for missing methods in such cases.
• CloseableResource instances stored in ExtensionContext.Store are now closed in the reverse order they were added in. Previously, the order was undefined and unstable.
• Inherited @BeforeEach methods are now executed on correct instances for @Nestedclasses.
• Registered TestInstancePreDestroyCallback extensions are now always called if an instance of a test class was created, regardless whether any registeredTestInstancePostProcessor extension threw an exception.
• Disabled @TestTemplate methods (e.g. @ParameterizedTest and @RepeatedTest methods) are now reported to registered TestWatcher extensions.

Deprecations and Breaking Changes

• MethodOrderer.Alphanumeric has been deprecated in favor of MethodOrderer.MethodNamewhich provides the exact same functionality but has a more descriptive name.

New Features and Improvements

• New @EnabledIf and @DisabledIf annotations can be used to enable or disable a test or container based on condition methods.
• New MethodOrderer named DisplayName that sorts test methods alphanumerically based on their display names.
• New DisplayNameGenerator named Simple (based on Standard) that removes trailing parentheses for methods with no parameters.
• assertThrows() for Kotlin can now be used with suspending functions and other lambda contexts that require inlining.
• The JRE enum now provides a static currentVersion() method that returns the enum constant for the currently executing JRE, e.g. for use in custom execution conditions and other extensions.
• The name attribute of @ParameterizedTest is now clearly documented to be aMessageFormat pattern.
• Synthetic constructors are now ignored when instantiating a test class.
• The Javadoc for the provideTestTemplateInvocationContexts() method inTestTemplateInvocationContextProvider has been aligned with the actual implementation. Providers are now officially allowed to return an empty stream, and the error message when all provided streams are empty is now more helpful.
• New getDisplayName() method in MethodDescriptor for use in MethodOrdererimplementations.
• New assertLinesMatch() method overloads in Assertions that accept twoStream<String> instances for comparison.
• assertTimeoutPreemptively() in Assertions now reports the stack trace of the timed out thread in the cause of the AssertionFailedError.
• assertTimeoutPreemptively() now uses threads with a specific name, conveying their use by the framework, to facilitate debugging and stack trace analysis.
• All @Enabled*/@Disabled* annotations now have an optional disabledReason attribute that can be used to provide an additional explanation as to why a test or container might be disabled.
• JAVA_16 has been added to the JRE enum for use with JRE-based execution conditions.
• New MethodOrderer.MethodName to replace MethodOrderer.Alphanumeric with the exact same functionality but a more descriptive name.
• New junit.jupiter.testmethod.order.default configuration parameter to set the defaultMethodOrderer that will be used unless @TestMethodOrder is present.
• New DynamicTest.stream() factory method that accepts a Stream instead of anIterator for the input source.
• @CsvFileSource now allows one to specify file paths as an alternative to classpath resources.
• @CsvFileSource and @CsvSource now provide a maxCharsPerColumn attribute for configuring the maximum number of characters per column.
• Arguments in display names of parameterized test invocations are now truncated if they exceed a configurable maximum length (defaults to 512 characters).
• New @Isolated annotation allows to run test classes in isolation of other test classes when using parallel test execution.
• New TypedArgumentConverter for converting one specific type to another, therefore reducing boilerplate type checks compared to implementing ArgumentConverter directly.
• New ExtensionContext.getConfigurationParameter(String, Function<String, T>)convenience method for reading transformed configuration parameters from extensions.

JUnit Vintage

Bug Fixes

• The Vintage engine no longer fails when resolving a MethodSelector for methods of test classes that cannot be found via reflection. This allows selecting Spock feature methods by their source code name even though they have a generated method name in the bytecode.

New Features and Improvements

• The internal JUnit4VersionCheck class — which verifies that a supported version of JUnit 4 is on the classpath — now implements a lenient version ID parsing algorithm in order to support custom version ID formats such as 4.12.0, 4.12-patch_1, etc.

5.6.2

Date of Release: April 10, 2020

Scope: Bug fixes since 5.6.1

For a complete list of all closed issues and pull requests for this release, consult the5.6.2 milestone page in the JUnit repository on GitHub.

JUnit Platform

Bug Fixes

• ReflectionSupport.findNestedClasses() no longer detects inner class cycles for classes that do not match the supplied Predicate. For example, JUnit Jupiter no longer throws an exception if an inner class cycle is detected in a nested class hierarchy whose inner classes are not annotated with @Nested.

JUnit Jupiter

Bug Fixes

• Test discovery no longer halts with an exception for inner class hierarchies that form a cycle if such inner classes are not annotated with @Nested.

JUnit Vintage

Bug Fixes

• Generating display names from JUnit 4 Descriptions now falls back to getDisplayNameif getMethodName returns a blank String instead of throwing an exception.

5.6.1

Date of Release: March 22, 2020

Scope: Bug fixes since 5.6.0

For a complete list of all closed issues and pull requests for this release, consult the5.6.1 milestone page in the JUnit repository on GitHub.

JUnit Platform

Bug Fixes

• In order to avoid file locking issues on Microsoft Windows, URLs are no longer cached when loading junit-platform.properties files.
• The presence of multiple junit-platform.properties files on the classpath now only results in a warning if the files have different URLs.

JUnit Jupiter

Bug Fixes

• TestInstancePreDestroyCallback extensions are now invoked in reverse registration order when PER_CLASS test instance lifecycle semantics have been configured.

JUnit Vintage

No changes.

5.6.0

Date of Release: January 20, 2020

Scope:

• New @EnabledForJreRange and @DisabledForJreRange execution conditions
• @Order allows to specify relative order
• Parameter names are included in default display names of parameterized test invocations
• Improvements to @CsvSource and @CsvFileSource
• New TestInstancePreDestroyCallback extension API
• Performance improvements and bug fixes for the Vintage engine
• Improved error reporting for failures during test discovery and execution
• Support for using any() and none() in tag expressions
• org.junit.platform.console now provides a java.util.spi.ToolProvider
• DiscoverySelectors for tests in inherited nested classes
• OSGi metadata
• Minor bug fixes and improvements