Prioritizing and Filtering Tests

By default, the Eclipse JUnit integration always runs the tests in a suite in the same order (although it is sometimes difficult in advance of the first run to determine what that order will be). For continuous testing, this can be frustrating. Consider our current failure, on testTopJoke. As you try to fix the error, the most interesting thing to you will be whether testTopJoke finally passes. However, every time continuous testing restarts, it runs up to three passing tests before trying testTopJoke. To verify this, open the Continuous Testing view, and choose the Test Order tab:

Since these tests are pretty fast, it doesn't seem too annoying at this point. However, you can imagine that the frustration would build if each test required 5 or 10 seconds of time, or if you had to wait for 100 tests to run before the one you're interested in. To help alleviate this issue, continuous testing allows for several kinds of test prioritization and filtering:
  1. Open the properties dialog for topten, and choose Continuous Testing Properties.
  2. On the bottom left side of the resulting dialog box, you have the option of choosing from six different test prioritization strategies. Select "Most recent failures first": we have found that this is often the most helpful strategy, since the most interesting tests are often those that are currently failed, or that we've recently fixed.
  3. Press OK. The tests are re-run, in order to figure out which tests are currently failing.

To see prioritization in action, leave the Continuous Testing view open. Right-click the marker in the Problem view, and Rerun this test. You'll see that testTopJoke, the failing test, has moved up to the top of the test run order.

Along with prioritizing tests, filtering can also be done on test suites. Some filters have been included with the plugin and are described here. The prioritizers are described here. Prioritizers and filters are also known as test manipulators. These manipulators are given information to help them order and filter the tests. They can use information about previous results, and information about which files have changed. Look here for more information about developing new filters or prioritizers.