Parameterized tests
You will often see that test methods look a lot like each other. As an example: In the fraction exercise, in most test methods you have two inputs and one or two results, then an operation is done followed by some assertion, often of the same kind. This quickly leads to the habit of copy and waste programming. Many errors are introduced this way: You copy the original, tweak the copy a bit and you are done. Then you forget one required tweak, because they are easy to miss, but you do not notice it until too late.
Warning:
Avoid copy and waste at almost all times. It is NOT a good programming style. If you see it in your code, refactor it to remove the copies, but instead make calls to one version of it. This will make you have less code overall. Coding excellence is never measured in number of code lines, but in quality of the code. Think of gems. They are precious because they are rare.
The copy and waste problem can even become worse: When the original has a flaw, you are just multiplying the
number of flaws in your code. This observation applies to test code just as well.
CODE THAT ISN’T WRITTEN CAN’T BE WRONG.
Parameterized test, Junit 5 style
Below you see an example on how you can condense the toString() tests of fraction from 5 very similar test methods into 5 strings containing the test data
and 1 (say one) test method.
Paraphrasing a saying: Killing many bugs with one test.
package fraction;
import static org.assertj.core.api.Assertions.assertThat;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.aggregator.ArgumentsAccessor;
import org.junit.jupiter.params.provider.CsvSource;
public class FractionCSVSourceTest {
@ParameterizedTest
@CsvSource( {
//"message, expected, n, d", 1
"'two thirds', '(2/3)', 2, 3", 2
"'whole number, '2', 2, 1",
"'one third', '(1/3)', -3, -9",
"'minus two fifths', '(-2/5)', 12, -30",
"'one + two fifths', '(-1-(2/5))', 35, -25"
} )
void fractionOps( String message, String expectedString,
int numerator, int denominator ) { 3
Fraction f = new Fraction( numerator, denominator );
assertThat( f.toString() )
.as( message )
.isEqualTo( expectedString );
}
}- Adding a comment is always a good idea. You may also want your values aligned for improved readability.
- Parameters are separated by commas, which maybe in the test values. In that case you can demarcate Strings with single quotes inside the csv record string. If you need another separator instead of comma, you can specify that too, see CsvSource API .
- The parameters are passed in as the type(s) provided by the test method’s signature. The Junit-5 framework will (try to) parse the csv record elements accordingly.
For more details see Junit 5 parameterized tests .
The CsvSource version of parameterized test is the simplest to use and easily understood. It keeps data and the test together, nicely near to each other, so it make it easy to read the tests and data in one glance. They typically fit easily on a screen. Remember the importance of readability of code. That too applies to test code.
Lookup in a map.
Sometimes there is no easy way to get from a string to some object, or even class.
Examples are that you want to test for the correct type or exception, or supply a method name, but you cannot put that into a string without doing complex
things like reflection which we may only do in later parts of the course. The trick is to use a Map that maps a string to the object of choice.
The lookup trick might also be applicable when you want to have short values in your test data table, like a message number which is mapped
to the actual longer expected message, or even a message that is a format string and can be used in Assertion.as( formatString, Object… args).
static Map<String, Person> emap = Map.of(
"piet", new Person("Piet", "Puk", LocalDate.of(1955-03-18),"M"),
// piet2 for equals test.
"piet2", new Person("Piet", "Puk", LocalDate.of(1955-03-18),"M"),
"rembrandt", new Person("Rembrandt", "van Rijn", LocalDate.of(1606,7,15),"M"),
"saskia", new Person("Saskia", "van Uylenburgh", LocalDate.of(1612,8,11),"F"),
);It is particularly applicable to lambda expressions. You can translate a string into a lambda, by using a map. You can then use simple names (strings), that can be put in a csv record.
final Map<String, BiFunction<Fraction, Fraction, Fraction>> ops = 1
Map.of(
"times", ( f1, f2 ) -> f1.times( f2 ),
"plus", ( f1, f2 ) -> f1.plus( f2 ),
"flip", ( f1, f2 ) -> f1.flip(), 2
"minus", ( f1, f2 ) -> f1.minus( f2 ),
"divideBy", ( f1, f2 ) -> f1.divideBy( f2 )
);- Note that we use a BiFunction<T,U,R> existing functional interface, with T, U, and R all of the same type: Fraction. This is legal.
- f2 is not used in the right hand side of the arrow. This is legal too.
@ParameterizedTest
@CsvSource(
{
"'one half times one third is 1 sixth', 'times', '(1/6)',1,2,1,3", 1
"'one thirds plus two thirds is 1' , 'plus', '1',1,3,2,3",
"'flip' , 'flip', '3',1,3,1,3", 2
"'one half minus two thirds is' , 'minus', '(-1/6)',1,2,2,3"
} )- The operation name is the second value in the csv record, here times. Note that you can quote strings, but that is not required.
- In the flip operation, the second fraction is ignored, so any legal value is allowed. Here we use the same values for both fractions.
void fractionOps( String message, String opName, String expected,
int a,int b, int c, int d ) { 1
// Arrange: read test values
Fraction f1 = frac( a, b ); 2
Fraction f2 = frac( c, d );
BiFunction<Fraction, Fraction, Fraction> op = ops.get( opName ); 3
// Act
Fraction result = op.apply( f1, f2 ); 4
// Assert(That) left out as exercise.
// Use assertThat on the fraction object result
// and check if it has the correct string value.
// Use the message in the as(...) method.
}- The fraction parameters a,b,c, and d are captured from the csvrecord. This makes the parameter list a tad longer, but also more understandable. JUnit 5 csvsource uses the annotation and the signature of the test method and can deal with most common types such as primitive, String and LocalDate (preferably in ISO-8601 format such as '2021-01-14' for the day of writing this). Strings in the csv-records will be automatically converted to the parameter types in your test methods in these cases.
- The fraction instances are created from a, b, c, and d.
- The operation (op) is looked up in the map.
- Apply the operation, or rather the function and capture the result.
You can apply the same trick of looking up with enums too, even easier, because the enum itself can translate from String to value, as long as the spelling is exact that is minding upper and lower case usage.
Study the examples above, they might give you inspiration with the exercises coming up and will score you points during the exam.
Test data from a file
Sometimes the amount of data you have in your test data-set is so big that it does not comfortably fit inside a @CsvSoure annotation.
You specify an external file as data source with to achieve the same, the annotation now is @CsvFileSource, which takes files as argument.
The file, as you might have guessed, can be a csv file, which can be edited quite comfortably with a NetBeans plugin or with the help of a spreadsheet program like Libreoffice calc or Microsoft excel.
Suppose you need to develop an input validator that has many test cases. Putting the inputs in a file along with other information relevant to your validator.
@ParameterizedTest
@CsvFileSource( resources = { "testdata.csv" } )
void testRegex( String message, String input,
boolean matches, int groupCount ){
// your code here
}Repeated use of same data.
In some cases, multiple tests need the same data. In the case of a CsvSourceFile, that is easily covered: Simple copy the annotations to all places where you need them. This is an acceptable form of copy and waste, because the annotations all point to one source of truth, the CSV file.
Sometimes you would like to keep or even generate the test data inside the test class. Do not take the simple and naive route to simply copy-and-waste the (largish) cvssource data, but instead stick to the D.R.Y. rule.
One problem is that a method in Java can return only one result, either object or primitive type. Luckily an array is also an object in Java.
There are two solutions to this issue.
- Create special test data objects of a special test data class, either inside your or outside your test class
easy to make a test data class to carry the test data. In this case make a data providing method and use the method name in the
@MethodSourceannotation. The test method should understand the test data object. - Use the JUnit5 provided
ArgumentsProvider. This wraps an array of objects into one object, so that all can be returned as one (array) value in a stream.
This saves the implementation of a (simple) test data class.
Have a look at the JUnit5 documentation to find a further explanation and examples.
Because we are collecting test tricks, here is another one:
Test Recipe I, Test Equals and hashCode
We may sprinkle our testing stuff with a few recipes for often occurring tests. This is the the first installment.
Equals and hashCode are not twins in the direct sense, but indeed methods whose implementation should have a very direct connection. From the java Object API follows that:
- Two objects that are equal by their equal method, than their hashCode should also be equal.
- Note that the reverse is not true. If two hashCode are the same, that does not imply that the objects are equal.
- A good hashCode 'spreads' the objects well, but this is not a very strict requirement or a requirement that can be enforced. A poor hashCode will lead to poor Hash{Map|Set} lookup performance.
Although hashCodes are invented to speedup finding object in a hash map or hash set, these collections use hashCode in the first part of the search, but must verify equality as final step(s).
The thing is that the equals method must consider quite a few things, expressed with conditional evaluation (if-then-else).
The good thing is an IDE typically provides a way to generate equals and hashCode for you and these implementations are typically of good quality. But in particular in the equals method there are quite some ifs, sometimes in disguise, coded as &&, so this will throw some flies in your code-coverage ointment.
However, we can balance this generated code by a piece of reusable test code, that can be used for almost all cases.
In fact we have not found a case where it does not apply.
Let us first explain the usage and then the implementation.
Suppose you have an object with three fields, name, birthDate and id. All these fields should be considered in equals and hashCode.
As an exercise, create such and object now in your IDE, call it Student, why not.
class Student {
final String name;
final LocalDate birthDate;
final int id;
}From the above, the IDE can generate a constructor, equals and hashCode and toString. What are you waiting for? Because it is not test driven?
You would be almost right, but why test drive something that can be generated.
However, if your spec does not demand equals and hashCode,
then do not write/generate them. That would be unwanted code. But if the requirements DO insist on equals and hashCode,
make sure that the fields to be considered match the requirements. Choose only final fields.
After having such a generated equals and hashCode you have the predicament of writing a test. HashCode is relatively easy. It should produce an integer, but what value is unspecified, so just invoking it would do the trick for test coverage. The devil is in the equals details, because it has to consider:
- Is the other object
this? If yes, returntrue. - Is the other object
null? Returnfalseif it is. - Now consider the type.[1].
- Is the other of the same type as
this? If not returnfalse. - Now we are in known terrain, the other is of the same type, so the object should have the same fields.
For eachfieldtest itthis.field.equals(other.field). If not return false. - Using
Objects.equals(this.fieldA, other.fieldA)can be an efficient solution to avoid testing for nullity of either field.
- Is the other of the same type as
@Override
public boolean equals( Object obj ) {
if ( this == obj ) {
return true;
}
if ( obj == null ) {
return false;
}
if ( getClass() != obj.getClass() ) {
return false;
}
final Student other = (Student) obj;
if ( this.id != other.id ) {
return false;
}
if ( !Objects.equals( this.name, other.name ) ) {
return false;
}
return Objects.equals( this.birthDate, other.birthDate );
}You see a pattern here: The number of ifs is 3 + the number of fields.
To test this, and to make sure you hit all code paths, you need to test with this,
with null, with an distinct (read newly constructed) object with all fields equal,
and then one for each field, which differs from the reference object only in said field.
Define those instances (for this example) as follows.
//@Disabled
@Test
void testEqualsAndHash() {
Student ref = new Student( "Jan", LocalDate.of( 1999, 03, 23 ), 123 ); 1
Student equ = new Student( "Jan", LocalDate.of( 1999, 03, 23 ), 123 ); 2
Student sna = new Student( "Jen", LocalDate.of( 1999, 03, 23 ), 123 ); 3
Student sda = new Student( "Jan", LocalDate.of( 1998, 03, 23 ), 123 ); 4
Student sid = new Student( "Jan", LocalDate.of( 1999, 03, 23 ), 456 ); 5
verifyEqualsAndHashCode( ref, equ, sna, sda, sid );
//fail( "testMethod reached it's and. You will know what to do." );
}- The reference object
- The equal object, equals true
- Differs in name.
- Differs in birth date (year).
- Differs in id.
The implementation of the verifyEqualsAndHashCode has been done with generics and a dash of AssertJ stuff.
/**
* Helper for equals tests, which are tedious to get completely covered.
*
* @param <T> type of class to test
* @param ref reference value
* @param equal one that should test equals true
* @param unEqual list of elements that should test unequal in all cases.
*/
public static <T> void verifyEqualsAndHashCode( T ref, T equal, T... unEqual ) {
Object object = "Hello";
T tnull = null;
String cname = ref.getClass().getCanonicalName();
// I got bitten here, assertJ equalTo does not invoke equals on the
// object when ref and 'other' are same.
// THAT's why the first ones differ from the rest.
SoftAssertions.assertSoftly( softly-> {
softly.assertThat( ref.equals( ref ) )
.as( cname + ".equals(this): with self should produce true" )
.isTrue();
softly.assertThat( ref.equals( tnull ) )
.as( cname + ".equals(null): ref object "
+ safeToString( ref ) + " and null should produce false"
)
.isFalse();
softly.assertThat( ref.equals( object ) )
.as( cname + ".equals(new Object()): ref object"
+ " compared to other type should produce false"
)
.isFalse();
softly.assertThat( ref.equals( equal ) )
.as( cname + " ref object [" + safeToString( ref )
+ "] and equal object [" + safeToString( equal )
+ "] should report equal"
)
.isTrue();
for ( int i = 0; i < unEqual.length; i++ ) {
T ueq = unEqual[ i ];
softly.assertThat( ref )
.as("testing supposed unequal objects")
.isNotEqualTo( ueq );
}
// ref and equal should have same hashCode
softly.assertThat( ref.hashCode() )
.as( cname + " equal objects "
+ ref.toString() + " and "
+ equal.toString() + " should have same hashcode"
)
.isEqualTo( equal.hashCode() );
});
}The above code has been used before but now adapted for AssertJ and JUnit 5.
It is of course best to put this in some kind of test helper library, so you can reuse it over and over without having to resort to copy and waste.
Links
- Not all equals implementation look at the type of this, See the java.util.List doc for a counter example ↩