Write your own tests!

Throughout the exercises of PRC1, you have become acquainted with the value of tests: You have a way of checking if your code is any good without having to test each and every part manually. You clicked on the nice TestMyCode (TMC) button in Visual Studio Code (VSC) and then you could see which parts of your code worked, and which didn’t. There is a catch though: Out there in the real world, there won’t be any button doing that magic for you, nor will some teacher or school provide the tests for you, so you will be on your own.

But fret not! Writing tests is typically much simpler than writing the actual code. At least it is, when you know the anatomy of a test:

Finally, if a test says boom, then at least you learned something …​

Topics week 1

Testing / Test Driven Development

What are tests and why do we need them?

The way that you have worked with Java so far is that you had to write some code to implement something. For example you had to implement a bird watcher system or a telephone book, in which you could save people with their telephone numbers and look them up, too. You probably created classes called PhoneBook or BookEntry, and you had variables such as String personName, int phoneNumber and maybe a List or a Map which contained all the people and their phone numbers. You had to think about how the phonebook works, and then you added classes, fields (variables) and methods. Slowly but surely, your phonebook started to look more and more like a phonebook.

But how did you know that your phonebook was actually working? You could have implemented a method that didn’t work, or maybe you forgot to add people to the phonebook, so the data wasn’t actually stored. What you did was you ran the TMC tests. They kept telling you whether the phonebook was working as intended. The test said "when I look for Alice in your phonebook, I will receive Alice’s phone number 1234-567.". And if the test didn’t receive Alice’s number, it would turn red and say something like "I expected 1234-567. But I received 0042-987." Therefore, the test fails.

The tests were a really useful way for you to know that you were indeed implementing a proper phonebook. You could have just written empty classes and said "here is the phonebook!" and there would have been no way to verify that the phonebook works. The test made sure that this doesn’t happen. They also give you confidence as you code. Because every test that turns green gives you that little "yes!" feeling that you wrote something that works.

Thus, tests fulfill two functions at once: they constantly probe your implementation and check whether it works, and they make you feel more confident about your programming skills. That’s really neat!

When you were at school, you probably had to write an English exam at some point. You wrote your answers, and then the teacher graded your exam. The teacher is just like a test: she reads your exam, expecting the correct answer. For example, you wrote "Alice and Bob goes to school.". Your teacher would get the red pen and highlight the word "goes". The teacher says "I expected: Alice and Bob go to school. Instead, I received: Alice and Bob goes to school." The teachers expectation, or assertion, was not met. Therefore, you get an error. As you grow older and you become more proficient at English, you write Emails in English, perhaps for work or at university. When you make a mistake, you spot your own errors: "oh hang on, I have to use 'go' instead of 'goes' here." You have internalized a test that checks for the correct conjugation of the word. You know what to expect, and when you deviate from the expectation, you spot the error.

Many people work the same way when they write code. They say "I know what I am doing because I have experience and I know the rules." But of course, we always make mistakes. Our brains are really bad testers. We stop seeing mistakes because we feel so familiar with our code. Have you ever handed in a report for university, only to find that the lecturer finds spelling errors you swear you didn’t do? That’s what happens in programming, too.

That is why we write our own little annoying English teachers that constantly check: is this correct? Even though we know how to program, we also acknowledge that our brains are terrible testers for things that you write yourself, so we give our brains a break and write a test instead.

Test Driven Development (TDD)

In Week 9 we wrote a little phonebook and then we ran the test to check that the phonebook was working. That worked well, because we no longer have to rely on our brains to spot the errors on-the-fly. But here is another bad message: not only does our brain stop recognizing errors in our own code, but it is also automatically biased to want our own tests to pass. You spent all this time writing your phonebook, you know how it works, so you know what test to write for it.

But your brain secretly deceives you: you have implemented a phonebook, and this is the only phonebook that you know. What if you removed your code, but leave in the tests- and ask a friend to write a phonebook? They start complaining that your tests are unfair. "Why do I have to use a List? I use Maps!" your friend says. But your tests insists that the phonebook has a List. So what have you done? You have written a test that proves YOUR implementation is correct.

When you work test-driven, you don’t implement the phonebook and then test it. You first write the test for a phonebook, and then you implement it. That sounds a bit weird at first. Why write tests for stuff that’s not there yet? We know it’s going to fail! But this is what we want. We need to keep telling our brains: this does not work. Figure out a way to make it work. This way, it is much harder to get married to your own code and to stop seeing problems with your test. You have found a way to deal with the human brain. Congratulations!

So how do I know what to test?

There is a little (actually, it’s big) catch with TDD. When you write a test for a thing that doesn’t exist yet, how do you know what to test for? If I write a test for a phonebook that is not implemented yet, what does my test expect?

The truth is, you never get around having to make implementation decisions. We just try to minimize our margin for error. So when you start writing your first test for your (non-existing) phonebook, you HAVE to have an idea of what the phonebook does. So perhaps you start with "I expect that the phonebook lets me look up a person. When I look up Alice, I expect Alice’s phone number.". What could such a test look like?

Test selection strategies

There is no golden rule that says "always start with text X, and then test Y, and finish with test Z.". Instead, we rely on heuristics: rules of thumb that guide us through the TDD process. There are several decisions that you can make when writing tests.

For instance, you can decide to first write tests for all elements of the system under test (SUT), but not go into detail yet what each SUT needs to have tested. This would mean focusing on width first, and depth later. For instance, you could write tests for the entirety of the phonebook, but you don’t, for instance, test each valid or invalid phonebook entry. The other approach would be to start with the details on one particular part of the test class, and moving on to other parts only when one section of the SUT is covered completely. In this version, you go depth-first instead of width-first. Both approaches have their merit and you need to decide which strategy is best in which situation.

Another strategy is to weigh the difficulty of each test. When you look at a list of tests to write, do you start with the easy ones that are implemented quickly, or do you start with the more difficult tests that require more implementation thinking? When you are stuck on a particular test, it might be useful to first implement a number of tests that are easier to write, so that you make progress. It might be that whatever was causing you to be stuck on that test is now easier to solve now that you have a list of other tests that you have implemented.

Where to start

Choosing is always hard, including choosing where to start. The typical order for test driven development is:

1. A Constructor. You typically need instances and this is how you can construct some.
2. Getters for the fields that are defined in the constructor. If such getters are not part of the public API of the class, make the package private, to hide them from client classes outside of the package.
3. The public String toString() method, which can show the result of the constructor. Note that toString is often used for debugging, and that is what the auto-generated to string typically is for.
4. Setters, if needed. (They are NOT in the fraction exercise). If you can get by with a class without setters, you are typically better off.
5. The business methods, one by one. In case of Fraction exercise, start at multiplication, because that is easiest.
6. Further refinements, like in case of the Fraction exercise, that the fraction should always be in normalised form.

---

Arrange Act Assert

When designing tests it is good to keep the ideas of making action movies in the back of your head. Looking at some the making of…​ can be quite instructive in this case. For a scene to be taken, there is typically quite some preparation involved like hanging Spiderman on a thin cable from the ceiling, making sure the crook is in the right position so he can land or receive the first punches and so on.
Then the director calls action and the scene is acted out
If the director is satisfied, all is okay
If not, he will want a redo later on.

A very popular style to write unit tests is the triple-A or Arrange-Act-Assert.

Arrange simply means that you set up the objects you need in the tests:

Act is quite similar to a film directory shouting action, that is do what you prepared for.

Assert to test if the actual result of the action is what is expected.

@Test
public void shouldAllowToAddAddress() {
  // Arrange
  Client client = new Client();
  Address addressA = new Address(221b, "Baker Street");

  // Act
  client.addAddress(addressA);

  // Assert
  assertThat( client.getAddresses() ).contains( addressA ));
}

Of course, Arrange, Act and Assert will vary with the actual business requirement, much as the setup, play, and act in movie making. And also, quite similar to movie making the arrange can be expensive or cheap, the action be fraught with risk or easy, and the assert can be complex or simple.

We hope to teach to keep all three as simple as possible, to keep the costs of testing so low that to make it never an argument to skip testing.

Later in this course we see some more complex scenarios, in which we need Extras as one would say in the movie industry.

Clues needed

A smart person is able to avoid work. A common strategy is to let others do it. You can achieve such smartness your selves by using your tools properly.

For instance your IDE will gladly complete code that your started by showing a list of possible completions of the code that you already provided. However, the IDE can only do that if you provide it with sufficient clues. For instance when you want the IDE to generate a non-existing generic class, create two instance with different type parameters before your accept the hint to create class.

Box<String> sb= new Box<String>(); // compiler complains about nonexisting Box class.
Box<Integer> ib= new Box<Integer>(); // compiler complains about  nonexisting Box class.
// now you can let the IDE to follow its urge to do the typing for you.

The same applies to methods. If you want a specific return type, provide that in a declaration where you specify such type.

It also applies to frameworks such as AssertJ, that provides an assertThat(…​) method as entry for almost all asserts. But what you put on the dots is important, because the IDE will inspect the type of the expression and then will look for the possible completions. The completions for an Integer or int are very different from those for a Collection or Iterable.

Combine this with the previous tip. First you specify a variable with a type, that uses the variable inside the assertThat(…​) and then the IDE will be able to find the possible completions.

Sometimes you have to nudge the IDE a bit, for instance when it suggest to create two constructors from the above example. Accept one of the suggestions, but then modify the generated constructor to accept the type parameter of the class as input type of the constructor. After saving all files, you will have convinced both IDE and compiler.

AssertJ examples.

In our course we to stick to AssertJ library where possible.

The rationale for that is:

We try to use the latest stable version of AssertJ and Junit 5.

  car.start();
  assertThat( car.motorIsOn() ) 1
      .as("Motor should be on after start") 2
      .isTrue(); 3

  assertThat(car.speedLimit())
      .as("Make it a legal car?")
      .isEqualTo(250);

    assertThat( f1Champ ).isEqualTo( louis );
    assertThat( f1Champ ).isSameAs( louis );

  Student student = ....

  assertThat( student.toString() )
      .isNotNull()                              1
      .doesNotContainIgnoringCase("student{")    2
      .contains( "Harry", "Potter", "1980-07-31", "12345"); 3

    List<Professor> professors = List.of( SNAPE, DUMBLEDORE, MCGONAGALL ); 1

    List<Professor> teachesTransfiguration = professors.stream()
            .filter( p -> p.teaches( "Transfigurations" ) ).collect( toList() );

    assertThat( teachesTransfiguration )
            .isNotNull()                         2
            .hasSizeGreaterThanOrEqualTo( 2 )    3
            .contains( DUMBLEDORE, MCGONAGALL ); 4

//  List<Student> hogwartsStudents = ...; 1
  List<Student> hogwartsStudents = List.of( HARRY, HERMIONE, RON ); 1

  assertThat( hogwartsStudents )
          .isNotNull()                                2
          .hasSizeGreaterThanOrEqualTo( 3 )           3
          .extracting( s -> s.getStudentNumber() )    4
          .containsExactlyInAnyOrder( 12346, 12345, 12347 ); 5

    @Test
    public void fileUsingMethod() throws IOException { 1
        Files.lines(Path.of ("puk.txt") );  2
    }

    Student draco = new Student("Draco", "Malfoy", LocalDate.of (1980,6,5));
    ThrowingCallable code = () -> {
        hogwarts.addStudent( draco );  1
    };

    assertThatCode( code)
            .as( "draco should be accepted to make the adventures possible")
            .doesNotThrowAnyException();

    @Test
    public void addIllegalProfessor() {
        var malfoy = new Professor( "Lucius", "Malfoy", LocalDate.of( 1953, 10, 9 ) ); 1

        ThrowingCallable code = () -> { 2
            hogwarts.addProfessor( malfoy );
        };

        assertThatThrownBy( code )
                .isInstanceOf( Exception.class) 3
                .isExactlyInstanceOf( IllegalArgumentException.class) 4
                .hasMessageContainingAll( "should","teach"); 5
        // fail( "addIllegalProfessor completed successfully; you know what to do" );
    }

Soft Assertions

Sometimes you have a pressing need to assert more than one thing in a test method, because two or more values always come in pairs and setting stuff up and dividing it over multiple methods would make the test less readable, which is bad.

However, having multiple asserts in one method is bad style, because the first problem that occurs is the only failure that will be reported by the test. You could say that the first failure dominates or monopolizes the test method.

However the rule above makes writing tests a bit awkward. In particular when there are multiple aspects to be tested on the same result object. In that case a soft assertion may be helpful. A soft assertion will fail "softly", meaning that the failure is recorded, but the test can still continue to verify another aspect. The soft assertion will collect all failures and will report them after the soft assertion is closed.

   void testGetResult(){
    // some setup
    Optional<KeyValue<Integer,Double>> = new gf(student).gradeFor("PRC2");
    SoftAssertions.assertSoftly( softly -> { 1
                AbstractMap.SimpleEntry<Integer, Double> result = gf.getResult();
                softly.assertThat( result )
                    .extracting( "key" )
                    .isEqualTo( student.getStudentId() );
                softly.assertThat( result )
                    .extracting( "value" )
                    .isEqualTo( grade );
    }); 2
  }

In the example, the key and value in the KeyValue pair are related, making a soft assertion applicable.

Assumptions

There are test cases that are only useful under certain conditions:

For this you can use Assume.

There are two variants relevant:

You can use the Assumption test in the setup methods (the one with @BeforeXXX), to completely disable all tests in one test class, instead of having each test doing the assumeXXX test. A test that has been canceled because of a failing assumption will appear as a skipped test, similar to a disabled test with @Disabled.

The JUnit 5 Assumptions are effective, but can only test for true or false, so you need to feed it a boolean expression. Look at the API to see all features. But simple may be just perfect here.

    @Test
    void testOnlyOnDeveloperWorkstation() {
        assumeTrue("DEV".equals(System.getenv("ENV")), "Aborting test: not on developer workstation");
        // remainder of test is only executed if on developer machine
    }

You guessed it, AssertJ Assumptions are richer. In most cases, the assumption has the same form as an assert that, as in replace assert with assume and you are done. So any assertThat(…​) that you already have can simply be turned into an assumeThat(…​).

If nothing else, using assertj may make your tests more readable and have a consistent look and feel.

    File f = new File(Path.of("testData.dat"));
    assumeThat(f)
       .as("Test data file not available, skipping test")
       .exists();

Additional pointers

Clean tests

Bad tests are anything but a pleasure to work with. They are slow, hard to understand, and hard to maintain. In order to write clean tests, we follow the F.I.R.S.T. principles of tests:

Parameterized tests

You will often see that test methods look a lot like each other. As an example: In the fraction exercise, in most test methods you have two inputs and one or two results, then an operation is done followed by some assertion, often of the same kind. This quickly leads to the habit of copy and waste programming. Many errors are introduced this way: You copy the original, tweak the copy a bit and you are done. Then you forget one required tweak, because they are easy to miss, but you do not notice it until too late.

Warning:

Avoid copy and waste at almost all times. It is NOT a good programming style. If you see it in your code, refactor it to remove the copies, but instead make calls to one version of it. This will make you have less code overall. Coding excellence is never measured in number of code lines, but in quality of the code. Think of gems. They are precious because they are rare.

The copy and waste problem can even become worse: When the original has a flaw, you are just multiplying the number of flaws in your code. This observation applies to test code just as well.

CODE THAT ISN’T WRITTEN CAN’T BE WRONG.

package fraction;

import static org.assertj.core.api.Assertions.assertThat;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.aggregator.ArgumentsAccessor;
import org.junit.jupiter.params.provider.CsvSource;

public class FractionCSVSourceTest {

    @ParameterizedTest
    @CsvSource( {
           //"message,          expected,     n,   d", 1
           "'two thirds',        '(2/3)',     2,   3", 2
           "'whole number,           '2',     2,   1",
           "'one third',         '(1/3)',    -3,  -9",
           "'minus two fifths',  '(-2/5)',   12, -30",
           "'one + two fifths', '(-1-(2/5))',   35, -25"
    } )
    void fractionOps( String message, String expectedString,
            int numerator, int denominator ) { 3

        Fraction f = new Fraction( numerator, denominator );
        assertThat( f.toString() )
                .as( message )
                .isEqualTo( expectedString );
    }
}

static Map<String, Person> emap = Map.of(
    "piet", new Person("Piet", "Puk", LocalDate.of(1955-03-18),"M"),
    // piet2 for equals test.
    "piet2", new Person("Piet", "Puk", LocalDate.of(1955-03-18),"M"),
    "rembrandt", new Person("Rembrandt", "van Rijn", LocalDate.of(1606,7,15),"M"),
    "saskia", new Person("Saskia", "van Uylenburgh", LocalDate.of(1612,8,11),"F"),
  );

    final Map<String, BiFunction<Fraction, Fraction, Fraction>> ops = 1
      Map.of(
              "times", ( f1, f2 ) -> f1.times( f2 ),
              "plus", ( f1, f2 ) -> f1.plus( f2 ),
              "flip", ( f1, f2 ) -> f1.flip(), 2
              "minus", ( f1, f2 ) -> f1.minus( f2 ),
              "divideBy", ( f1, f2 ) -> f1.divideBy( f2 )
      );

@ParameterizedTest
@CsvSource(
    {
    "'one half times one third is 1 sixth', 'times', '(1/6)',1,2,1,3", 1
    "'one thirds plus two thirds is 1'    , 'plus',      '1',1,3,2,3",
    "'flip'                               , 'flip',      '3',1,3,1,3", 2
    "'one half minus two thirds is'       , 'minus', '(-1/6)',1,2,2,3"
  } )

void fractionOps( String message, String opName, String expected,
              int a,int b, int c, int d ) { 1
    // Arrange: read test values
    Fraction f1 = frac( a, b ); 2
    Fraction f2 = frac( c, d );
    BiFunction<Fraction, Fraction, Fraction> op = ops.get( opName ); 3

    // Act
    Fraction result = op.apply( f1, f2 ); 4

    // Assert(That) left out as exercise.
    // Use assertThat on the fraction object result
    //   and check if it has the correct string value.
    // Use the message in the as(...) method.

}

  @ParameterizedTest
  @CsvFileSource( resources = { "testdata.csv" } )
  void testRegex( String message, String input,
          boolean matches, int groupCount ){
    // your code here
  }

Test Recipe I, Test Equals and hashCode

We may sprinkle our testing stuff with a few recipes for often occurring tests. This is the the first installment.

Equals and hashCode are not twins in the direct sense, but indeed methods whose implementation should have a very direct connection. From the java Object API follows that:

Although hashCodes are invented to speedup finding object in a hash map or hash set, these collections use hashCode in the first part of the search, but must verify equality as final step(s).

The thing is that the equals method must consider quite a few things, expressed with conditional evaluation (if-then-else). The good thing is an IDE typically provides a way to generate equals and hashCode for you and these implementations are typically of good quality. But in particular in the equals method there are quite some ifs, sometimes in disguise, coded as &&, so this will throw some flies in your code-coverage ointment.

However, we can balance this generated code by a piece of reusable test code, that can be used for almost all cases. In fact we have not found a case where it does not apply.
Let us first explain the usage and then the implementation.
Suppose you have an object with three fields, name, birthDate and id. All these fields should be considered in equals and hashCode.
As an exercise, create such and object now in your IDE, call it Student, why not.

class Student {
  final String name;
  final LocalDate birthDate;
  final int id;
}

From the above, the IDE can generate a constructor, equals and hashCode and toString. What are you waiting for? Because it is not test driven? You would be almost right, but why test drive something that can be generated.
However, if your spec does not demand equals and hashCode, then do not write/generate them. That would be unwanted code. But if the requirements DO insist on equals and hashCode, make sure that the fields to be considered match the requirements. Choose only final fields.

After having such a generated equals and hashCode you have the predicament of writing a test. HashCode is relatively easy. It should produce an integer, but what value is unspecified, so just invoking it would do the trick for test coverage. The devil is in the equals details, because it has to consider:

@Override
public boolean equals( Object obj ) {
    if ( this == obj ) {
        return true;
    }
    if ( obj == null ) {
        return false;
    }
    if ( getClass() != obj.getClass() ) {
        return false;
    }
    final Student other = (Student) obj;
    if ( this.id != other.id ) {
        return false;
    }
    if ( !Objects.equals( this.name, other.name ) ) {
        return false;
    }
    return Objects.equals( this.birthDate, other.birthDate );
}

You see a pattern here: The number of ifs is 3 + the number of fields.
To test this, and to make sure you hit all code paths, you need to test with this, with null, with an distinct (read newly constructed) object with all fields equal, and then one for each field, which differs from the reference object only in said field.

Define those instances (for this example) as follows.

//@Disabled
@Test
void testEqualsAndHash() {
    Student ref = new Student( "Jan", LocalDate.of( 1999, 03, 23 ), 123 ); 1
    Student equ = new Student( "Jan", LocalDate.of( 1999, 03, 23 ), 123 ); 2
    Student sna = new Student( "Jen", LocalDate.of( 1999, 03, 23 ), 123 ); 3
    Student sda = new Student( "Jan", LocalDate.of( 1998, 03, 23 ), 123 ); 4
    Student sid = new Student( "Jan", LocalDate.of( 1999, 03, 23 ), 456 ); 5
    verifyEqualsAndHashCode( ref, equ, sna, sda, sid );
    //fail( "testMethod reached it's and. You will know what to do." );
}

The implementation of the verifyEqualsAndHashCode has been done with generics and a dash of AssertJ stuff.

/**
 * Helper for equals tests, which are tedious to get completely covered.
 *
 * @param <T> type of class to test
 * @param ref reference value
 * @param equal one that should test equals true
 * @param unEqual list of elements that should test unequal in all cases.
 */
public static <T> void verifyEqualsAndHashCode( T ref, T equal, T... unEqual ) {
    Object object = "Hello";
    T tnull = null;
    String cname = ref.getClass().getCanonicalName();
    // I got bitten here, assertJ equalTo does not invoke equals on the
    // object when ref and 'other' are same.
    // THAT's why the first ones differ from the rest.
    SoftAssertions.assertSoftly( softly-> {
    softly.assertThat( ref.equals( ref ) )
            .as( cname + ".equals(this): with self should produce true" )
            .isTrue();
    softly.assertThat( ref.equals( tnull ) )
            .as( cname + ".equals(null): ref object "
                    + safeToString( ref ) + " and null should produce false"
            )
            .isFalse();
    softly.assertThat( ref.equals( object ) )
            .as( cname + ".equals(new Object()): ref object"
                    + " compared to other type should produce false"
            )
            .isFalse();
    softly.assertThat( ref.equals( equal ) )
            .as( cname + " ref object [" + safeToString( ref )
                    + "] and equal object [" + safeToString( equal )
                    + "] should report equal"
            )
            .isTrue();
    for ( int i = 0; i < unEqual.length; i++ ) {
        T ueq = unEqual[ i ];
        softly.assertThat( ref )
                .as("testing supposed unequal objects")
        .isNotEqualTo( ueq );
    }
    // ref and equal should have same hashCode
    softly.assertThat( ref.hashCode() )
            .as( cname + " equal objects "
                    + ref.toString() + " and "
                    + equal.toString() + " should have same hashcode"
            )
            .isEqualTo( equal.hashCode() );
});
}

The above code has been used before but now adapted for AssertJ and JUnit 5.

It is of course best to put this in some kind of test helper library, so you can reuse it over and over without having to resort to copy and waste.

Links