PRC2

Write your own tests!

Throughout the exercises of PRC1, you have become acquainted with the value of tests: You have a way of checking if your code is any good without having to test each and every part manually. You clicked on the nice TMC button in NetBeans and then you could see which parts of your code worked, and which didn’t. There is a catch though: Out there in the real world, there won’t be any NetBeans button doing that magic for you, nor will some teacher or school provide the tests for you, so you will be on your own.

But fret not! Writing tests is typically much simpler than writing the actual code. At least, it is when you follow a basic set of steps:

  1. Arrange: Prepare or set up the thing you want to test
  2. Act: Interact with the object you are testing
  3. Assert or Ensure that the observable result(s) is/are as expected.
  4. If it says boom, then at least you learned something …​.

Topics week 1

  • Test Driven Development.
  • Maven configuration as exercise.
  • Arrange Act Assert
  • JUnit (5) as test framework.
  • AssertJ as assertion library.

Testing / Test Driven Development

What are tests and why do we need them?

The way that you have worked with Java so far is that you had to write some code to implement something. For example you had to implement a bird watcher system or a telephone book, in which you could save people with their telephone numbers and look them up, too. You probably created classes called PhoneBook or BookEntry, and you had variables such as String personName, int phoneNumber and maybe a List or a Map which contained all the people and their phone numbers. You had to think about how the phonebook works, and then you added classes, fields (variables) and methods. Slowly but surely, your phonebook started to look more and more like a phonebook.

But how did you know that your phonebook was actually working? You could have implemented a method that didn’t work, or maybe you forgot to add people to the phonebook, so the data wasn’t actually stored. What you did was you ran the TMC tests. They kept telling you whether the phonebook was working as intended. The test said "when I look for Alice in your phonebook, I will receive Alice’s phone number 1234-567.". And if the test didn’t receive Alice’s number, it would turn red and say something like "I expected 1234-567. But I received 0042-987." Therefore, the test fails.

The tests were a really useful way for you to know that you were indeed implementing a proper phonebook. You could have just written empty classes and said "here is the phonebook!" and there would have been no way to verify that the phonebook works. The test made sure that this doesn’t happen. They also give you confidence as you code. Because every test that turns green gives you that little "yes!" feeling that you wrote something that works.

Thus, tests fulfill two functions at once: they constantly probe your implementation and check whether it works, and they make you feel more confident about your programming skills. That’s really neat!

When you were at school, you probably had to write an English exam at some point. You wrote your answers, and then the teacher graded your exam. The teacher is just like a test: she reads your exam, expecting the correct answer. For example, you wrote "Alice and Bob goes to school.". Your teacher would get the red pen and highlight the word "goes". The teacher says "I expected: Alice and Bob go to school. Instead, I received: Alice and Bob goes to school." The teachers expectation, or assertion, was not met. Therefore, you get an error. As you grow older and you become more proficient at English, you write Emails in English, perhaps for work or at university. When you make a mistake, you spot your own errors: "oh hang on, I have to use 'go' instead of 'goes' here." You have internalized a test that checks for the correct conjugation of the word. You know what to expect, and when you deviate from the expectation, you spot the error.

Many people work the same way when they write code. They say "I know what I am doing because I have experience and I know the rules." But of course, we always make mistakes. Our brains are really bad testers. We stop seeing mistakes because we feel so familiar with our code. Have you ever handed in a report for university, only to find that the lecturer finds spelling errors you swear you didn’t do? That’s what happens in programming, too.

That is why we write our own little annoying English teachers that constantly check: is this correct? Even though we know how to program, we also acknowledge that our brains are terrible testers for things that you write yourself, so we give our brains a break and write a test instead.

Test Driven Development (TDD)

In Week 9 we wrote a little phonebook and then we ran the test to check that the phonebook was working. That worked well, because we no longer have to rely on our brains to spot the errors on-the-fly. But here is another bad message: not only does our brain stop recognizing errors in our own code, but it is also automatically biased to want our own tests to pass. You spent all this time writing your phonebook, you know how it works, so you know what test to write for it.

But your brain secretly deceives you: you have implemented a phonebook, and this is the only phonebook that you know. What if you removed your code, but leave in the tests- and ask a friend to write a phonebook? They start complaining that your tests are unfair. "Why do I have to use a List? I use Maps!" your friend says. But your tests insists that the phonebook has a List. So what have you done? You have written a test that proves YOUR implementation is correct.

When you work test-driven, you don’t implement the phonebook and then test it. You first write the test for a phonebook, and then you implement it. That sounds a bit weird at first. Why write tests for stuff that’s not there yet? We know it’s going to fail! But this is what we want. We need to keep telling our brains: this does not work. Figure out a way to make it work. This way, it is much harder to get married to your own code and to stop seeing problems with your test. You have found a way to deal with the human brain. Congratulations!

tdd lifecycle
Figure 4. The TDD Lifecycle. Click for image source.

So how do I know what to test?

There is a little (actually, it’s big) catch with TDD. When you write a test for a thing that doesn’t exist yet, how do you know what to test for? If I write a test for a phonebook that is not implemented yet, what does my test expect?

The truth is, you never get around having to make implementation decisions. We just try to minimize our margin for error. So when you start writing your first test for your (non-existing) phonebook, you HAVE to have an idea of what the phonebook does. So perhaps you start with "I expect that the phonebook lets me look up a person. When I look up Alice, I expect Alice’s phone number.". What could such a test look like?

Test selection strategies

There is no golden rule that says "always start with text X, and then test Y, and finish with test Z.". Instead, we rely on heuristics: rules of thumb that guide us through the TDD process. There are several decisions that you can make when writing tests.

For instance, you can decide to first write tests for all elements of the system under test (SUT), but not go into detail yet what each SUT needs to have tested. This would mean focusing on width first, and depth later. For instance, you could write tests for the entirety of the phonebook, but you don’t, for instance, test each valid or invalid phonebook entry. The other approach would be to start with the details on one particular part of the test class, and moving on to other parts only when one section of the SUT is covered completely. In this version, you go depth-first instead of width-first. Both approaches have their merit and you need to decide which strategy is best in which situation.

Another strategy is to weigh the difficulty of each test. When you look at a list of tests to write, do you start with the easy ones that are implemented quickly, or do you start with the more difficult tests that require more implementation thinking? When you are stuck on a particular test, it might be useful to first implement a number of tests that are easier to write, so that you make progress. It might be that whatever was causing you to be stuck on that test is now easier to solve now that you have a list of other tests that you have implemented.

Where to start

Choosing is always hard, including choosing where to start. The typical order for test driven development is:

  1. A Constructor. You typically need instances and this is how you can construct some.
  2. Getters for the fields that are defined in the constructor. If such getters are not part of the public API of the class, make the package private, to hide them from client classes outside of the package.
  3. The public String toString() method, which can show the result of the constructor. Note that toString is often used for debugging, and that is what the auto-generated to string typically is for.
  4. Setters, if needed. (They are NOT in the fraction exercise). If you can get by with a class without setters, you are typically better off.
  5. The business methods, one by one. In case of Fraction exercise, start at multiplication, because that is easiest.
  6. Further refinements, like in case of the Fraction exercise, that the fraction should always be in normalised form.

Arrange Act Assert

Some action.

When designing tests it is good to keep the ideas of making action movies in the back of your head. Looking at some the making of…​ can be quite instructive in this case. For a scene to be taken, there is typically quite some preparation involved like hanging Spiderman on a thin cable from the ceiling, making sure the crook is in the right position so he can land or receive the first punches and so on.
Then the director calls action and the scene is acted out
If the director is satisfied, all is okay
If not, he will want a redo later on.

A very popular style to write unit tests is the triple-A or Arrange-Act-Assert.

Arrange simply means that you set up the objects you need in the tests:

  • The System Under Test, the SUT its the term in the tesing world. In movies it is the protagonist or main actor.
  • The Dependent On Components or DOCs, the scenario needs to play out the action. Supporting actors or even props in the movies.

Act is quite similar to a film directory shouting action, that is do what you prepared for.

  • This typically is one method call on the SUT.

Assert to test if the actual result of the action is what is expected.

  • The actual value can be the return value of the called method, but also the (new) state of the SUT.
    In the movie: did the explosion happen, was the punch landed correctly and was the emotion credible.
AAA in action
@Test
public void shouldAllowToAddAddress() {
  // Arrange
  Client client = new Client();
  Address addressA = new Address(221b, "Baker Street");

  // Act
  client.addAddress(addressA);

  // Assert
  assertThat( client.getAddresses() ).contains( addressA ));
}

Of course, Arrange, Act and Assert will vary with the actual business requirement, much as the setup, play, and act in movie making. And also, quite similar to movie making the arrange can be expensive or cheap, the action be fraught with risk or easy, and the assert can be complex or simple.

We hope to teach to keep all three as simple as possible, to keep the costs of testing so low that to make it never an argument to skip testing.

Later in this course we see some more complex scenarios, in which we need Extras as one would say in the movie industry.

Clues needed

A smart person is able to avoid work. A common strategy is to let others do it. You can achieve such smartness your selves by using your tools properly.

For instance your IDE will gladly complete code that your started by showing a list of possible completions of the code that you already provided. However, the IDE can only do that if you provide it with sufficient clues. For instance when you want the IDE to generate a non-existing generic class, create two instance with different type parameters before your accept the hint to create class.

complete a generic
Box<String> sb= new Box<String>(); // compiler complains about nonexisting Box class.
Box<Integer> ib= new Box<Integer>(); // compiler complains about  nonexisting Box class.
// now you can let the IDE to follow its urge to do the typing for you.

The same applies to methods. If you want a specific return type, provide that in a declaration where you specify such type.

It also applies to frameworks such as AssertJ, that provides an assertThat(…​) method as entry for almost all asserts. But what you put on the dots is important, because the IDE will inspect the type of the expression and then will look for the possible completions. The completions for an Integer or int are very different from those for a Collection or Iterable.

Combine this with the previous tip. First you specify a variable with a type, that uses the variable inside the assertThat(…​) and then the IDE will be able to find the possible completions.

Sometimes you have to nudge the IDE a bit, for instance when it suggest to create two constructors from the above example. Accept one of the suggestions, but then modify the generated constructor to accept the type parameter of the class as input type of the constructor. After saving all files, you will have convinced both IDE and compiler.

AssertJ examples.

In our course we to stick to AssertJ library where possible.

The rationale for that is:

  • The AssertJ assertion API is very powerful and can easily turn overly strict or brittle tests into more effective tests.
    You have come across tests in the mooc that insisted on specific formatting. With AssertJ you can specify elements you want to see and do not want to see. See examples below.
  • AssertJ tests tend to be quite readable, once you get used to the fluent style. You will see long method names, but they help understanding what is going on a no-brainer.
  • AssertJ can protect the test methods against unexpected exceptions, such as null pointers (NPE) before doing further tests.
  • With the exception of testing exceptions, the userinterface and api of AssertJ is quite consistent.
    You always start with assertThat( objectOfInterest ).assertingMethod(…​).

We try to use the latest stable version of AssertJ and Junit 5.

Simple Tests

test boolean value.
  car.start();
  assertThat( car.motorIsOn() ) 1
      .as("Motor should be on after start") 2
      .isTrue(); 3
  1. Object of interest, return value of method, which in the test fails if not true.
  2. Optional description, use in particular when testing primitive values such as boolean.
    Use it to improved the information you get when the test fails.
  3. The actual verification. Use isTrue() and isFalse() for boolean, isEqualTo() and isEquals() for other types.
Another SimpleTest
  assertThat(car.speedLimit())
      .as("Make it a legal car?")
      .isEqualTo(250);
Equals or same?
    assertThat( f1Champ ).isEqualTo( louis );
    assertThat( f1Champ ).isSameAs( louis );

Quiz: what is the difference between the two here: equals and same?

String Containment

test string containment
  Student student = ....

  assertThat( student.toString() )
      .isNotNull()                              1
      .doesNotContainIgnoringCase("student{")    2
      .contains( "Harry", "Potter", "1980-07-31", "12345"); 3
  1. Ensure that the next test can inspect a String without tripping over a NPE.
  2. Auto generated is too simple. Such autogenerated string typically starts with "Student{" for NetBeans IDE.
  3. The test ensures that the birth date is contained, and 12345 is Harry’s student number.

How the string is formatted does not matter for this test to pass. It only requires the shown strings in any order.

Collection Containment

For collections, there are numerous tests. Collections in this context include everything Iterable, making it very powerful as can be seen in the Javadoc page of Iterable by following the link in this sentence. Look at the sub-interfaces and the implementing classes.

simple containment.
    List<Professor> professors = List.of( SNAPE, DUMBLEDORE, MCGONAGALL ); 1

    List<Professor> teachesTransfiguration = professors.stream()
            .filter( p -> p.teaches( "Transfigurations" ) ).collect( toList() );

    assertThat( teachesTransfiguration )
            .isNotNull()                         2
            .hasSizeGreaterThanOrEqualTo( 2 )    3
            .contains( DUMBLEDORE, MCGONAGALL ); 4
  1. Assume this to be in the business code or SUT.
  2. Not required but a good tip to make sure a test does not trip over an NPE, which might be confusing.
  3. The following contains-test makes this redundant
  4. because we want these two at least.
//  List<Student> hogwartsStudents = ...; 1
  List<Student> hogwartsStudents = List.of( HARRY, HERMIONE, RON ); 1

  assertThat( hogwartsStudents )
          .isNotNull()                                2
          .hasSizeGreaterThanOrEqualTo( 3 )           3
          .extracting( s -> s.getStudentNumber() )    4
          .containsExactlyInAnyOrder( 12346, 12345, 12347 ); 5
  1. Assume that the business code produces such a list.
  2. Mostly self (test) protection.
  3. Just to show that you can test for not empty, but also exact size, greater, etc. This assertion is made redundant by the test in 5.
  4. Extract a feature out of each student using a lambda. Its function should be obvious.
  5. Then test the list of extracted values for containment.

Assert Exceptions

In most business processes you want to avoid exceptions, but when they are expected, they must be thrown by the code under test, so that too needs to be tested.

There are three cases:

  1. You are not expecting an exception and do not (want to) care about it.
    Then let it simply occur and if it is a checked exception make your test method throw it.
  2. You want specific code not to throw an exception and you want to test for that.
    Wrap the suspect code in a lambda and invoke it using assertThatCode( suspectCode ).
  3. You want a specific exception to be thrown under specific a circumstance.
    Wrap the exception-causing code in a lambda and catch and inspect the resulting exception using assertThatThrownBy( causingCode ).

In AssertJ the exception testing helpers have a format that deviates from the assertThat().someCheck(…​) style. This inconsistency has to do with the way the exceptions causing code must be called, and cannot easily be avoided. We propagate one form, declaring a lambda first, and use that as the parameter to the exception asserter.

Ignore or pass on

In case you are not interested in an exception in your test, but it is a checked exception, simply declare your test method to throw it.

Case 1: not interested in the (checked) exception, add a throws clause.
    @Test
    public void fileUsingMethod() throws IOException { 1
        Files.lines(Path.of ("puk.txt") );  2
    }
  1. This code potentially throws an IOException, but you are not interested in testing the exception. If it occurs, let the caller (Test Runner) deal with this unexpected situation. The IOException is an example.
  2. This is the method that throws the checked exception. This is an example. Normally it should be a business method.

Exception NOT wanted.

If you want the check for an exception NOT to occur when invoking a code sequence, isolate the sequence in a lambda expression of the form ThrowingCallable code =() → { suspectCode(); }.
ThrowingCallable is a Functional interface and is part of AssertJ.

Case 2: the business code should explicitly NOT throw an exception.
    Student draco = new Student("Draco", "Malfoy", LocalDate.of (1980,6,5));
    ThrowingCallable code = () -> {
        hogwarts.addStudent( draco );  1
    };

    assertThatCode( code)
            .as( "draco should be accepted to make the adventures possible")
            .doesNotThrowAnyException();
  1. Is the only code that is checked for exceptions. This isolates the "suspect" code from any other code that may cause issues.

Exception needs to occur.

When you want your business code to throw an exception, wrap that business code (the method invocations) in a lambda expression, in the same way as in the previous paragraph, then pass that code to the exception assert method.

Catch a specific exception.
    @Test
    public void addIllegalProfessor() {
        var malfoy = new Professor( "Lucius", "Malfoy", LocalDate.of( 1953, 10, 9 ) ); 1

        ThrowingCallable code = () -> { 2
            hogwarts.addProfessor( malfoy );
        };

        assertThatThrownBy( code )
                .isInstanceOf( Exception.class) 3
                .isExactlyInstanceOf( IllegalArgumentException.class) 4
                .hasMessageContainingAll( "should","teach"); 5
        // fail( "addIllegalProfessor completed succesfully; you know what to do" );
    }
  1. Someone who knows his classics understands that this crook can’t be a professor at Hogwarts.
  2. The lambda defines the throwing code. org.assertj.core.api.ThrowableAssert.ThrowingCallable is the functional interface for this purpose.
  3. Sometimes it is good to be a bit relaxed on the exception type like in this line.
  4. Or you need to be quite specific. You need only one, so choose either line 2 or 3. This is just an illustration.
  5. You might want to inspect the message for keywords.

In this fluent style, you can check many more things. See the AssertJ user guide and API for that.

Soft Assertions

Sometimes you have a pressing need to assert more than one thing in a test method, because two or more values always come in pairs and setting stuff up and dividing it over multiple methods would make the test less readable, which is bad.

However, having multiple asserts in one method is bad style, because the first problem that occurs is the only failure that will be reported by the test. You could say that the first failure dominates or monopolizes the test method.

However the rule above makes writing tests a bit awkward. In particular when there are multiple aspects to be tested on the same result object. In that case a soft assertion may be helpful. A soft assertion will fail "softly", meaning that the failure is recorded, but the test can still continue to verify another aspect. The soft assertion will collect all failures and will report them after the soft assertion is closed.

soft assertion, testing both key and value of a KeyValue object.
   void testGetResult(){
    // some setup
    Optional<KeyValue<Integer,Double>> = new gf(student).gradeFor("PRC2");
    SoftAssertions.assertSoftly( softly -> { 1
                AbstractMap.SimpleEntry<Integer, Double> result = gf.getResult();
                softly.assertThat( result )
                    .extracting( "key" )
                    .isEqualTo( student.getStudentId() );
                softly.assertThat( result )
                    .extracting( "value" )
                    .isEqualTo( grade );
    }); 2
  }
  1. Starts the lambda that encloses the soft assertions. Softly serves as the collector of the failures
  2. Closes the sequence of soft assertions. This is the spot where SoftAssertion will report the the failures, if any.

In the example, the key and value in the KeyValue pair are related, making a soft assertion applicable.

Assumptions

There are test cases that are only useful under certain conditions:

  • The test is only meaningful on a specific Operating System.
  • The test needs a database connection and only executes when available.
  • A file must be present.
  • A 'slow' test flag is set in some file, which only executes the test when that flag is set. Used to avoid slow tests in the normal write-compile-test cycle.

For this you can use Assume.

There are two variants relevant:

You can use the Assumption test in the setup methods (the one with @BeforeXXX), to completely disable all tests in one test class, instead of having each test doing the assumeXXX test. A test that has been canceled because of a failing assumption will appear as a skipped test, similar to a disabled test with @Disabled.

The JUnit 5 Assumtions are effective, but can only test for true or false, so you need to feed it a boolean expression. Look at the API to see all features. But simple may be just perfect here.

Disable on staging machine. From JUnit 5 user guide.
    @Test
    void testOnlyOnDeveloperWorkstation() {
        assumeTrue("DEV".equals(System.getenv("ENV")), "Aborting test: not on developer workstation");
        // remainder of test is only executed if on developer machine
    }

You guessed it, AssertJ Assumptions are richer. In most cases, the assumption has the same form as an assert that, as in replace assert with assume and you are done. So any assertThat(…​) that you already have can simply be turned into an assumeThat(…​).

If nothing else, using assertj may make your tests more readable and have a consistent look and feel.

Disable test if file does not exists
    File f = new File(Path.of("testData.dat"));
    assumeThat(f)
       .as("Test data file not available, skipping test")
       .exists();

Additional pointers