Testing exceptional behaviour

Last updated on 2026-06-29 | Edit this page

Overview

Questions

  • How do I verify that my code fails in the right way?
  • What should happen when a function receives invalid input?

Objectives

  • Explain the difference between testing error handling and testing normal behaviour
  • Use EXPECT_THROW to assert that a specific exception type is raised
  • Write tests for the boundary conditions of invariant_mass(): negative energy, unphysical mass squared.
  • Explain why the choice of exception type matters and how to test for it specifically.

Testing error handling

Testing that our software fails under the conditions we expect is just as important as testing that it succeeds. These failure conditions are as much a part of a function’s specification as those for its success. In invariant_mass() for example, we’ve specified:

CPP 

// 1. Return invariant mass m = sqrt(E^2 - p^2) in natural units
// 2. throws std::domain_error if E < 0
// 3. throws std::domain_error if E^2 - p^2 < 0
double invariant_mass(double energy, double momentum)

Thus we should test that (2) and (3) really do result in a thrown exceptions when given inputs as specified. The key point here is that we are not testing “what goes wrong”, rather “that the function does what it is supposed to do when given invalid input”, in essence:

  • Normal behaviour: provide valid input, check the return value.
    • Here, the assertion is about what comes out.
  • Exceptional behaviour: provide invalid input, check that the error is triggered and is the right type of error.
    • The assertion here is about what the function refuses to do.

This is another reason specifications for (i.e. documentation of) function behaviour is so important. We’ve made a design choice to handle invalid input to invariant_mass by throwing exceptions - but other programmers might make different decisions on error handling:

  1. If E < 0 or E^2 - p^2 < 0, return -1.0 to indicate failure.
  2. If E < 0 return -1.0, if E^2 - p^2 < 0 return -2.0
  3. Have bool invariant_mass(double E, double p, double& mass), return false and set mass to -1.0 if E < 0 or E^2 - p^2 < 0.
  4. If E < 0, terminate execution completely.

All of these are defining what happens on and just outside the boundaries of applicability of the function, which are often the most trouble prone parts of our codes. Thus no matter how we handle errors here, we should always test that they are handled, and as we expect.

Testing for exceptions with GoogleTest

To check that a function throws an exception and that the thrown exception is the correct one, GoogleTest provides the EXPECT_THROW (and corresponding ASSERT_* form) assertion. This is very simple, so let’s use it to implement the E<0 error case in the UnphysicalEnergies test case:

CPP 

// Case 2. Test unphysical energies
TEST(InvariantMass, UnphysicalEnergies)
{
  EXPECT_THROW(invariant_mass(-1.0, 0.0), std::domain_error) << "negative input energy does not throw";
}

The first argument is just the expression we want to assert on, the second is the C++ type of what we are asserting the expression will throw. Building and running again, we’ll now see the UnphysicalEnergies passes:

BASH 

$ ctest --test-dir build --output-on-failure
Test project /tmp/ccptepp-test/build
    Start 1: TestInvariantMass
1/1 Test #1: TestInvariantMass ................   Passed    0.23 sec

100% tests passed, 0 tests failed out of 1

Total Test time (real) =   0.24 sec

Let’s go on to add the case for negative mass squared but deliberately make the expected assertion type wrong:

CPP 

// Case 2. Test unphysical energies
TEST(InvariantMass, UnphysicalEnergies)
{
   EXPECT_THROW(invariant_mass(-1.0, 0.0), std::domain_error) << "negative input energy does not throw correctly";
   EXPECT_THROW(invariant_mass(1.0, 1.1), std::runtime_error) << "negative mass-squared does not throw correctly";
}

Building and running now trigger an error:

BASH 

...
[ RUN      ] InvariantMass.UnphysicalEnergies
/tmp/ccptepp-test/test/test_invariant_mass.cpp:29: Failure
Expected: invariant_mass(1.0, 1.1) throws an exception of type std::runtime_error.
  Actual: it throws std::domain_error with description "unphysical mass^2".
negative mass-squared does not throw correctly

[  FAILED  ] InvariantMass.UnphysicalEnergies (1 ms)
...

Thus we get a helpful message when an exception is thrown but it is not the right type of exception. We can also confirm that the assertion will fail if the call does not throw at all by changing the assertion to:

CPP 

// right exception type, but it won't throw!
EXPECT_THROW(invariant_mass(1.1, 1.0), std::domain_error) << "negative mass-squared does not throw correctly";

BASH 

[ RUN      ] InvariantMass.UnphysicalEnergies
/tmp/ccptepp-test/test/test_invariant_mass.cpp:29: Failure
Expected: invariant_mass(1.1, 1.0) throws an exception of type std::domain_error.
  Actual: it throws nothing.
negative mass-squared does not throw correctly

[  FAILED  ] InvariantMass.UnphysicalEnergies (0 ms)
Callout

As you might anticipate, GoogleTest also provides EXPECT_NO_THROW, which asserts that an expression does not throw any exception. This is most useful when a function can throw for some inputs and you want to explicitly document that a particular valid input is safe. It’s less useful for general. It is less useful when a test already makes assertions about the return value, since a thrown exception would cause those assertions to fail anyway.

Together, these cover all of the possible cases we’ll need, but there are two small things to watch out for.

First, we’ve chosen to throw the same exception type for both error cases. This isn’t unreasonable since they are both domain errors, but strictly speaking this means our two test cases aren’t completely distinguishing the E<0 and E^2 - p^2 < 0 cases. We are testing the specification is met though, which the main thing here. If we did want to be specific here, we might introduce our own exception types to distinguish both.

Second, and somewhat related, GoogleTest’s check on the type of the exception thrown uses C++ “is-a” inheritance rules if class types (as std::domain_error is) are involved. What this means is that if invariant_mass threw an exception, say foo_exception, that inherits from std::domain_error in these assertions, the test would actually pass. We can mock this by writing:

CPP 

EXPECT_THROW(invariant_mass(1.0, 1.1), std::exception) << "negative mass-squared does not throw correctly";

as std::domain_error inherits from std::exception. Building and running this will show a passing test case:

BASH 

(ccptepp-test) $ ctest --test-dir build --output-on-failure
Test project /tmp/ccptepp-test/build
    Start 1: TestInvariantMass
1/1 Test #1: TestInvariantMass ................   Passed    0.23 sec

100% tests passed, 0 tests failed out of 1

Total Test time (real) =   0.24 sec

This is intended and semantically correct behaviour from GoogleTest - we have asked it to check that a std::exception is thrown, and std::domain_error “is-a” std::exception, so the assertion condition is met. There’s no real way around this other than to be as specific as possible when declaring the type you expect to be thrown, and don’t have deeply nested inheritance hierarchies for exceptions!

Callout

How would we test the other possible error handling mechanisms we outlined at the start of the episode? The first three are logically handled by EXPECT_EQ or ASSERT_EQ and their variants we’ve seen already. We can actually test for termination with so-called “death tests”. These are rather specialised, but do have their place.

Key Points
  • Testing what your code refuses to do is as important as testing what it does
  • A function’s error handling is part of its specification and should be documented and tested like any other behaviour.
  • These often determine boundary conditions where bugs most commonly live, making them vital to test.
  • EXPECT_THROW checks both that an exception was raised and that it was the right type — the type is part of the function’s specification.
  • With invariant_mass() now fully tested, we have seen a near complete range of GoogleTest assertion types — the remaining episodes apply these tools to more complex code.