Get best of the week

Assert messages in tests

Hello again. Ahead of the start of the "C # Developer" course , we have translated interesting material about assert messages in tests and are happy to share the translation with you.



In this post we will talk about whether you should use Assert messages in your tests.

I received an interesting question from a reader colleague, which I would like to dwell on in more detail:

I have a question about Assert messages: should I use the overload containing the message parameter and use it to send a string describing the reason for the failure of Assert (also “Claims”) ?

The answer to this question comes down to two aspects:

  • Readability of the test - how easy it is to understand what the test does.
  • Ease of diagnosis - how easy it is to understand why the test fails.

Let's discuss each of them individually

Test readability


People often use Assert messages to help team members and themselves in the future understand what is happening in the test. Let's look at the following example:

[Test]
public void Hiring_a_new_team_member()
{
    var company = new Company();
    var person = new Person(UserType.Customer);

    company.HireNewMember(person);

    Assert.AreEqual(UserType.Employee, person.Type); //  
    Assert.AreEqual(1, company.NumberOfEmployees); //  
}

Instead of a bare Assert, you can also indicate the reason why the test Assert validates something:

[Test]
public void Hiring_a_new_team_member()
{
    var company = new Company();
    var person = new Person(UserType.Customer);

    company.HireNewMember(person);

    Assert.AreEqual(UserType.Employee, person.Type, "Person must become an employee after hiring");
    Assert.AreEqual(1, company.NumberOfEmployees, "Number of employees must increase");
}

Such statements help, but they come at a price. These messages require you

  • Spend time writing them
  • Keep them moving forward

Here the set of pros and cons is the same as in the comments to the code. And, as in the case of the comments, I advise you: do not write assert messages purely for readability purposes. If you feel that the test is not obvious without assert messages, try refactoring it instead. In the long run, it’s easier to make the test speak for itself than to keep it in sync with assert messages (and this is only a matter of time before the synchronization is broken).

Enter assert messages only when absolutely necessary - when you cannot improve the readability of the test in any other way. But even then, incline to choose not to write them.

The easiest way to get a quick gain in test readability is to switch to a readable statement record. For example, NUnit has a special constraint based assertion model that helps you write your assertions as follows:

[Test]
public void Hiring_a_new_team_member()
{
    var company = new Company();
    var person = new Person(UserType.Customer);

    company.HireNewMember(person);

    Assert.That(person.Type, Is.EqualTo(UserType.Employee));
    Assert.That(company.NumberOfEmployees, Is.EqualTo(1));
}

Or you can use my favorite Fluent Assertions :

[Test]
public void Hiring_a_new_team_member()
{
    var company = new Company();
    var person = new Person(UserType.Customer);

    company.HireNewMember(person);

    person.Type.Should().Be(UserType.Employee);
    company.NumberOfEmployees.Should().Be(1);
}

These statements are read in plain English — exactly the way you would like all your code to be read. We humans prefer to perceive information in the form of stories. All stories adhere to this model:

[] [] [].

For instance,

  .

Here - the subject, - the action, and - the object. The same goes for code.

This version

company.NumberOfEmployees.Should().Be(1);

reads better than

Assert.AreEqual(1, company.NumberOfEmployees);

precisely because history is traced here.
, . , .


Another view of assert messages is in terms of ease of diagnosis. In other words, the simplicity of understanding the cause of a test failure without examining the code for this test. This is useful when reading CI assembly results.

From a diagnostic point of view, follow this guide: if you can easily restart the test locally, this test does not need an assert message. This is true for all unit tests (since they do not work with out-of-process dependencies), but to a lesser extent for integration and end-to-end tests.


Testing pyramid

As you climb higher in the test pyramid, you may need more detailed information, because integration (and especially end-to-end) tests are slower and you may not be able to re-run them at will.

But even with integration and end-to-end tests, there are ways to facilitate diagnostics without resorting to assert messages:

  • Make the test test one behavior module - when a test checks one thing, it is often easy to determine what went wrong. (This is not always applicable to end-to-end tests, as you might want these tests to check how several units of behavior work closely).
  • Name your unit tests accordingly - The ideal test name describes the behavior of the application in business terms so that even a non-programmer can understand it.

And don't forget that even without user statements, you still have the messages that the unit testing environment generates for you.

For example, an error in the following statement:

person.Type.Should().Be(UserType.Employee);

gives the following error message:

Xunit.Sdk.EqualException: Assert.Equal() Failure
Expected: Employee
Actual:   Customer

The combination of such messages generated by the environment and human-readable test names makes 90% of user-assert messages useless even in terms of ease of diagnosis. The only exception is lengthy end-to-end tests. They often contain multi-stage checks, so it makes sense to use additional assert messages to understand which of the steps failed. However, there should not be many such end-to-end tests.

Of course, in order to take advantage of the error messages generated by the framework, you need to avoid common Boolean comparisons, such as:

(person.Type == UserType.Employee).Should().BeTrue();

Because they lead to the following error message:

Xunit.Sdk.TrueException: Assert.True() Failure
Expected: True
Actual:   False

Which does not help with the diagnosis at all.

Summary


Do you often feel lazy when it comes to writing statements? Well, now you can justify it and send your colleagues to this article.


“I am glad that this has a name”

Jokes aside, however, here is a summary:

  • There are two aspects to using assert messages:
    • Readability of the test (how easy it is to understand what the test does).
    • Ease of diagnosis (how easy it is to understand why the test fails during CI build).
  • In terms of test readability, assert messages are code comments. Instead of relying on them, refactor your test to achieve readability.
  • In terms of ease of diagnosis, the best alternative to assert messages is:
    • Testing a single behavior module by a test
    • Naming tests in business terms
  • The only exception is lengthy end-to-end tests.

That's all. You can learn more about our course at the free webinar , which will be held today.

More articles:


All Articles