Don't write tests against configuration

The main idea of a test is that it is simpler than what is being tested. If we write tests that execute against production configuration then we face quite high risk that our test code becomes equally complex as production code. Effectively we are just duplicating our configuration in a more cumbersome format. In addition such tests suffer terribly from magic values. When looking at the test there is no explanation why our code behaves as it does with given set of parameters. We have to look up not only the production code but also configuration to fully understand what is happening.

Example test that relies on production configuration:

Just like in this example testing against production config often requires loading either whole application or some good chunk of it. This means that test will be slow to execute and often strange mix between full application test and unit test (check Testability on Building module apps using Spring).

It is unclear by just looking at this test why should EUR-GBP be valid while the reverse route is not. Good test should be self explanatory.

To fix this test we have to make it possible to initialize Routes from within the test:

This test removes the need to look at the implementation of Routes to understand why some routes are supported and some not.

But what if we have some bug in our configuration file? First, we should ask if we really need this configuration in a separate config file. If we just hardcode the list of supported routes in Routes or the context that creates new routes object then compiler can do some validation for us. If we still need to keep it it in a config file we can probably just write separate test to verify we can parse config correctly but that test does not need to test if we use config correctly.


Used still from Spike Jonze's Her