I noticed a few days that nobody of our team is watching to the outcome of the automated integration tests. Our tests run on an integration server. Since the beginning of our sprint, one test is failing. The result is that the build was not a single day in our sprint green.
The first day of our sprint, one of our tests broke. The test was already running for several months without failing. One of the stories had impact on code that could break this test. In normal circumstances, the fix is there within a few hours. That is fine. Green tests give confidence.
This time, things are different. The test is failing until the story that has impact on it, is implemented. But still today, the day before the review meeting, the test is still failing. And indeed, the story is not finished yet.
This is a problem for me. More teams are working on the same repository. The other teams also rely on the integration tests. Their safety net is gone already for at least 2 weeks. Of course they can have a look at the status of the testing job and see that only that test is failing. But because it is taking such a long time now, nobody looks at it at this moment.
How can we solve this problem?
- Disable the test?
- Ignore the test result until it is fixed?
- Mark the test as unstable until it is fixed?
What option is chosen is not important. Only, ignoring the test results as it is today is a risk. If something else fails, nobody will notice it. And that is just not what we want.
So next retrospective this problem will be on the table.