What are you going to test for this story?
That was a question that I ask myself a lot. This is a question that needs to be answered for every user story that I test.
We do have a lot of automated tests. Tests that are written in python. It are tests on API level, so without interaction of the user interface. I also have to test manually. Not every part of our huge system can be automated. For manual testing, I created some test documentation. That documentation contains what will be tested. I do not just write in detail what I look for, but in general. There should be room for doing crazy stuff while testing too.
How should the system react? What should the software do? These are some of the questions that are answered in the testing documentation.
Documentation?
The first version of this test documentation was a document written in word. It worked, because I was the only one who edited the document. When the company grew, this was also not suitable anymore. I and my colleagues can not work on that document at the same time. That is the reason that I looked at a new format for my test documentation.
What did I want?
- It must be simple to edit.
- No code language like html or so. Preferable in markdown or another simple format.
- Different users should be able to edit the same document at the same time.
- It should be placed in version control, so that we can merge it, like other code.
The new documentation
Sphinx is a tool that makes it easy to create intelligent and beautiful documentation, written by Georg Brandl and licensed under the BSD license.
Our testing code was already in python. Python has some nice documentation. Why not use the same documentation framework as the python community uses? That could be a good alternative. I found the Sphinx project. This project seems to be what I wanted. On the main page the first sentence was promising. It talks about easy to create and beautiful documentation.
The first thing I did was installing Sphinx. So I entered:
pip install Sphinx
That was an easy one. The next one was in an empty directory, I typed the command
sphinx-quickstart
That is also explained in the First Steps With Sphinx Tutorial.
This sphinx-quickstart program asked me a lot of questions. I answered them one by one. One of the last question was if it should generate a make file. I answered yes. Now I do not need to remember some large command like:
sphinx-build -b html sourcedir builddir
Now the program has generated some files, that can be placed in version control. Building the documentation is very simple:
make html
Test documentation itself needs to be in reStructuredText format. It is also very easy to learn. It is nearly the same as markdown. More information on reStructuredText can be found on their webpage.
Because now the complete test documentation is in text files, it is possible to store it in version control. We use git in our company, so that infrastructure can be re-used. I also make use of our jenkins build server. This build server generates the documentation for us after each check-in. And that documentation is afterwards copied to a web server. Now everybody can have a look at what is tested at any time.
It is a system that is easy to use now, and it is transparent for everybody. Isn’t that what I wanted?