Review of Test Case Management system – ApTest Martin Jansson
Introduction
ApTest Manager is test management system for manual testing. It is easy to use and easy to adapt to your own liking. You have the option to have different levels of features depending on what test project you are in. The more advanced contain ability to set test time per test case as well as determining how long a test actually took among other features. The tool has been created by a company that does test consultancy as well as tools to assist their own testing.
Reviews
The tool focuses on manual testing part of testing and has no support for automated test scripting at all, which is good. Still it is possible to import test result into the a test session (the actual test job) which makes it possible to have the automated tests in the system but when they are run they are imported. It might also be possible to script this import, but I’ve not come that far yet.
Database
ApTest uses an embedded database called Berkley DB. There are various drivers to use the database and here is one of them. It should therefore be possible to link the test case database into external analytics tools such as Spotfire DXP.
UI
All UI:s are web-based, quite good, easy to use and manage. Features such as Search/Replace with Regexp, moving folders, copying testcases are easy to use, even though it is web-based.
Configurability
It is possible to link this into CVS, we’ve not done this ourselves.
It is possible to connect to external bug system by adding fields for bug number, actual link to system. It is also possible to supply information from the test case system into the link using variables.
When preparing a test suite you are able to setup the session variables. These
are variables that can be a single-select list, a multi select list, or any other field you wish to create. Example of these could be IE versions, Web Browser, Operating System and so on. In the test case you are then able to use these session variables by adding <% variable name %>. This is extremely powerful because it is possible to add further context to the test case. Since it is possible to use the test case information when reporting a bug it is possible to have a lot of information ready for copy/paste into the bug system directly from the test case.
The test case ID
The test case ID has always been a topic of discussion. In ApTest the ID consists of all folders and sub-folders plus the test case name, such as /UI/Menus/File/open_file. This means that you cannot have duplicate test case
ID:s in one folder, but you are able to copy a created test cases into other folders easily.
Running a test job
Creating a test session (an actual test job) means that you create a test set where you define which session variables are applicable and then save this as a repeatable test set. You will be able to go back and change the test set if new test cases has been added. When the test set is ready you create a test session that might be applicable for a certain build or iteration. You are able to allocate test sessions to resources and let anyone pick any session. The progress of the test session is visible in a management page for the test sessions. Each test session contain a Summary for the session which shows a minor report of progress and result. It is also possible to go into each test case in the session to adjust result, notes, bugs or other information easily.
While running a test session you are able to edit a test case and update the test session directly.
Reports
There are many reports available, but most of all you are able to create your own reports easily. Adding new fields and different setup of their tables is done quickly.
Scalability
It is possible to work many on the same sessions, test cases etc. The system handle this well.
Documentation
There are two manuals; one for users and one for administrators. All of them are quite extensive and cover many aspects of the tool.
Price
The price of ApTest is far below their competitors. You pay 400 dollars per active online user and 100 dollars per year for each license for their support.
Summary
So far I can highly recommend using this tool. It was a long time since I felt that it was fun to use a test case management system, but ApTest does the trick.
Links
ApTest Website
I have seen ApTest, but never tried it in production. I have a feeling that it won’t scale for thousands of tests; but that seems OK when considering the price.
Test management seems extremely difficult to create software for; I have never seen a really good system, and can not recommend the one we are using now (starts with QA… and ends with …Director)
The problem is probably that testing should be performed in many different ways, and that you want to change parameters inside test cases, outside in test environment; you want to re-use detailed scripts, but also have scripts that leave room for improvisation.
Another problem is that data is disparate, and can’t be connected in a good way (especially when systems are built from different bought components): requirements & use cases; test ideas and scripts; jobs & results; bug reports & test information. Not easy to map these things in the many different ways different users might want to. Ergo; not easy to make a general product…