Not all testing is software testing Martin Jansson
In many discussions about testing methods, courses, techniques, approaches etc it is usually software testing that is in focus. I cannot see why the limit is set to just software. For instance, the excellent course Rapid Software Testing advocates, by its name, that is meant for personel who perform testing of software. It could perhaps be called Rapid Testing, because the esssence of the course is not limited to just software. Does the name limit what non-software testers and their managers think of this course?
I’ve seen similar limited thinking when someone was building production test equipment to be used on each unit created in the factory. For production testing there are some important factors of the tests performed namely speed of the tests and a high yield. The production test equipment should also be user friendly because the person running the test should do it correctly and quickly so that you get even higher yield. You can use the concept of a smoke test, then take some ideas on how unit testing is done but there are several other areas that can be considered when creating the tests in the test station. Naturally you can use your entire tool box of techniques when testing the test station/production test equipment.
When testing hardware you do many checks (as Michael Bolton calls them), measuring power, temperature and so on, but you can do these with an exploratory approach. There are usually several well defined tests found in various standards with test scripts on how to execute them. Usually it is the designer who does this task. Do they use any of the methods and techniques that are created and discussed in software testing? I think that would be very rare.
Me and a collegue used exploratory approach and pair-wise testing when doing testing on a radar used to trigger a blinking light. There is some software included in all this, but I would not say I was doing software testing, rather system testing or just testing. I was not doing any verification or validation, since we had no specs and the creators had given us too little information on how the system was supposed to work. So, we explored it and found bugs that did not fit in our model of thinking. Some of the issues found were indeed bugs others were not.
Do you say that you do software testing when testing a car? 20 years ago software was probably never used in a car, but the test ideas and test techniques from that time might apply today with the cars that do have software to control all major systems.
Before being too quick in saying that it is software testing that your approach, technique, course or thingie is focusing on, do think again.
I think it’s OK to say software testing for things that apply elsewhere as well.
Probably you have used software testing as a base when developing the course, or what you are talking about, meaning it is most appropriate for just software testing.
Rather, I think it is important to know that you can learn a lot from areas that aren’t exactly the same as yours.
It’s not the name that sets the limit, it is the thinking of non-software testers and managers that sets the limit.
Regarding testing of other things than software, I think it would be very fruitful to use software testers (with a lot of critical and creative thinking) when designing laws in the society; build scenarios and look at the outcome; is this what we want to accomplish; are the negative side-effect acceptable?
Naturally you are free to say whatever you want, but when we talk about testing I think it is possibe to consider if talk about just software testing or about testing in general. There is a lot written about software testing, how much is written about other kinds of testing? Are there a growing group who identify themselves as production test equipment tester or do the tag along on the software testers ride? I do not identify myself as a software tester. I know a lot about the general software environment, but I dare say that noone is familiar with a general system environment.
I agree with you Martin.
And especially since software and hardware systems becomes more and more tangible and integrated in our lives.
E.g., System Testing of a cell phone includes both hardware and software testing simultaneously and the test approach should include both.
On the other hand there are techniques that are better suited for software testing and others for hardware testing that of course must be taken into consideration.
I believe, and this is most subjective, that software testing has tried to resemble hardware testing as much as possible such that it tries to treat software testing a deterministic system. This has lead to a lot of bad software testing being performed over the last years. And this is the reason why so many want to treat software testing as a “factory” where testing is reduced to scripted recipes and bad metrics are used to assess quality and control the work.
Perhaps software testing has developed more (or at least been forked more times) than hardware testing and perhaps it is time to bring back some of the most interesting ideas regarding sapient software testing back into hardware testing!? And as Martin says, the testing techniques and approaches could benefit from including the hardware dimensions.
Martin, I am looking forward to reading more from you regarding this subject!
I also agree, and I think we have the same problem in “system development”. We tend to talk about HW development, SW development, Test, Configuration Management etc as separate disciplines instead of focusing on the bigger picture. And this gives an imbalance. Its like having a recipe of a dish – by focusing on getting the best version of one ingredient doesnt mean the overall taste of the dish actually improves.
With this I mean that one should only separate and differentiate when needed – not use it as starting point. Meaning that we should talk about testing (if needed to be separated from the bigger picture), then talk about SW testing if we need to differentiate even further.