Grounded Test Design Rikard Edgren
For quite some time I have felt that the classic test design techniques don’t add up to the needs of software testing that tries to find most of the important information.
At EuroSTAR 2009 it dawned on me that it is time to describe the method that I, and many, many others, have been using for a long time.
By talking more around this, I hope we can spread the usage of this test design, improve the way we do it, and make it more understandable for other people, avoiding the comment: “oh, you’re error guessing.”
What I will describe could be called a method, activity or a skill, but I like to think of it as a test design technique, that is seen as a tester’s most powerful weapon, much better than equivalence partitioning, decision trees etc.
Grounded Test Design: learn a lot, and synthesize important test ideas.
To explain this further: Software testers should follow the thoughts of social science Grounded Theory; we should use many sources of information regarding the subject (requirements, specifications, prototypes, code, bugs, error catalogs, support information, customer stories, user expectations, technology, tools, models, systems, quality objectives, quality attributes, testing techniques, test ideas, conversations with people, risks, possibilities); we should group things together, use our creativity, and create a theory consisting of a number of ideas capturing what is important to test.
Grounded Test Design is for those that want to test a lot more than the requirements, that understand the need to combine things of different types, to look at the whole and the details at the same time.
This might seem heavy, but people have phenomenal capacity.
It has a lot more to it than the closest technique error guessing; for instance, it is not only about errors, it can also be investigations, or questions regarding dubious behavior.
It isn’t guessing, it is a qualified synthesis of knowledge.
And also: it isn’t (and doesn’t sound like) something done lightly, something unstructured, or unprofessional.
Being a technique, it is something that can be applied in many different situations. The most informal Grounded Test Design might happen extremely fast in the expert tester’s head when thinking of a completely new idea while performing exploratory testing; or it could be the base for test scripts/cases/scenarios, that are carefully investigated and designed in advance. Good software testing uses many different methods.
A formal Grounded Test Design (to be defined, tried and evaluated…) might be best used by someone completely new to a product, or maybe most efficiently performed in diversified pairs.
Grounded Test Design has a loose scientific, theoretical base in Grounded Theory (Strauss/Corbin), used in social science as a qualitative, thorough analysis of a phenomenon.
The scientists examine the subject very carefully, gather a lot of material, document codes and concepts, combine them to categories, from which a theory is constructed.
There is no hypothesis to start with, as in the traditional natural science, and that’s quite similar to software testing; it would be dangerous if we thought we knew the important questions at once.
I think it is good to be slightly associated with this theory, especially since it emphasizes that the researcher must use her creativity.
On the other hand; do we really need this word, maybe we already have enough words about how to do good software testing?
Or is it a good word to have in reserve when explaining to managers why testing is difficult, takes a lot of time, and must include manual work?
Or should we go further with this and try to develop a framework for performing structured analysis of everything that is important for testing?
Opinions are very welcome.
Rikard, this does not strike me as all that different from exploratory testing. I think that you have thought a lot more on this than can be explained in one small article. Also, there are loads of theories out there that can be applied to testing. General systems thinking is one path to explore.
Well, to me it sounds like the core ideas of the context-driven group. Diversification in techniques, designing tests, executing tests, finding useful information wherever you can. I would say like James Bach says about ET. It is not a technique, it is an approach.
That said, I need to read more about Grounded Theory and probably need to discuss much more with you before beeing able to give better feedback.
Looking forward to learning more.
I agree it is a lot like Exploratory Testing and Context-driven school, which is natural since I believe the people behind are the ones that have written the very best things about software testing.
But at the same time, since they talk about an approach, wouldn’t it be good to have a name for the technique/method used when designing tests, coming up with really good test ideas?
So I see two ways to go with Grounded Test Design (it’s a too good name to just let it go):
* use it inside Exploratory Testing to explain better
* go more formal, try to annotate the testing knowledge, do something similar to Grounded Theory’s codes/concepts/categories/theory
(I also think there could be positive side-effects like better knowledge transfer hidden here.)
There is a public counter-example where Grounded Test Design fits much better than Exploratory Testing:
Fiona Charles: A Modeling Framework for Scenario-Based Testing
http://www.stickyminds.com/sitewide.asp?ObjectId=14544&Function=DETAILBROWSE&ObjectType=CP&sqry=%2AJ%28MIXED%29%2AR%28createdate%29%2AK%28simplesite%29%2AF%28cern+kaner%29%2AZ%28SM%29%2A&sidx=5&sopp=10&sitewide.asp?sid=1&sqry=%2AJ%28MIXED%29%2AR%28createdate%29%2AK%28simplesite%29%2AF%28cern+kaner%29%2AZ%28SM%29%2A&sidx=5&sopp=10
Stumbled on GT in a different context and remembered you’d written something about it, so I’m re-visiting…
I like the idea of Grounded Test Design – it could be a useful concept to help explain the difficulty with testing (as well as an approach for testers.)
On the one hand a tester may have a hypthesis-led approach. But it’s also useful to demonstrate the need for a data-led (quantitive analysis) approach.
Yes, there are similarities to the heuristic test strategy and activities in good exploratory testing – but I think emphasizing the need for the opposite of the “scientific method” is useful in explaining testing to non-testers.
Then Grounded Test Design is maybe a useful term – but I think some more exploration is needed (by me anyway.)
Of course, whether trying to explain testing in terms of a scientific approach and its opposite is a good way of communication is another story. But relating testing more to the social sciences is a way of explaining the complexity that can be involved with testing a product (I’ve started recently to talk about testing and it’s relation to cognitive science – to help highlight bias in thinking about a product, test activity or test result.)
I think I need to read some more about GT.
Thanks for the comment, Simon.
I’m working on elaborating the concept, so there is more to come; I have written a blog post on “The Complete List of Testing Inspiration”, and forthcoming on “Synthesizing Test Ideas”, “Testworthy” and “Fractionation/Factoring” hopefully tying it together in a paper or simiilar.
Not sure what others would get from reading Grounded Theory, but for me, the similarities with testing were striking.
These thoughts are finalized and put in The Little Black Book on Test Design http://www.thetesteye.com/papers/TheLittleBlackBookOnTestDesign.pdf
I changed name, because I realized that Grounded Test Design mainly was vanity, wanting to coin a term.
(I have also experienced that coining doesn’t work, e.g. Charisma…)
Hi Rikard
Quite independently I’ve been looking into this, and it’s something I did as part of my degree in psychology. Where I think it can be very handy, and where I’ve informally used it without really realising is when listening to people talk about the project.
For example, I used to sit in on client meetings, and people would be talking about what they wanted. I would write down specific things people said and group them. For example:
“Must be easy to use” “The current system is a nightmare to navigate” “we just want it clean” “Make it simple for anyone” I would group all together. These are things that will never appear in a formal document, but it paints a picture of the client’s current reality and what is important to them. I then use that to guide my testing and as evidence when advocating for bugs. So Grounded Theory I think very much has a place in testing. See also: Social Constructionism
Cool!
I have been wanting to try memos with coding, but haven’t had a suitable project yet.
But the informal information gathering that is done by testers, why not put it in a formal document?
Not only is it useful to others, it is also a chance to correct misunderstandings from the tester.
[…] Six Thinking Hats (1985) Collins, Harry. Tacit and Explicit Knowledge (2010) Edgren, Rikard. Grounded Test Design (2009) Glaser, Barney and Strauss, Anselm. The Discovery of Grounded Theory (1967) Harty, Julian. […]