Bug Magnets are thinking as criminals Henrik Emilsson
I know of some testers who are pointed out by others to be Bug Magnets; people recognized for their ability to somehow draw bugs to them. Bug Magnets can be found in many workplaces and I bet that you know of someone that falls under this description. I have been appointed a Bug Magnet by some and it have made me thinking on what this phenomena boils down to.
Is it luck? Is it faith? Is it an ability that some are born with and some aren’t? Can you learn this ability? Can you improve it?
I have wondered about this for some time.
However, this summer I had a revelation when I watched an episode of the marvelous TV series “Homicide: Life on the Streets”:
Homicide: Life on the streets
Season 1 Episode 9 “Night of the dead living”
Det. Frank Pembleton and Det. Tim BaylissBayliss [sits pensively. To Frank]: What are you looking at?
Pembleton is playing cat’s cradle.
Bayliss: You have something that you wanna say to me?
Pembleton: Adena Watson. So many unanswered questions.
Bayliss: And you’re saying that I’m not asking them.
Pembleton: I’m saying that you’re not answering them. [He peers at Tim through the cat’s cradle.]
Bayliss [sighs]: What questions aren’t I answering, huh?
Pembleton [gets up and flips through a notebook, draws a picture]: Okay, these sixteen row houses on the north side of Kirk Avenue. Adena’s body was found outside the kitchen door in the red yard at 718 Kirk. Now the killer could have dropped her anywhere. Why not the common alley? Why not the yards at either end of the block? These three row houses are empty. One, two, three. The killer would have stood much less of a chance of being seen if he’d dumped her body in any one of these yards. Why would he risk bringing a little girl’s body inside a closed fence of an occupied house?
Maybe he wanted her body to be found immediately. Maybe he wanted to cast suspicion on the people in 718. Maybe he had some … perverse sense of remorse, some impulse to leave her body inside an enclosed yard to protect her from stray dogs.Bayliss: These are *exactly* [taps the notebook] the questions that I have been trying to answer.
Pembleton: Well, you can try, but you never will.
Bayliss: Why?
Pembleton: You don’t think like a criminal. You don’t have a criminal’s mind.
Frank walks away. Tim grimaces in disbelief.
Bang! Suddenly it struck me. “You don’t think like a criminal. You don’t have a criminal’s mind”.
In order for a police to think of possible outcomes of a crime they have to be able to think as criminals, and put themselves in a criminal’s way of thinking.
If you don’t do this, you are trying to understand and explain the crime based on your logic.
Similarly, in order to “attract” bugs you have to wanna see problems; you have to identify problems that might bug several kinds of stakeholders; you have to put all your knowledge about the project in to consideration; and you have to be able to see the problems that matter. You are not trying to understand and explain the system by using your own logic, instead you are using several input sources to do this: logic, subjective thoughts, people’s skill levels, complexity of system, technology, etc.
So let me present my take on a definition of “Bug Magnet”:
Being a Bug Magnet = Able to foresee possible problems (or able to spot opportunities for things to go wrong), in context.
The most important thing here is the last two words “in context”. That is, even if you have all the bug taxonomies and oracles in the world to support you, you have to be able to understand what matters in this project.
Knowledge about “all common problems in .NET applications” can help you sometimes. Knowledge about “all common problems in .NET applications that developer X often produce”, is however much more useful.
Understanding what bugs our users is more helpful than knowing about what bugs users in general.
Knowing about problems with focus in Windows applications is one thing, finding these problems during testing is to be able to spot opportunities when they are presented to you in your context (see http://thetesteye.com/blog/2010/11/windows-focus/ ).
Now, back to the questions in the beginning of the post.
- Is it luck?
— No, even if luck sometimes help. For others it is more of welcoming serendipity. - Is it faith?
— I don’t believe in faith. - Is it an ability that some are born with and some aren’t?
— Maybe. Some people might have a more developed talent, but I think that most people have this talent. Some are born with variations of narcissistic personality disorders and might have difficulties with this. - Can you learn this ability?
— I believe so. - Can you improve it?
— Yes. However, you might need to consider one or several dimensions to improve: Empathy, reasoning, attention for detail and seeing the whole, recognizing patterns of your own and other peoples mistakes, subjectivity, general systems theory, context-driven testing, and more.
Notice that these dimensions are not technical but rather comes from social sciences.
This is my response to one part of what I and Rikard have discussed during the last year: What constitutes skilled software testing?
More to come!
I like this post – as it makes me think. Some initial thoughts:
I’m not sure what a bug magnet is. How do you distinguish one from a good tester? Can you be a bug magnet without being a good tester, and vice versa? (You can think of a good tester in whatever terms you might distinguish someone doing good testing – good feedback on the product and their testing, good in the stakeholder’s view)
(Henrik: Good questions, I don’t know. I guess that you can be both a good tester and bug magnet, as well as you can be a bug magnet without being a good tester; i.e. I think that you can value a tester in many ways, where finding lots of bugs is one. So maybe being a bug magnet is one enhanced characteristic amongst several that constitutes a (good) tester.)
Is the distinction important? To who?
(Henrik: The distinction might be important if we wanna learn more about a tester’s quality characteristics. It is important to note that finding many bugs is not the only thing that a tester is valued of, which sometimes seems like a given.)
When I think of bug magnet it makes me think of someone generating a lot of bug issues/reports – but I don’t know if that is how you think of it. Or does there have to be a certain significance around the bugs reported/found?
(Henrik: I think that I see it as a mix of the two. Or rather a person that seems to stumble upon many important bugs.)
I’ve always thought of one of the elements of a good tester being a detective / investigative element. This might be research, but part of it is getting in the ‘zone’:
– What can go wrong in this domain
– What issues might the user be most afraid of with this product
– What assumptions have been ‘baked in’ to this product
– etc, etc
(Henrik: I agree.)
In your example there are lots of why’s – Pembleton didn’t say he could answer them either (or did he?) – so he wasn’t thinking like a criminal either. I think what Pembleton is using is his experience from previous cases – he’s pattern matching, using some lateral thinking and filtering (based on his experience and the facts of this case).
(Henrik: I cannot find the original quote from episode 3 or 4 where Pembleton says a similar thing the first time to Bayliss (who by that time is a rookie at homicide), so in the latter case Pembleton is teasing Bayliss with a reference to the first scene. My understanding of Pembleton is that he is aware of that he cannot fully think as a criminal, but he uses models of criminal minds in order to at least try to think as one.)
I’d say this theorizing (sp?) gives him the opportunity to be on the right trail – and he will filter that as his research (investigation) continues – things will be ruled out, other ideas added. But then in testing terms he is more fault localizing rather than finding new bugs (the crime being the bug in this analogy) – he’s got a bug and trying to narrow it down.
(Henrik: I would like to turn this around a bit. Think of the crime as the product instead. Investigating the crime is an exploration in order to get the (full) picture on what has happened – how it was committed, all the involved parties, the logic and reasoning behind the crime, etc. In testing we explore the product in order to dig up useful information about the product in a similar fashion.
However, I can also see it the way you saw it, i.e. troubleshooting – many times you use the same approach in order to solve the problem.)
Yes, your list under “can you improve it” would be something anthropologists would observe if they studied the species “good tester” – both for finding bugs and localizing them.
You drew attention to “in context” – you said this was important to say. I’d suggest it’s important if
– We’re not doing good testing. (Think of an experienced tester that ‘filters out’ a lot of information – they are implicitly relating to the context although to themselves they may just be doing good testing)
– You’re giving this information to a newbie (who might grab the nearest bug taxonomy)
(Henrik: An experienced tester might catch all bugs that a Bug Magnet catches, but it might take some more time. I think that this has to do with being able to rapidly filter out unimportant things. And in order to do that successfully, in my take on this, you use the context variables as your filter criteria in order to focus on what matters.)
I’m going to think a little more about this…
(Henrik: Do it, your thoughts are most welcome! 🙂 )
Good post!
One thing that is important to become a bug magnet is knowing the problems a developer introduce, as you mention. Related to that is the fact that all developers take shortcuts, thus increasing the technical debt. If we were able to communicate more openly about these shortcuts it would also add an extra flavour to becoming a bug magnet.
@Simon: Thanks for the comments and questions!
I have added my answers inside your comment, beginning with (Henrik: …)
@Martin: Thanks!
I agree, technical debt and the actions that leads to it, are major context factor that is really good to be aware of.
I think you nailed it in the ” in order to attract bugs you have to wanna see problems…” section.
But I don’t see the direct connection to the TV show, and the criminal metaphor isn’t the right one for me.
I’d rather go with something like having “green fingers” for plants, but rather the opposite; whatever you do, you end up with problems (which is a good thing as a tester.)
This can be explained in many ways; some are naturals, some are experts utilizing the Explore Data Relationships or Background Complexity Heuristics; some take advantage of error-prone machines; many have a curiosity that make you find interesting stuff.
Good post, and my definition of Bug Magnet would be – someone who often runs into software problems.
@Rikard: Thanks for the comments!
The analogy with thinking as a criminal is in one way “you wanna see problems”, but it is also “able to see problems that matter”. I don’t know too much about how criminals think, but for me this triggered something in my mind. A good crime (OK, this is in theory) is a crime that you can come away with without being caught. So it is not only something illegal, it is a holistic view of the crime where different outcomes have been considered. I.e. out of all possible crimes you could commit in a given situation, there is always some crimes that are “better” than others. Also, being able to spot many possible crimes in a given situation is to see the world from that perspective.
A good practice that you could do is to observe any situation in your daily life and think of all possible crimes you could do in that situation. Compare this with the thoughts you had the day before where you would not have thought of possible crimes. Could be interesting!?
I can also see that many would have difficulties with seeing the metaphor in the same way as I did; and the title of the post is of course exaggerated…
Great post and interesting comments! I am also a “fan” of the idea of looking at testers as detectives and investigators. A funny thing I come to think of is how many detective stories shows detectives finding their own way to solve the crime, knowing when they need to break the rules and rigid processes set up by their bosses to catch the criminals in the end. Perhaps this is a way to become a hero as a tester as well? 🙂
@Daniel: Thanks!
In addition to that, an interesting thing with detectives/investigators in the literature is that they are creative, subjective and adhocratic; which in my view is three very important qualities of great software testers.