I spent two days in Malmö attending a developer conference with a fantastic test track (put together by Sigge Birgisson.)

I did a presentation on Curing Our Binary Disease (slides, abstract), which was much better received than I hoped for (I thought it was a binary love/hate talk)
Good questions and talk about being inside the potato, how do you know where you are?

Pradeep and Rikard in front of Øredev projector

Pradeep Soundararajan talked convincingly about caring for the users, e.g. by using Twitter as a source for test ideas.
He also said the context-driven community has moved from Pass/Fail to “Is there a problem here?” (still a binary question though…)

Shmuel Gershon shared an experience report of a 100% exploratory testing project, with qualitative reporting and everything (“it’s called quality report, not quantity report”.)

Gojko Adzic talked about testers needing to adapt to how development is done now, and put most of their time ensuring others write good automated tests.
“There’s so much mistrust in the processes.”

Henrik Andersson wants, with right, a diversified testing team, and therefore only hires context-driven testers
Just joking, he said people can read the same excellent books, as long as they think critically and learn other things.

Selena Delesie explained how to focus the testing effort on customer needs; “testers are information radiators”.

Zeger van Hese made the presentation that I enjoyed the most. A thoughtful walkthrough similarities between the fine arts and testing.
“The Hungry Eye – thoughtfully looking at software”

Janet Gregory highlighted five enlightened areas since Agile Testing was released: feature acceptance, collaborative automation, large organizations, distributed teams, continuous learning.

Outside the venue I had many interesting conversations, there were testing games, food and drinks, Black Viper and even imitations, and all in all a very nice experience.
Thumbs up for a testing session with Pradeep and Shmuel that showed (the need for) different note taking styles.

The presentations will be made available online at www.oredev.org/

I have completed a trilogy on the theme The Eye of a Skilled Software Tester

edition 1: Lightning Talk, Danish Alliance, EuroSTAR 2010

edition 2: Article, The Testing Planet, March 2011 – Issue 4

edition 3: Presentation, Scandinavian Developer Conference, april 2011

Some things have changed over time; in the first two I didn’t focus on the most important “look at many places”, besides specifications we need to know about business usage, technology, environments, taxonomies, problem history, standards, test analysis heuristics, quality characteristics, and more…

I also did the necessary switch from errors/bugs to problems; because it is broader, and better pinpointing the psychological paradox: “want to see problems“, things that make Done further away.

While at it, I uploaded a presentation from SAST Öresund yesterday.
77 Test Idea Triggers, a presentation I’d be happy to give again!

Even if most of the program is set, there is still a chance for you to talk at CAST 2011. Matt Heusser and Pete Walen is running the Emerging Topics session:

If you would like to speak at CAST 2011, you still have the option of proposing a twenty-minute emerging topics session. (Emerging topics are anything you feel like talking about after the more formal conference program was decided.) Talks should generally be twenty minutes in length, with at least five minutes planned for discussion. If you want more time, please make your case and reasons clear, as that means other people don’t get to speak and we would need to make a special exception.

Read more at: http://xndev.blogspot.com/2011/02/casting-wide-net.html

Be there or be square!

Monday

Henrik started his journey by car from Karlstad and went down to Gothenburg to join with Martin. Both of us were going to take the train down to Copenhagen. Not surprisingly there were delays and the train was cancelled… Instead we headed back to Martin’s house and loaded Henrik’s car and took off. After several hours drive we arrived to Copenhagen, where we went directly to Bella Center and the TestLab at EuroSTAR.

This was the first time we met James Lyndsay and Bart Knaack in person; we had only talked with them over Skype/phone. So we all started with saying hello but there were really no time to sit down and have a cup of coffee – we needed to get to work straight away.

One of the burning issues to resolve the week before the conference was to bring client machines. Henrik was able to bring four laptops (thank you Compare Testlab and Know IT for sponsoring with this!) and Martin one. We worked in the TestLab for a few hours setting up and testing the two networks (James had one and Bart had another) with the servers connected to them.

Each network consisted of a bug tracker (Mantis), a server with OpenEMR, a server with a WebShop based on OsCommerce as well as a link to download a version of FreeMind.

When the conference center was getting near closing hours we rounded up and identified tasks needed to be done for the next coming days. We were assigned to setup and fill both the OpenEMR and the WebShop with new test data. We determined that we first should create a data model in FreeMind; both for reviewing before inserting the data and also so that we could print out the model and use it as a means to discuss with participants in the TestLab. With the model at hand, we thought the participants would also be able to identify test ideas of their own that they could try out.

We packed up everything and took Henriks car together with James and Bart. Half an hour later we were down in the hotel bar for dinner. We first met up with Steve Öberg, Rikard Edgren, Björn Karlsson, and Stefan Thoresson; and later on Markus Gärtner, Neil Thompson & Michael Bolton and some more joined our table and we were discussing testing long into the night.

Tuesday

At 8.45 we met up with James and Bart at the hotel and took a cab to the conference center where we setup the TestLab again. We started on the data models while James and Bart worked on setting up the details of the two networks. Meanwhile the sponsors were installing their applications on the client machines. Late into the afternoon we were starting to get ready with the new data for both systems. During the afternoon several people were visiting us and some of them started testing the applications. In fact, the first bug entered in the bug system was the one that won the prize “Best bug” – Markus Gärtner found it pretty fast…
Around 18:00 we were able to move into the room where the TestLab should be. We moved in all servers, routers and clients and setup the power for it all. Funny thing is that we didn’t find more than one electricity socket, so all machines were hooked up on this only one and we decided to load test the electricity over night.
During the evening we attended the Rebel Alliance (or Danish Alliance) for 11 interesting lightning talks. The discussions went on after that long into the night. But to our defense we must say that we got in bed in time.

Wednesday

Despite that we got in bed fairly early, we both overslept and missed the grand opening of the TestLab… Such a good start!
But no time to cry over that; we dug in and assisted in welcoming and introducing people that entered the TestLab and told them about it and how to use it; as well as guiding them to start testing. Bugs were reported and the sound of the chicken echoed all over. Nice!

During the day a lot of people were in and really sat down testing. There were also a lot of interesting collaboration going on between people that hadn’t met before. Some of the people that spent more time than others were Shmuel Gershon, Markus Gärtner, Isabel Evans, Michael Bolton, Ajay Balamurugadas, Zeger Van Hese, and Rob Sabourin (we might have forgot some of the names, but you know who you are).

During the day there were a lot of non-arranged  testing going on. But we had two really appreciated sessions during the day.
Firstly, there was a pair-testing session led by James Lyndsay where close to thirty people attended. There was a good mixture of experienced and less experienced testers. Even though the session was only about 30 minutes divided in three 10 minutes mini-sessions, it seemed to give the participants some very nice hands-on experience to bring home. And a lot of good collaboration was happening!

Secondly, during lunch hour it was an improvised weekend-testing session, Ajay Balamurugadas had invited as many as possible during his talk on Weekend Testing right before the session. Ajay and Markus Gärtner led the session with the rest of the participants that had showed up. Since there were so many that showed up we teamed up in pairs or smaller groups. The collaboration in the TestLab was yet again really great! After close to 30 minutes session we had found a bunch of interesting bugs while at the same time many have had the opportunity to team up with these fabulous  testers.

Despite that we had to cancel some planned sessions, we all thought that the first day was a real success . And we measured that by realizing that we didn’t have had time to take breaks or eat a proper lunch because the TestLab wasn’t empty for even a minute; there were at least 5 people there at minimum (except during the key note by Stuart Reid).

On the evening there was the Gala Awards dinner at Copenhagen City Hall and we ended up at a pub called Bryggeriet with the Rebel Alliance discussing testing in all forms. We went home pretty early and met up with some people in the hotel and continue discussing testing.

Thursday

The day kicked off at 08:00 in the TestLab and we used the first 30 minutes preparing for yet another interesting day!

At the AllSTAR testers, a few renown testers were able to join in. We split the TestLab into three groups where each focused on a specific area which they knew well. Each group attacked the systems in various ways, teaching the participants new ideas on how to test the systems. Ari Takanen was showing how he did some security testing (and/or fuzz testing); Rob Lambert showed how he did accessibility testing; and James Lyndsay had a exploratory testing session discussing a lot of test ideas with the participants.

At the round the table of managing ET, James Lyndsay moderated the group consisting of Rob Sabourin, Michael Bolton, Carsten Feilberg, Shmuel Gershon, Zeger Van Hese, Henrik Andersson, Henrik Emilsson, Martin Jansson, Markus Gärtner, John Stevenson, Neil Thompson and some more (we are sorry if we missed to mention someone, but we try our best to remember all people that were there out of those 200 TestLab attendees). We talked about several aspects around managing ET; and the discussion began with how we had managed our own time in the TestLab. James Lyndsay moderated the group in an excellent way. One of the subjects was how often you change/update your charters. Another interesting topic was on how much information you should provide different testers with.

During the lunch session, Markus Gärtner launched a Testing Dojo that was very appreciated. And Michael Bolton was doing a transcription of the session. The testing dojo meant that one tester tested the application with a projector that showed the computer screen. The tester then described through the testing and explaining what happened. All other around the tester could ask questions. The tester should be switched every 5 minutes letting several people show off their skills.
While this was the intention, the testing dojo slightly turned out to something different. But still as much appreciated by those involved.

Thursday evening we, Markus Gärtner, and Michael Bolton headed in to Copenhagen for a dinner with Ajay Balamurugadas, Teemu Vesala, Kalle Huttunen and Mika Hänninen. It took us two hours to get to the restaurant because we were forced to explore the Copenhagen Metro and train system…

Friday

We headed home to Sweden in the morning and arrived in Gothenburg at 15:00 in the afternoon. Then Henrik continued to Karlstad and arrived at 18:30. Seeing the problems other people had with getting out of Denmark by plane or train, we were lucky that car traffic wasn’t too affected by the blizzard…

Some numbers

• 180 people got in to the TestLab and got introduced to what we do (most of them sat down and tested)
• 100 hours of testing
• 100 reported bugs (plus several bugs not reported in the bug system)

Reflections

Considering how the TestLab was run, it was like a mix of having an open house and being a test lead handling several projects at the same time. We served different stakeholders such as the participants in the lab, the speakers, the conference personal and ourselves as apprentices as well as the masters (James and Bart). The plan is ever changing. Each stakeholder value different things. Just like in the real world outside the TestLab.

There were also 5 sponsor presentations from the tool vendors that had their tools on the client machines. We really appreciated those sponsors that had put in some time to really show how their tool could be used on the same applications as we tested in the TestLab. Very useful and meant that there were more value for those interested in the tools to try things out for themselves on the lab machines.

During the TestLab we got a lot of ideas on what we want to do next year. A lot of ideas that we had on beforehand were though only good in theory; seeing the real obstacles and problems in practice was a good lesson.

And finally, we really appreciate the great effort that James and Bart have done in creating the EuroSTAR TestLab and made it to such a remarkable success. We will have the benefit of building upon a very solid and well thought-out platform & concept!

We had a lot of fun and hope that all others had a great time!
See you all in Manchester 2011!

Cheers,
Henrik and Martin

EuroSTAR 2010 took place during the coldest Copenhagen November in 140 years.
Being a program committee member it was a bit different, but not a lot; I met a lot of interesting people, spent too little time in the test lab, and listened to thought-worthy presentations (+ 11 Danish Alliance Lightning Talks.)

The theme was “Sharing the Passion”, and during the presentations there was also a lot of mentioning of collaboration, trust, that testing is difficult, and that people are most important.
I was at an excellent full day tutorial with energetic Rob Sabourin, that is so right in his notions of multiple information sources and diverse test ideas that are generated all the time, but specified in more detail ALAP – as late as possible.
I saw parts of Michael Bolton’s Test Framing tutorial, that had a lot of audience involvement and resulted in a lot of notes put on the walls – beautiful.

I will not do any summary of sessions, but I have some quotes you might like:
“when you make a mistake, think: that was exactly what I needed to know” (Bolton)
“I’m teaching them to write the requirements in Latin, to remove all ambiguity” (Sabourin)
“I wouldn’t put my child in the respirator unless it has been tested exploratory” (Rydberg)
How do you follow up “Right from Me”? -”We don’t. We trust people. We need leaders that trust people” (Nordström)
“MTBS – Managing Testing Based-on Sessions” (Feilberg, opposite-method in order to make it easy to sell SBTM to managers)
Weekend Testing: “freedom for the testers!” (Ajay)
“don’t assume your view is the only view” (Thomas)
How can testers help developers go faster? (inferred from Harty)
“I have nothing against CMM. It just doesn’t work.” (Gerrard)
“Use examples, metaphors and visualizations to improve effectiveness and efficiency in testing” (Zimmerer)
“the industry is broken” (Patti)
“always have two 30-seconds ‘commercials’ in your back pocket” (Galen)
“choose a job you love, and you will never have to work a day in your life” (Confucius)

Last week, me and Martin won the competition “The EuroSTAR Test Lab Apprentices”!
Read more at: http://www.eurostarconferences.com/delegates/the-test-lab-apprentice.aspx

See you in the Test Lab in Copenhagen!

Cheers,
Henrik & Martin

It was Stockholm again this year. Good to not have to travel far, but since you are travelling I wouldn’t object to something more exotic, and warmer. Next year it is Copenhagen, again.
I had a full-packed program with 4 days of tutorials, workshops, tracks, short talks, test-labbing, conversations, so in total it is quite an amount of ideas since new things appear when you combine what you hear with your won reality and thoughts on testing. I am a bit exhausted after all this, which is a very good thing!

Monday gave Exploratory Testing Masterclass with Michael Bolton. Even though I would have expected a bit more advanced level, it was very good; here are some highlights:
it’s the scripted guys that sit and play with the computer; most of what we look for is implicit, it is tacit; develop a suspicious nature, and a wild imagination; checks are change detectors; some things are so important that you don’t have to write them down; codingqa.com episode 28 is important (I listened to it, but didn’t understand what was so important); managers fear that Exploratory Testing depends on skill, is unstructured, unmanageable, unaccountable; we need to build a “management case” (or should it be middle-management case?)
Michael showed an improved Boundary Value Analysis for a more complex example, where there are many boundaries; whatever focus for coverage you have, you will get other coverage for free; visualizing Test Coverage with sticky notes on a model is a good way of creating charters for Session-Based Test Management.
Go beyond use cases, create rich scenarios; emphasize doing, relax planning; Test Coverage Outline and Risk List to guide future sessions; don’t try to find bugs in the beginning, it takes time away from building a model; HICCUPPS + F (Familiar Problems); you learn best when you are in control of the learning process (and have fun); who said something valuable should be that easy?
Reports to make number people happy; SBTM debriefs are important for keeping quality of the testing and the report (and good for coaching and mentoring); the principal interrupter of testing work is bugs; Weinberg: “everything is information”; Dr. Phil: “How’s that working for you?”
He also had good exercises, and a nice movie, a Detective Story.
I haven’t been to Michael’s tutorials before, so it was about time.

Tuesday started with Tim Koomen tutorial “From Strategy to Techniques”; there’s a gap between the test strategy and the actual tests.
He is very knowledgeable, and walked through the basic testing techniques that every tester should have in his toolbox: Equivalence Partitioning, Boundary Value Analysis, Classification Tree Method, Pairwise, Path Coverage, Condition/Decision Coverage, Input/Output Validation, CRUD, Operational profiles, Load profiles, Right/Fault paths, Checklist.
The examples are focused on functionality, and a magazine discount example is shallow; it doesn’t consider if the person is just about to become 20 or 65 years old, or if you don’t know the age, or if an incorrect age is corrected. And now we haven’t even considered everything else that interacts with this small piece of functionality.
Every time I see this list, I think that they don’t sum up to the testing techniques I actually use when I design tests.
So my highlight was this feeling combined with my shallow knowledge about Grounded Theory; maybe we could have a super-advanced error guessing test technique, that describes the really, really good test design that happens all over the world, where we are looking at a lot more things than the requirements (more to come on this…)
Tim showed PICT tool (consider 1-wise!), and audience mentioned that Mercedes-Benz also has a free tool (see pairwise.org for a long list of tools)
I learned a new thing: the modified conditional coverage, where you omit tests that aren’t likely to catch errors.
Sometimes I wonder how many of the tests from the classic test techniques that preferably are automated in unit tests.

The actual conference started with Lee Copeland talking about nine innovations you should know about: Context-Driven School (the search of best practices is a waste of time); Test-Driven-Development (help you write clean code); Really Good Books (too few testers read the books!); Open Source Tools; Testing Workshops (Specialized focus, participatory style); Freedom of the press (He is no fan of twitter, but like blogs); Virtualization (Rapid setup, state capture, reduced cost); Testing in the Cloud (rent an awful amount of machines, very cheap); Crowdsourced Testing (Lee did not mention the ethical payment dilemma)
“sincerity is the key – once you learn to fake it…”
Keys for future innovation: creative, talented, fearless, visionary, empowered, passionate, multiple disciplines. Do we have all of these???

Johan Jonasson explained Exploratory Testing and Session-Based Test Management, but since this was a short track, there wasn’t so much time left for the real juice. “ET has specific, trainable skills” (Bolton)
Julian Harty, Google (where the testers seem to have huge responsibility areas) explained the concept of Trinity Testing, 30-90 minutes walkthrough(s) by Developer, Tester, Domain Expert. Not radically new, but it felt very fresh and effective. Julian was the only one I saw that brought a hand-out, one paper with the essentials.
Geoff Thompson talked about reporting, that “it’s the job of the communicator to communicate.” 1/10 of men (1/50 for women) are color-blind, and maybe you want everyone to understand the report? (I saw two other presentations, where red-green was used to highlight important differences.) “Know your recipients, what information do they want?”, “honesty is always the best option.”
Michel Bolton had a short session on “Burning Issues of the Day” that is available here. Very funny, very thought-worthy, very good.
Jonathan Kohl talked about Agile having lost a lot of its original value, it is re-branded, old stuff, and has become business. Process focus can distract from skill development, the point is: focus your work on creating value.
I asked Jonathan afterwards about Session Tester (where not much has happened lately), and he said that the programmers are too busy, but it will happen things pretty soon.

Wednesday’s first keynote was Naomi Karten about change; change that represents the loss of control, change that we often respond to in an emotional and visceral way.
Hofstadter’s Law: It always take longer than you expect, even when you take into account Hofstadter’s Law.
Regularly communicate the status of the change, also when you don’t have any news, or when you’re not allowed to tell the news (say that you can’t say anything!)
Listening and empathy are the most important change management tools.
The biggest mistake is to forget the chaos; and in chaos: don’t make any irreversible decisions.
This was my favorite keynote, and as I’m writing this I understand there was some really important information in the presentation.
Mike Ennis talked about software metrics, that help you manage the end game of a software project.
The end game term is taken from chess, where the outcome is almost decided, it is just a matter of technique, primarily about not making mistakes. Mike used the analogy that if you can anticipate what will happen, you know what to do next.
He defined example release criteria, which often aren’t met, but business decisions can overrule the criteria.
40% of the code is about positive requirements, “not a huge fan of exploratory testing, do it if you have time, after the standard tests have been run”.
He used a Spider Chart (aka Radar Plot) to visualize The Big Six Metrics (Test Completion Rate, Test Success Rate, Total Open Defects, Defects Found this week, Code Turmoil, Code Coverage.)
A question was raised that there is a risk of over-simplifying things, and the answer was: “Yes, but these are indicators only.”
Erik Boelen talked about Risk-based test strategy, if you do it with different roles it is like Läkerol, it makes people talk.
He likes games, the we-versus-them game with developers is good, at his place developers with many bugs buy drinks to the testers; and testers aren’t allowed to say one word for a week; last week that word was testing…
A very interesting and nice thing about the presentation was that he explained their (very good, but for some, very provocative, I assume) test method as a natural and obvious way:
They take the entry paths from the Risks and perform Exploratory Testing. For High and Medium risk they document test cases as they explore, and for Low risks they just report the results.
“Eventually testing will rule the world.”
Shrini Kulkarni talked about dangerous metrics, and that software development must consider where it is suitable with measurements. (Shrini hates SMART by the way, so I like him.)
A root cause is that metrics/measurements represent rich multi-dimensional data, there is inevitable information loss.
People might say “we can’t improve without metrics”, but you could use metrics as clues to solve and uncover deeper issues.
We can report with stories attached to the numbers, but still, we are losing information.
Susan Windsor had a double session on communication styles where time flied. In the audience, everyone said No to “Exploratory Testing adds no value”
Art of Storytelling involves: Random, Intuitive, Holistic, Subjective, Looks at wholes (two of my favorite adjectives!)
Research shows that interviewing is the most ineffective method when hiring.
She noted that a high proportion of testers also do creative things like music, poetry (which seems natural, it is good to have trained a lot at being creative.)
We looked at four different Personal Communication Styles (why is it always 4 different types of persons??): Strategist, Mediator, Presenter, Director.
Gitte Ottosen had the ending keynote of the day with a presentation about combining Agile and Maturity Models (“CMM = Consultant Money Maker”)
“Metrics, I know they are dangerous, but also necessary.”
Manual Testing involves using the story to do exploratory testing (“continuous learning as we implement the feature.)

On Thursday I was wise enough to skip 2 sessions in order to have a late breakfast and practice my presentation.
So the first presentation of the day was Zeger van Hese (he won the best paper award for the second time this year) that shared his experiences of introducing Agile, but only doing parts of the full-blown, capital A stuff (resulting in a Real-world, semi-Agile process.)
They used this strange mix of Waterfall and Agile that many, many companies have, and got a better and better situation as more members of the team sat in the same room.
But in the end they fell back to old behavior, there were many late changes, many Release Candidates, and a one month delay. But; excellent quality and stability.
3 Agile goals: better feedback, faster delivery, less waste.
They did a big Agile no-no by using manual testing, which seems like a wise deviation to me.
A quote attributed to Einstein, and several others: “In theory, there is no difference between theory and practice; In practice, there is.”
Next presentation was my favorite of the whole conference: Fiona Charles, Modeling Scenarios with a Framework Based on Data.
They built a conceptual framework at 2 levels: an overall model of the system (testing), and the tests to encapsulate in that model.
They did a structured analysis of all attributes for each framework element, and then used these attributes to build simple, and then more complex, scenarios. This is difficult to do for many testers, so careful review of this work is a way to make sure the results are good.
I think this is an example of the test design technique I thought about on Tuesday, a very advanced, structured way of designing tests that can’t be captured by the classic test design techniques (error-guessing is closest, but there’s a lot more to it.) I like to call this Grounded Test Design (more to come on this…)
“scenario testing is a nice thing to add to your repertoire”, “combine two or more models”, “don’t ever fall in love with your model”; they found 478 bugs, and all except 20 was essential to fix for the customer.
What you need to do something like this: testers with domain experience, business input and scenario review (and maybe an industry book), a model, structured analysis.
After lunch, I had a second session in the Test Lab, so I could report some of the bugs Zeger and I found the day earlier. It was great to test on real stuff, but I didn’t have the time that I would have liked in order to understand the product and its failures. There weren’t time (at least for me) to discuss in depth the findings with other testers, which is something I hope to be able to do next year (I’m hoping the Test Lab will continue.)
At the last presentation slot, I did my thing on “More and Better Test Ideas”. People were tired, but looked interested, so I’m happy with the presentation. I won’t recapitulate the session, but I did talk about the potato, but had to skip the new Find Five Faults analogy (unexpected time pressure, I’m still in doubt that I got the 38 minutes I was supposed to.)
The paper is available here, the presentation here, and it will be given as a EuroSTAR webinar at December 15th.
Good questions, and also examples of how similar approaches are used by others. A bit more than 10% of (almost 100?) attendants use test ideas/conditions.
The next day I got a mail stating that ideas from my presentation could be used at once; the best feedback to hope for.

The Test Lab organizers (James Lyndsay and Bart Knaack) seemed happy when presenting the results, and it’s good to know that the efforts might make open-source medical product OpenEMR a bit better (there is certainly room for improvements…)
At the final panel debate half of the audience voted that certification is important, Tobias Fors shared the insightful “as a developer I was scared about code review, but then I realized it really was about my low self-esteem.”
Regarding teaching testing in school, it was said that critical thinking should be taught early.
“How do we breach the barriers and invite the developers to our world?”
Dorothy Graham (who reviewed every presentation!) ended the conference and announced the next years programme chair John Fodeh.

Overall it was a very nice conference, at the expo Robert from ps_testware was nice and let me win a chess game this year also.
Recurring themes were Agile/Exploratory Testing (why are they grouped together?) and now and then the importance of a Story was emphasized.
Unknown source: “The higher and more complex quality objectives you have, the more manual testing is needed.”
Attending a conference isn’t about learning truths from the experts, it’s more about getting input to be able to create your own ideas that apply to your job, and to meet people, hear stories, interact with people that share your passion: software testing.
See you next year!

/Rikard

A cheer to those of you who were able to attend this conferance: http://www.waterfall2006.com/

My favorites:

http://www.waterfall2006.com/crispin.html

http://www.waterfall2006.com/jeffries.html

Thank god everyone is not a believer of the hype around the Agile Movement. Process is king!

I spent one day at Øredev 2008 (http://www.oredev.org) since they invited me to give the Where Testing Creativity Grows (http://www.thetesteye.com/papers/where_testing_creativity_grows.doc) presentation.
I arrived ten minutes after the start of James Bach’s keynote The Renaissance Thinker, where he argued that 1972 (Chapel Hill) ruined good software testing.
People started focusing too much on templates, processes, best practices; people forgot about people.
But things are getting better since the Exploratory Testing has been coined and used with more respect.
We also have Agile that is using cyclic learning, in opposite to Waterfall and V-Model where no learning is modeled.
He likes it if people say “James, maybe you are one of the mystics?” and explained his self-learning style of Buccaneer-Scholar.
“Software Engineering is primarily social.”
He has a lot of good things to say; and he presents them in a capturing manner.

This conference is nice since it includes developers, testers, project mangers et.al.
But I stayed at the Test Track all day; starting with Isabel Evans, who is interested in processes and people; and stresses that it is the people in the team, that needs to define the process.
Quality is difficult, since it has so many attributes; the customers view being the most important.
Isabel is criticizing the “standards” from the inside, but think it is important to measure things and report back, e.g. by showing how much money is saved by doing software testing.
“Agile works since it is cross-functional; we work together.”
She also broadened the spectra of our risk thinking; which too often focus on internal risks. But there are Contractual or Regulatory Risks as well as Social Ethical Risks. It is good to think bigger.

Then it was time for my own presentation with about 60 attendees. I had an interesting start with technical problems as my machine wouldn’t use the projector after I launched a movie inside PowerPoint (Yes, I use Vista.)
So I had to improvise a bit and focus on the audience instead of my slides.
After this the presentation went well; and I was happy that an attendant could tell about their paired testing (both scripted and exploratory) where they found three times as many bugs (plus the learning!)
A good question after the presentation was “When do you know you have used enough creativity?” with the answer “Well, you don’t…”

After a lunch break Kevlin Henney talked about “Know Your Units”, with the clarification that a unit test that interacts with something (file system, database, network) isn’t a real unit test.
Programmers are responsible for unit testing (“The programmer who wrote the code is in the best position to test it”) and ”QA” for system testing. (He didn’t mention any collaboration, which I think might be the best.)
POUTing -Plain Ol´ Unit Testing – is passive.
TDD – Test-Driven Development – is active.
DDT – Defect-Driven Testing – is reactive.
And sometimes we can settle with GUTs – Good Unit Tests.
He recommends a black-box perspective for unit tests, where you focus on behaviors, or even requirements. White-box testing can end up testing that the code does what it does.
“We say that system testing takes one month, but in reality, the testing takes a few days, the rest is spent fixing bugs.”
“Testing is a way of showing that you care about something.”
Kevlin is one of many presenters that seem to think and talk better while walking.

Mattias Göransson from Sony Ericsson presented how they have started to use Exploratory Testing with Session-Based Test Management with a Heuristics twist (the original title was “Heuristic Based testing”)
He had borrowed material from Michael Bolton (who has borrowed from James Bach), “with pride”, and explained the Heuristics concept and the flavors Guideword, Trigger, Model, Process.
“We spend more time on test execution and – SURPRISE – we find more bugs.”
They have about 25.000 requirements on a mobile phone, and uses test cases for the major areas, and Exploratory Sessions with Heuristics to get the depth, and higher coverage from the user’s perspective.
Unfortunately he wasn’t allowed to share their heuristics, but he told about the fantastic Staffanstorp Heuristic: if the phone works at a particular place in Staffanstorp, it probably works well almost everywhere.
He explained their heuristics as “experiences they have gained” and pointed out that the heuristics are reviewed after each session.
Other tips: Time-boxing of debriefings is very important.

Nikolai Tillman of Microsoft presented PEX (http://research.microsoft.com/Pex/); a tool that helps you explore single-threaded .NET code by automatically creating unit tests.
The tool analyzes the code and creates representative sets of input values; with 100% Code Branch Coverage.
I think this can be a really good help in getting started with some basic unit tests (and avoid null exceptions); and also a way of learning special cases in code you are re-using.
As merit it should also be mentioned that PEX has been used in .NET components, which resulted in bugs that were fixed.
The tool exists as Academic license for Visual Studio 2008; and early versions of Visual Studio 2010.
“dynamic symbolic execution”

The last presentation of the day before my train left was Pradeep Soundararajan (http://testertested.blogspot.com/) from India. He told his own stories, about how he got fired from a company because he didn’t meet the 95% pass rate of test cases (He found too many bugs!); about very successful testing in a team; and about changing the culture at a company that ran 3000 test cases every week, but never found any bugs.
A very nice end of the day from this member of the context-driven community.
“What is everything? It is secret.”
“Think about the customer’s customer”
“Scripts make people go mad, and sad”
“Using good practices in the right context”
“The brain is an important tool that you should consider using.”

It was a nice day at a nice conference which also included a discussion (or lecture?) in the hallway with James Bach about the schools of testing concept. I was pretty upset with him after some e-mail stuff, but in person I was pleasantly surprised. In his eyes, you can see that he is a really nice person; a main reason for his provocations is that he learns best by arguing; and probably thinks that other people does that as well.

Well, this was enough conferencing for me in a while; now it’s back to the wonderful world of parental leave.

This years EuroSTAR took place in den Haag, a city that had quite some rain, but also beautiful autumn leaves, and big churches.
The theme of the conference was “The Future of Software Testing”, and a recurring image was a traffic sign informing that the future is next turn to the right. I always thought the future was right ahead of us.

James Whittaker started with a really good and entertaining keynote where he argued that software is extremely important, and that the standard software testing will disappear (see http://blogs.msdn.com/james_whittaker/ for details.) On the other hand, he didn’t mention users or value (as Michael Bolton pointed out at the next sessions (sic!) questions), and he didn’t mention that testers do more than just verify requirements on different environments. We perform low-level and high-level validation and verification simultaneously.

Next I saw Derk-Jan de Grood who presented a mix of Exploratory Testing and structured process-focused testing. It didn’t feel revolutionary, since we Swedes have known about both these concepts for quite some time now.
The Austrian Strasser was a pleasant surprise. He told about Borland’s route from V-Model to Agile, and he stressed the people aspect many times. E.g. the teams themselves could decide if they would be split in smaller groups, and how. They have come a long way, and has a lot of unit test collaboration between developers and testers. “1 + 1 = 3″.
Tuesday ended with a keynote from the Google guys behind Testing On the Toilet (http://googletesting.blogspot.com/2007/01/introducing-testing-on-toilet.html) who has a really good story to tell. And it is a lot bigger than the Toilet, they have done a lot of things for more than three years in order to raise the awareness about testing at their company. A big accomplishment that I think is difficult to repeat at companies that don’t have “20% Time” (At twenty percent of your working time on Google you can do something completely outside your normal activities.)
After the presentation they were asked: “Do you have any objective metrics that can show your results?”
And I was really happy about the answer: “Metrics are difficult, there are so many factors involved. We rely on anecdotal information.”
They also stated: “There will never be a time when you don’t need a solid QA group.”
Wednesday started with Randall Rice sharing his thoughts on the trends that may shape the near future. Some general and good information, but not so much to write home about.
He recommended howtosplitanatom.com for articles on Web 2.0/3.0 and he said that a drawback of SaaS (Software as a Service) is that customers won’t have any control over which version they are running.
Julian Bensaid talked about outsourcing, and how to avoid, and quickly get out of “The Damage Zone”, the phase when thing are going a bit too bad for too long. He stressed “One Team” and said that “You have to innovate to make outsourcing successful.” “Devil’s Square”: quality, time, functionality, money. At question time, he let the audience answer their questions!

Then it was time for Michael Bolton’s double session workshop on Heuristics, which was the next best event for me at the whole conference. He really knows how to talk, he has important things to say, and the exercises stimulated good conversations with the other attendants. (The results are available at http://www.flickr.com/photos/23076491@N02/sets/72157608997920373)
Also, a good presentation stimulates your thinking to new ideas; and I thought of a new definition: “A good tester knows when to break the rules” and I’m wondering if some very good heuristics could be created at your company, where you have the details.
“Maybe testing isn’t supposed to be easy”, “We have to look complexity in the eye”, and he recommended positivedeviance.org.

Next it was my own presentation: Testing is an Island – A Software Testing Dystopia(link!). It went really well, except for one moment when I just couldn’t find any words. I excused myself, took a sip of water, and just started talking again; and after that I really can’t remember exactly what I said. I remember I said the important things, that I referenced some earlier presentations in the way I intended, but I can’t remember any details. I’m pretty sure I explained something in a completely new way that was really good, but I don’t know what it was. I see this as a good thing; my mind was in a state of flow.
Of course you only hear from the positive ones, but it seemed like people agreed that the human element is very important for us testers. I was also happy to hear that my presentation was very different from the other sessions, and that they like this different, thought-provoking angle.
I guess the introduction movie Mårten Ivert and Henrik Emilsson did was very helpful, just because it is so good!
One question afterwards was if/how I could convince people who think that emotions is a bad thing when testing. I said something about not having any proof, but that I might try by saying that we like emotions in life; so why shouldn’t we have them at work as well (Work is a part of our life.) Testers should also in some sense mimic users, that has feelings. I didn’t think about referencing Boltons excellent lightning talk on Emotions (http://www.developsense.com/2007/05/lightning-talk-on-emotions-and-oracles.html)
There was also a questioning regarding Agile development contradicting my dystopia; wasn’t that a way towards testers being more integrated in software development. I agreed, but also said that if the testers are brought in, in order to create unit tests instead of the developers, maybe it’s not totally positive. We need to look at many aspects.
Another good input was that testers who changes projects and companies, get a more holistic view, since they see more different things.
That’s a valid objection, and my defense was the team aspect, which is lost if people switch companies often. Best of both worlds might be to spend some time at a different place, and then come back.
Questions are the best part of presenting your ideas.

After this I went to a “Manifest Workshop”: Can the Past tell us the Future? This was nice because you got to spend some time talking to people about what has happened with our profession over the years. There are a lot of agreement and good ideas, but you also hear some things you can’t agree with, e.g. “Testers should have the mandate to change”, “Testers must have a special education with a title” etc.
This Manifesto extended over several workshops, and will be available at www.softwaretestingmanifesto.org; but I didn’t sign the acknowledgement list, even though they had an idea of “Anti-Manifesto” statements.
Wednesday ended with a casual session that generated some laughs: The EuroSTAR Testing Quiz.
Erik van Veenendal started Thursday with a keynote on TMMi. This is not my favorite subject, since “process-only” doesn’t say anything about how good the content is.
So I was glad that Erik started with saying that process isn’t enough; you need skilled people and a good infrastructure as well.
I also liked his statement that if you can take a Test Policy and apply it on a similar company; then the Policy isn’t worth anything. 
To succeed with a Process Improvement you need to have Management on board; and unfortunately they only listen to numbers (how long will that be true?)
You also need to spend a lot more time than Friday afternoons (Erik said 60% was necessary); and you need to focus on why you are doing it; what isn’t working good enough?
But Erik sometimes goes back to the Dutch formalized approach: the testing lifecycle should be Test Plan -> Test Design -> Test Script -> Execution.
“People want to change, but they don’t want to be changed.”
It was a good presentation, but unfortunately Test Process Improvement schemes aren’t generally good; most of all because it lacks aspects like creativity, and the ability to find all important errors.
I didn’t ask if there is a risk of becoming too mature, over-ripe, rotten.
On the other hand; you can read TPI, TMMi etc. and get really good ideas on what to do.

Egbert Bouman had presented his five favorite inspirational management Gurus, and how they could be applied to testing.
E.g. “Make company and personal interests run in parallel”, “People are complex.”
But I think it might be dangerous to transfer ideas from different areas without critical thinking. For instance, Collins Hedgehog Concept is about a company in the market that stick to doing what they are great at.
But in testing, we aren’t market-driven, we rather provide services/information to the project; and wouldn’t it be a bad idea to test software in the same way year after year?
He agreed a bit on this at the Gala Award Dinner, but also said that it differs if you are employed by a company, or if you are a consultant. But we discovered that we both were fans of Kierkegaard, and Morten Hougaard joined in this appraisal. “The truth is the subjectivity.”

I skipped Stuart Reid’s workshop on ISO29119. I was interested in how they are working, but I was sure I wouldn’t like the result; too much focus on standardization is killing creativity.
Tim Koomen jumped in at last minute and gave an overview of the testing industry in the past, present and future. He has a good understanding, and talked a bit about Pairwise (I don’t like it), Exploratory (I like it) and Model-based Testing (Surely good in the very few situations it is truly applicable.)
He also had survey results showing that only 1/3 of software testers are using Test Design Techniques (on the other hand; who isn’t implicitly using Equivalence Partitioning all the time?)
“Let’s use the existing techniques a little bit more.”
It was also nice that he brought up the classic Triangle Exercise.

John Kent have created a better Entity Model for Software Testing. “All models are wrong, but some models are useful.”
He used the name Test Condition for Test Ideas, which I think are central in our universe.
He showed a Test Case example, and I really feel sorry for all those testers that has to write or follow Expected Results for each and every test step in a Test Script.
He had a Real World Waterfall Model that started with “Produce requirements which are incomplete and slightly wrong.”
And he ended with the idea that in some situations you might better throw out these Entities and go for only Test Ideas; the tester will know what to do.

After lunch break Michael Bolton gave probably the best testing presentation I have ever seen: Two Futures of Software Testing.
He was wearing horns when talking about the Dark Future that is static and mechanic, and a halo when talking about the Light Future that has the testers human skills in the centre.
His message is very close to my own, with the difference that he has a lot more to say, and he does it extremely well.
It was so good, so I didn’t have time to write down any of all the good things he said, an old version of the presentation is available at http://www.developsense.com/presentations/twofuturesofsoftwaretesting.pdf.

The last keynote was James Lyndsay: Becoming Agile – Reshaping Testing for an Agile Team.
Agile exists, it is growing, and often succeeds with its projects, independent of the inclusion of testers or not!
But the testing performed is confirmatory, and that isn’t enough, so the tester approach is needed.
And in some ways; Agility and Testing is conflicting. Agile is highly optimistic, and testing is perceived as the opposite. Testing is used to be independent, must now be part of a team commitment. To explore for dangerous information might need feel good.
It is not necessary for a tester to know programming to enter an Agile team, but it is an invaluable asset.
He showed some statistics from www.ambysoft.com/surveys/agileFebruary2008.html.
And his main point in a very good presentation was that testers should help the team learn more about what they are building.

The conference was concluded with dinner in Grote Kerk where I sat next to a penetration tester; and beers at a pub, which really encourages many conversations.

Top 3 themes on the conference as I saw it was: People, Complexity and Agile.
EuroSTAR is a good conference, it has a nice aura of knowledge sharing.
Other things: Hoegaarden in Holland has so much more aroma than in Sweden.
Stuart Reid was walking around with a yo-yo.
My main attraction on the EXPO was the company that had a chess board. I played a game, won, and was thereby included in a lottery; and since only one other person won, I had the 50% luck of actually winning an IPhone!
I didn’t have the chance to see: Mieke Gevers, my nice Track Chair; The good title “Imagination is more important than Knowledge”; Fabian Scarano who in his paper and presentation mention serendipity.
It was a tie between really good Mexican and Argentinian dinner.
Isn’t it true that good testing should create low-hanging fruit?

Next Year EuroSTAR will be held in Stockholm, again! A bit boring; Barcelona seems like a more exotic place at least for me.
My current idea is to create a presentation consisting only of images; have a good name like “7 Ways to Explain Software Testing”, and then write a fluffy and nice abstract…
Or maybe try titles like Test Ideas – the Center of the testing universe; Generic Test Ideas for Data Types; What I Learned After 9 Years At Spotfire; The Holistic-Subjective School of Testing; The Eye of a Skilled Software Tester; Verification, Validation, and Everything In Between.
/Rikard