thoughts from the test eye

To add to the list of testing questions you could ask for this exercise:

Is this a good test for finding a good developer? How would you know?

As a testing exercise, I don’t think I would add that question.
It is interesting, but I’m not sure testers should be involved in those judgments.

But thanks for a 5th question:
Is this exercise good for finding good testers? How would you know?

If a tester was to ask “what is the point of this exercise?” rather than just diving in and trying to find bugs then they would get a +1 from me.

As a test exercise – not that valuable, really simple to automate and test all values

Otherwise, it’s the same sort of thing as the triangle test or ParkCalc, the most interesting part of the exercise is what questions a tester would ask ( or not )

The automation would be one of the main points for me in a debrief, but I would also be interested in how they did it, and what test coverage it gives (e.g. what does “all” in “test all values” mean?)

I agree with the questions being very important, so here is the answer to yours:

What is the point of this exercise?
The point of the exercise is to practice testing skills like asking questions, acting on answers, test strategy, modeling, test design, test execution, result interpretation, test reporting.

I think this would be a good example, like knowing the requirement of what it does and what can be the scenarios can be written for it . And how many test values can be entered. – If the test is to take for 10 mins and write down the test values (This would be interesting)

I think it could be an interesting exercise. I have seen it before somewhere, but can’t really remember where.

There are so many aspects you could talk about, as you have pointed out in the comments already, with regards to strategy, design, interpretation, reporting and so on.

Limiting yourself to automating valid (1-1000) and some invalid (0, 1001, b, null, 10204343433434, whatever) inputs misses a lot of interesting questions outside of the functionality aspect. Performance, usability, reliability, etc. can all be interesting topics to discuss.

Good find.

/Johan

Srinivas; not sure if you actually mean that it mainly should be a “writing” exercise, i.e. more thoeretical about values to enter.
I would prefer that it is hands-on, that you actually use the values, and observe results (having a physical aspect makes you learn more, in y experience.

If someone did that in an exercise for me, I would ask them if they are not too focused on the functionality and the explicit requirements.

Johan; it seems to be a pretty well known programming exercise, and also a kids game (even though I hadn’t heard of it before.)
If you find it as a testing exercise, I am interested to hear more!

Only doing funcional testing would, as you suggest, be another main point in a debrief (or during the exercise if appropriate.)
But it would not only be interesting to “discuss” performance, usability, reliability etc; I would like to see which ones you find interesting and what aspects of them are relevant for this software (this should be in the test strategy.) I would like to see which tests you designed and executed, what the results were and how you report them.

[…] Exercise: Lateral Exercise V: FizzBuzz – Rikard Edgren – http://thetesteye.com/blog/2014/01/lateral-tester-exercise-v-fizzbuzz/ […]

I realize the original exercise might seem boring, because of the simple functionality. So I think the instructions should be explicit about broader perspectives, e.g. by stating:

The functionality is so straightforward so you will have time to investigate other things as well.
Your testing mission is to find any threats to this being a useful exercise for testers around the world.

If you give the exercise a go, feel free to send me a mail (maybe available at About page?) with questions or results.

Sorry for this late reply…
Just to share my thoughts about black box testing being kind of week on testing a formula.

As a tester I would probably do the following things:
– ask for or decide on some range: each system has limitations
– validate the system black box with valid and invalid classes, to see the system’s ability to input and output stuff (and doing that functionally correct)
– test and/or inspect the code to find out: if it works for one value, what is the risk of failing on other values? (in other words: how is the formula/algorithm implemented)

Automating black box test to check the validity of a formula within all values of a certain domain feels somewhat silly to be honest. But if you have no access to the code: better than nothing.

And I agree, the testing program itself is less important than the discussion about the testing strategy.

Hi Kees

Thanks for good suggestions.
No replies are too late, I sometimes comment on my own posts a year later…

Actually, the code is available, so there are opportunities to use it.
When I did the exercise myself, I did 2 solutions for all valid values, one outside/black-box and one inside/unit tests; both with their pros and cons.
For error handling, I prefer to put focus on the end user product (due to the actual implementation in this case.)

But the calculation is one thing, what are the test strategies for other quality characteristics, which are relevant, and why?