Soft crashes give a false sense of safety Henrik Emilsson
At my former company we had a vision of removing/fixing all unhandled exceptions that we found (and that we thought a customer might encounter). After some discussions we decided that the unhandled exceptions should be treated in the same way as we had treated severe errors on previous platforms. I.e. Crashes in C++; NullPointerExceptions in Java; “Runtime Errors” in VB; and “Script Errors” in JavaScript.
This was a philosophy that both testers and programmers agreed upon (more or less) and it fitted our context.
Now, for the last two years I have been working as a test consultant; and during these years I have been on a couple of customer sites where their applications have been developed in C# .Net.
When unhandled exceptions occur, many team-members and users that I have been observing just click “Continue” and continue with their work as if nothing happened. And when I afterwards have asked them why they did so, it appears that they never reflected upon the error and if it might have been severe.
Many say that they don’t see the unhandled exception as a crash, or “Soft Crash”, instead they have seen it as a warning or an error that “really shouldn’t be in there”; some people see it as an error that in fact is handled, hence the error message and the pop-up dialog box.
Sometimes I point out that after an unhandled exception you cannot be sure about the state that the application is in, and that any behavior afterwards might be affected by the first “crash”. But I rarely hear anyone else saying this.
I am not saying that hard crashes are to be preferred in any way, but then you at least understood that something was severe. 🙂
I think that soft crashes give users/developers/testers/stakeholders/etc a false sense of safety.
And I blame the “good handling of unhandled exceptions” that the .Net framework provides. In fact, I blame the “Continue” button of those dialogs.
What are your experiences and thoughts regarding this?
(This was originally posted as a new thread on the Context-Driven Software Testing list:
http://groups.yahoo.com/group/software-testing/message/5794 )
I agree, you don’t really know the state of the software after an unhandled exception.
But you shouldn’t blame the .NET framework; rather it is up to the team making the software to make sure it is robust, well-tested and well-fixed. Should users really have to decide if they should restart the software?
Maybe it is a trend of “we can probably get away with this”; the last years my impression is that script errors on the web has increased, probably because default is to just show an icon in the bottom left corner.
But this will change in a dozen of years or so, when more serious errors occur more often, and people start realize that it doesn’t have to be that way; software can be reliable.
I like your observation about script errors; and I agree about that trend as well.
I guess that you are true about who to blame. But if the .NET framwork hadn’t provided the “error-handling”, or if internet explorer hadn’t supressed script errors, then the development teams probably couldn’t get away with things that easily? (Chicken or the egg!?!)
But I must confess that it was a lot easier for me to blame the framework in a sligthly cynical tone, than to blame development teams for not making robust software. 🙂
I think the continue button is a wonderful thing. I would rather have that instead of the hard crash where the application closes. Still, the text should read … “This is an unhandled exception. We have no idea what state we are in and things might eventually crash, so please save as quickly as possible save as a copy of your original file, restart the application and then verify that nothing was affected.” It is better than a hard crash, but in many cases it will leave you in a state where you do not trust the work you have done.
If I remember correctly the reason for not making a general catch of errors is that you wish to know what each “unexpected” state means so you can give the appropriate message. So in a sense it is good that you wait with fixing the unhandled exceptions till you know what is wrong, but when it comes to the end-user all of these should have been found. Thus, they should not be treated lightly since they as you say “give a false sense of safety”.
So why should someone try to save something that not even the application doesn’t know the state of?
In many cases you will save something really corrupt and in many cases it is no longer possible to load the same file again.
No, I think that these errors should have been found and fixed before a user in need of saving something bumps into this.
“I think that soft crashes give users/developers/testers/stakeholders/etc a false sense of safety.
And I blame the “good handling of unhandled exceptions” that the .Net framework provides. In fact, I blame the “Continue” button of those dialogs.”
How could this give any good developer or tester any feeling of safety?
If you are of the opinion that unhandled exceptions must be fixed, then a soft or hard landing shouldn’t matter.
If the developers don’t think that it needs to be fixed – shame on them.
If the testers don’t write up a bug report – shame on them.