<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>thoughts from the test eye &#187; performance</title>
	<atom:link href="http://thetesteye.com/blog/tag/performance/feed/" rel="self" type="application/rss+xml" />
	<link>http://thetesteye.com/blog</link>
	<description>by rikard edgren, henrik emilsson and martin jansson - with torbjörn ryber and henrik andersson</description>
	<lastBuildDate>Sun, 13 May 2012 17:27:07 +0000</lastBuildDate>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=3.3.1</generator>
		<item>
		<title>Lightweight Performance Testing</title>
		<link>http://thetesteye.com/blog/2012/03/lightweight-performance-testing/</link>
		<comments>http://thetesteye.com/blog/2012/03/lightweight-performance-testing/#comments</comments>
		<pubDate>Fri, 30 Mar 2012 10:01:42 +0000</pubDate>
		<dc:creator>Rikard Edgren</dc:creator>
				<category><![CDATA[Skills]]></category>
		<category><![CDATA[lightweight]]></category>
		<category><![CDATA[performance]]></category>

		<guid isPermaLink="false">http://thetesteye.com/blog/?p=2544</guid>
		<description><![CDATA[<img src="http://thetesteye.com/blog/wp-content/uploads/skills.png" width="48" height="48" alt="" title="Skills" /><br/>If performance is crucial for product success, you probably need pretty advanced tools to measure various aspects of your product, to find all bottlenecks and time thiefs. For all other software, performance is just very important, and you might get by with lightweight test methods. You may, or may not have quantified performance requirements, but [...]]]></description>
			<content:encoded><![CDATA[<img src="http://thetesteye.com/blog/wp-content/uploads/skills.png" width="48" height="48" alt="" title="Skills" /><br/><p>If performance is crucial for product success, you probably need pretty advanced tools to measure various aspects of your product, to find all bottlenecks and time thiefs. For all other software, performance is just very important, and you might get by with lightweight test methods. You may, or may not have quantified performance requirements, but you should test performance to some degree anyway; for the whole, but also for each detail (when appropriate.)</p>
<p>In TheTestEye&#8217;s <a href="http://thetesteye.com/posters/TheTestEye_SoftwareQualityCharacteristics.pdf">classification</a>, performance consists of:</p>
<p><strong>Performance</strong>: Is the product fast enough?</p>
<p>- <strong>Capacity</strong>: the many limits of the product, for different circumstances (e.g. slow network.)<br />
- <strong>Resource Utilization</strong>: appropriate usage of memory, storage and other resources.<br />
- <strong>Responsiveness</strong>: the speed of which an action is (perceived as) performed.<br />
- <strong>Availability</strong>: the system is available for use when it should be.<br />
- <strong>Throughput</strong>: the products ability to process many, many things.<br />
- <strong>Endurance</strong>: can the product handle load for a long time?<br />
- <strong>Feedback</strong>: is the feedback from the system on user actions appropriate?<br />
- <strong>Scalability</strong>: how well does the product scale up, out or down?</p>
<p>Be aware of different definitions of performance testing, e.g. some include reliability, stress handling, robustness, and what stakeholders believe is most important might differ (even when using the same words&#8230;)</p>
<h2>Ongoing Violation Awareness</h2>
<p>The number one lightweight method starts by finding out which of these characteristics that are relevant for your product. Then keep them in the back of your head, and whenever you see something fishy, investigate further and communicate. Often the OK zone is easy to reach, but testers should notice when violations occur. When appropriate, apply the destructive principle: Increase the amount of everything that can be increased.</p>
<h2>No Tools</h2>
<p>Perceived performance is what matters for end users (but maybe not for a product comparison check list) so think about how it feels, and try using a stop watch. You might get pretty far by load testing with colleagues with several instances each.</p>
<h2>Tools</h2>
<p>There exists limiters for CPU, RAM, bandwidth etc. and many of them are free (and some of them become obsolete.) A task manager/resource utilization tool can give you hints on memory, CPU, disk, network et.al. Scripting your product to run over weekend is good for endurance and stability testing. JMeter is free and often quick to get running.</p>
<h2>Summarizing</h2>
<p>Summarizing performance test results is difficult. Aggregations of measurements don&#8217;t tell the full story, and the whole story takes a long time to tell. Communicate what is important, which is easier if you have asked stakeholders beforehand.</p>
<p>Warning: For some products, users aren&#8217;t as interested in Performance as the developers&#8230;</p>
<p><a class="a2a_dd a2a_target addtoany_share_save" href="http://www.addtoany.com/share_save#url=http%3A%2F%2Fthetesteye.com%2Fblog%2F2012%2F03%2Flightweight-performance-testing%2F&amp;title=Lightweight%20Performance%20Testing" id="wpa2a_2"><img src="http://thetesteye.com/blog/wp-content/plugins/add-to-any/share_save_171_16.png" width="171" height="16" alt="Share"/></a></p>]]></content:encoded>
			<wfw:commentRss>http://thetesteye.com/blog/2012/03/lightweight-performance-testing/feed/</wfw:commentRss>
		<slash:comments>2</slash:comments>
		</item>
		<item>
		<title>You might be an expert at non-functional testing</title>
		<link>http://thetesteye.com/blog/2010/01/you-might-be-an-expert-at-non-functional-testing/</link>
		<comments>http://thetesteye.com/blog/2010/01/you-might-be-an-expert-at-non-functional-testing/#comments</comments>
		<pubDate>Sun, 17 Jan 2010 18:03:28 +0000</pubDate>
		<dc:creator>Rikard Edgren</dc:creator>
				<category><![CDATA[People]]></category>
		<category><![CDATA[Skills]]></category>
		<category><![CDATA[compatibility]]></category>
		<category><![CDATA[manual product testing]]></category>
		<category><![CDATA[non-functional testing]]></category>
		<category><![CDATA[performance]]></category>
		<category><![CDATA[quality attributes]]></category>
		<category><![CDATA[security]]></category>
		<category><![CDATA[usability]]></category>

		<guid isPermaLink="false">http://thetesteye.com/blog/?p=767</guid>
		<description><![CDATA[<img src="http://thetesteye.com/blog/wp-content/uploads/people.png" width="48" height="48" alt="" title="People" /><img src="http://thetesteye.com/blog/wp-content/uploads/skills.png" width="48" height="48" alt="" title="Skills" /><br/>Now and then I read that testers don&#8217;t know enough about Usability, that there is a need for a Performance Testing expert, that a Security consultant should be called in, or that a master of the used technology would make Installation and Compatibility testing possible. This might be true in the general case, but there [...]]]></description>
			<content:encoded><![CDATA[<img src="http://thetesteye.com/blog/wp-content/uploads/people.png" width="48" height="48" alt="" title="People" /><img src="http://thetesteye.com/blog/wp-content/uploads/skills.png" width="48" height="48" alt="" title="Skills" /><br/><p>Now and then I read that testers don&#8217;t know enough about Usability, that there is a need for a Performance Testing expert, that a Security consultant should be called in, or that a master of the used technology would make Installation and Compatibility testing possible.<br />
This might be true in the general case, but there are many testers that are working with the same product suite for years, and over time you become an expert of most aspects of quality <strong>for your specific product</strong>.</p>
<p>Let&#8217;s look at some examples:</p>
<p>One sub-category of Usability is Operability; that an experienced user can perform functions in a fast and efficient way. As a system tester, you have done most operations many times, and you <strong>know</strong> what you can expect from features in the sense of operability. You can point at places where for instance the ability to delete multiple items with a few clicks would make a difference.<br />
Learnability is another sub-category of Usability. You only learn the basics once, but maybe you have heard customer stories of confusing things, or you could let a new member of the test team think in this direction.<br />
Regarding Accessibility it doesn&#8217;t cost too much to use High DPI, speakers turned on, Code Blind or Magnifier sometimes.</p>
<p>Large-scale Performance might require an expensive tool, but I bet you&#8217;re doing some simulations without them; maybe just by telling the whole development team to hit the same server at the same time.<br />
There is also a small-scale, low-level Performance aspect that shouldn&#8217;t be understimated. If a dialog takes more than a second to display, it might be something that put the user&#8217;s confidence in doubt.<br />
If you have tested your product for quite some time, you will immediately notice when something takes just a bit longer than it could take. There of course might be valid reasons for this, but talking to the developer about it might be beneficial to both the producers and consumers of your software.</p>
<p>Security testing seems more difficult, but as a product tester you know at which moments authentication takes place, you know if there are passwords stored, and that they should be encryted; you might not know how to exploit a crash, but you are an expert at provoking the crashes.</p>
<p>For Hardware/OS/Application Compatibility testing you will become more of an expert the longer you work with them, at least if you have the curiousity to learn more things when you get the chance.<br />
You might not know a lot about how the iPhone works, but you know all the details that can be used for interacting with your web site.</p>
<p>Sometimes I also read the extreme that testers shouldn&#8217;t bother with Usability/Security/Performance testing, which to me seems like an incredible waste of knowledge and resources.<br />
When testing functionality manually, you can look at quality attributes <strong>at the same time</strong>, and get a lot of coverage for free.</p>
<p>I&#8217;m not saying true specialists aren&#8217;t needed, but I&#8217;m saying that there are a lot of expertise in your building that at least can be used complementary.<br />
If you are in a situation where you know a lot of these things, but aren&#8217;t allowed/encouraged to test these things, I think you should try to convince your managers of a better and more fun way to test your product.</p>
<p><a class="a2a_dd a2a_target addtoany_share_save" href="http://www.addtoany.com/share_save#url=http%3A%2F%2Fthetesteye.com%2Fblog%2F2010%2F01%2Fyou-might-be-an-expert-at-non-functional-testing%2F&amp;title=You%20might%20be%20an%20expert%20at%20non-functional%20testing" id="wpa2a_4"><img src="http://thetesteye.com/blog/wp-content/plugins/add-to-any/share_save_171_16.png" width="171" height="16" alt="Share"/></a></p>]]></content:encoded>
			<wfw:commentRss>http://thetesteye.com/blog/2010/01/you-might-be-an-expert-at-non-functional-testing/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
	</channel>
</rss>

