In the third entry of a five-part series outlining 5 Steps to Ensure Performance, Technical Solutions Director, Joel Deutscher explains what you are looking for and how to interpret those results.

Step 3 – Understand Your Results

Performance testing has suffered an image problem. The problem is that performance testing has traditionally been viewed as a dark art; a view that is perpetuated by testers producing reports that nobody can understand. Regardless of the reasons, if you don’t understand what has been done and what it means, you will end up in front of the media making claims like “There is no way it can fail” right before it does.

As I mentioned in step 1, the key to getting what you need out of any exercise is to know what you want. So what should you be asking for from your performance results?

Firstly, you should ensure that the results are going to give you a clear indication of your ability to meet objectives and mitigate the associated risks. This should be up front and clear in any report. Look for phases such as “the objective of this test” early in the summary to ensure that everyone is clear on why the test was run, and why it was considered a success or not.

Once the testing has been completed, expect to see a summary table listing which risks have been mitigated, and which ones were not. It is easy in the compressed timelines of testing to reduce the scope, and this is exactly why we take a risk based approach. However, it’s crucial that the risks that have not been mitigated are understood.

Secondly, the graphs and tables in the reports need to be easily understandable and not just dumped directly out of the performance tool. While the reporting aspects of performance testing tools have increased significantly over the last few years, we still need to add context through comments and overlays to ensure that the reader doesn’t need to do the interpreting.

One method I use to train junior performance testers is to cover a graph or table with my hand, lift it off, and see how long it takes to understand what I am looking at. If it’s not clearly evident within 5 seconds, then it needs to be reworked.

I also use the “so what?” method of reviewing a report where everything included must have a purpose. If the infrastructure performed well, I would expect the 60 graphs of proof to be included in the Appendix, not the main document where it isn’t adding any value.

Finally, make use of walkthroughs of the results, and ask that the performance results are presented rather than attached to an email that will never be read. This is particularly important when issues are found. A quick presentation can be much more effective, requires less interpretation and is more enjoyable for everyone.

Performance testing results don’t have to be difficult to understand, and you don’t have to hand the power over to your teams or a vendor to decide if your application is ready to launch. If you don’t understand what has been done, and what it means, then demand it of your teams or vendors.

← Step 2 – Own Your Performance Step 4 – Save Money with On-Demand Services →