Web Performance Matters

Buying test tools is sometimes just like buying a new car: the salesman tells you that the car is reliable and has a great warranty; then the finance person warns of everything that could go wrong that isn't covered in the warranty and trys to sell you an extended warranty and maintenance contract.

Ben Simo, the author of the first paragraph, writes the blog Quality Frog. He is a software tester and test automation developer. His blog contains his "ramblings about software testing" and links to useful resources. In his post Performance Testing Lessons Learned, Ben shares his experiences with load testing and load testing tools.

Ben notes that ... "Web and client/server load testing can easily become a complex task. Most people I've met got started in load testing with only minimal training in using the test tools. This is how I got started... I had watched others perform load tests. I had read numerous load test plans and reports. However, I had never directly participated in executing automated load tests... then I was asked to lead a load testing project".

Through the years, I have made many mistakes designing, scripting, and executing load tests. Load testing easily becomes complex. Tool sales people sometime tell us that nearly anyone can create tests with their tools. Learning the mechanics of how to use a tool are often the easy part. It's what you do with the tool that matters.

Ben lists ten lessons he's learned:

1. Bad assumptions waste time and effort
2. Get to know the people in your neighborhood: no single person or group is likely to have all the required information
3. Don't script too much too soon: you may end up tossing out much of what you script
4. Different processes have different impacts: what users do can be as, or more, important as how many users are doing it
5. Modularize scripts: simplify script maintenance -- but only when you intend to run the script again
6. Data randomization is not always a good thing: randomization can make result comparison difficult
7. Code error detection and handling
8. Know your tools and test environment
9. Try to translate results into stories that matter to the applicable stakeholders
10. Most performance and load-related problems are due to software code or configuration; not hardware

Ben also includes some advice that follows from many of these lessons. Check out Ben's post for the details.

I have added Ben Simo's Quality Frog blog to my blogroll (see the sidebar).

Tags: performance wisdom, performance, Ben Simo, Quality Frog, testing, load testing, performance testing, performance tools, Performance matters, Web performance, Web application