Web Performance Matters

Performance Management (SLM) Challenges for Web 2.0, Ajax, and Rich Internet Applications (RIA's)

Last week, TechTarget published an article by Patrick Lightbody about the performance of Web 2.0 applications. The article's technical core -- which I review below -- is a useful checklist of ten recommendations for developing and testing Web 2.0 applications with performance in mind.

Ten guidelines

Here are Patrick's ten guidelines; I have abbreviated the commentary. See the article for the full descriptions of each:

Ten ways to improve testing, performance of Web 2.0 applications

1. Engineer Web experience quality into your product. Take the entire experience into account up front, from the moment you conceive the application. Ensure that your release criteria include specific performance and reliability metrics that you can measure often during and after development.
2. Know (and manage) what feeds into your customer's experience. Keep tabs on every factor that affects your customers' experience, including third-party data and services.
3. Know your customers, their profiles and their usage patterns. What kind of browsers do they use? What kind of machines? How do they connect to the Internet? Where in the world are they located? What are their usage patterns ...
4. Create a browser compatibility lab consisting of all the possible browser operating system combinations users could have, including cell phones and the BlackBerry. Consider using the open source tools Selenium, Watir, and Firebug.
5. Capture screenshots and movies of actual tests on those platforms so you can gain real insight into any problems, their impact and how to fix them.
6. Capture logged activity in the browser during automated tests and during production. There's too much application logic in the UI to ignore.
7. Understand the connection between performance and perception. If a user's browser window is full, but parts of the Web page that are off your screen haven't loaded, perception-wise the page is complete.
8. Incorporate the browser into your continuous integration (CI) processes. Most implementations of CI typically test server code, but they don't account for the increasing amount of activity occurring in the browser.
9. Consider "on-demand" testing. Leverage someone else's testing horsepower, architecture and setup investment.
10. Refactor tests as Web applications evolve. In the old days, defining an automated test on a Web application was easy: every step in an application use case corresponded to a new page view. With Ajax, things are more complicated, so refactoring is critical.

-- Patrick Lightbody, TechTarget, 6/11/2007

Because I believe in systematic performance engineering, I am always pleased when writers advocate proactive approaches to application performance. It's the only rational way to ensure acceptable performance in production applications.

So it's too bad that Patrick feels the need to justify his good advice by surrounding it with an introduction and conclusion that suffers from all the worst features of Web 2.0 coverage. In those sections, a few half-truths are buried in an amalgam of excessive hype, false claims, meaningless analysis, and an optimism that underestimates the real technical challenges.

Excessive hype

The Web 2.0 vision promises many wonderful things, including rampant social networking, grassroots content creation and broad-based collaboration. On the tech side, Web 2.0 promises access to all the riches of the Internet with the blazing performance of the fastest desktops.

-- Patrick Lightbody, TechTarget, 6/11/2007 [emphasis added]

Who made that promise? Has anyone ever before claimed that Web 2.0 applications would deliver the riches of Internet with "blazing performance"? And if so, did any actual Web users believe them? How many Web users can even define Web 2.0? Can you? Don't feel bad if you're having difficulty; according to the TechTarget definition linked above: There is no clear-cut demarcation between Web 2.0 and Web 1.0 technologies, hardware and applications. The distinction is, to a large extent, subjective.

OK, so Web 2.0 is poorly defined. But that doesn't justify describing it as the nirvana of online experience, does it?

False claims

Times were simpler a few years ago when the vast majority of Web users used Internet Explorer (IE) on Windows. This single delivery platform led to few surprises when applications hit production. Since developers were using the same platform as their users, problems showed up sooner rather than later.

Although tomorrow's Web 2.0 experience promises wondrous things, we are currently going through a painful "rock bottom" trough first. Users are on a host of different browsers (IE, Safari, Firefox, Opera, iPhone, BlackBerry and more) on myriad operating systems (Windows, Mac, Linux, and mobile OSes).

-- Patrick Lightbody, TechTarget, 6/11/2007

First, regardless of hype or anti-Microsoft sentiments, actual statistics about usage show that the vast majority of Web users are still using IE on Windows. I suspect that the real issue is that, compared to the population of all Web users, a disproportionately large percentage of Web developers prefer Firefox, because it offers many open-source plugins and better support for CSS editing and DOM inspection. Even so, no sane Web developer would bypass testing their code on IE and Windows.

Second, I agree that Web 2.0 and Rich Internet Applications have yet to pass through the Gartner Group's trough of disillusionment, a subject I have discussed here in an earlier post on Ajax Wisdom. But Patrick's throwaway reference does not actually explain that trough, its consequences, or what needs to happen for Web 2.0 technology to evolve past it.

Instead he suggests that the trough is related to browser platform diversity, and that the answer is to update our software development and testing strategies, offering his checklist items as suggestions. But while it is always better to adopt systematic approaches to application development and testing, doing so will not absolve the creators of browsers, measurement tools, and application development frameworks from their responsibilities to produce technologies that enable performance management.

Meaningless analysis

Yesterday's Web Experience

• Reliability: B+ (one consensus platform was easy to troubleshoot)
• Appearance: C- (quite gray)
• Performance: B (expectations were lower and applications were simple)
• Overall: Mediocre

Today's Web Experience

• Reliability: F ("You forgot to test for my particular browser and OS!" "There's a hole where that Web service should be!")
• Appearance: A- (pretty slick, I must say)
• Performance: C (expectations are up, and so are numbers of parts that break)
• Overall: Poor

So what is it going to take to get these reliability and performance grades up, deliver a consistent user experience despite all the component pieces, and deliver on the promise of Web 2.0?

-- Patrick Lightbody, TechTarget, 6/11/2007

If a completely subjective analysis concludes that the Web experience in general is slipping from mediocre to poor, will someone who manages the development and testing efforts of an actual Web application feel that the conclusion applies to them? Will this sweeping judgement actually alter anyone's behavior?

People understand and can relate to evaluations and grades, but only if they are supported by some degree of analytical rationale.

Half-truths

More logic than ever runs inside the browser. Meanwhile, more content than ever is beyond the host organization's control -- advertisements, analytics and content delivery networks, for starters. It's a composite world, but companies typically haven't figured out how to test anything besides their own stuff.

...

Since we are dealing with an entirely new development and delivery paradigm, we must update our software development and testing strategies. Here are some suggestions ...

-- Patrick Lightbody, TechTarget, 6/11/2007

It's true that more logic than ever runs inside the browser today, but the rest of this paragraph does not describe features that separate Web 1.0 and Web 2.0 applications. Content and advertisement delivery networks have been around since the late 1990's and ways to distribute function have been evolving steadily ever since Netscape released the very first commercial browser. Dealing with those features does not require an entirely new development and delivery paradigm.

And that is probably why the software development and testing strategies that Patrick is proposing are not really that new either. They may be new to someone who has never thought about systematic performance engineering before, but many leading companies already have Web development processes that incorporate such principles. The first three are actually fundamental principles of proactive performance engineering.

Underestimating the challenges

Patrick concludes:

Adopting these strategies will ensure that your QA testing is sophisticated, thorough and agile enough to keep up with the evolving complexity of Web 2.0 applications. Good, clean, reliable, high-performing code will help transform the Web 2.0 vision and its enormous potential into a reality that raises the value of the Internet.

-- Patrick Lightbody, TechTarget, 6/11/2007

There are two flaws in the logic of this conclusion. Although both sentences are true, the first does not ensure the second, and the second is incomplete.

First, sophisticated testing will NOT produce high-performing applications -- you must first design and build the application with performance in mind. Multi-million dollar rollouts of new Web applications have been deferred or cancelled because developers focused on function not performance, and senior management assumed that testing and tuning would be sufficient.

Second, to deliver any application's full potential, and keep it performing well, it must be monitored and managed after it is rolled out. And if you cannot measure it properly, it will be impossible to manage it effectively.

But don't get me started on that subject again ...

Tags: Patrick Lightbody, TechTarget, management, performance engineering,
Web application, Web performance, testing, software testing, performance testing,
Web 2.0, IE, Internet Explorer, Firefox, checklist, Performance Matters