« Apdex as a (Key) Performance Indicator | Home | Choosing Performance Management Tools »

Measuring Mobile Web Sites

Illustration: Web Analytics Report

The methods commonly used to measure Web sites often don't work for mobile sites, because of differences among mobile platforms and devices. The biggest obstacles are lack of support on mobile devices for Cookies, JavaScript, and client IP addresses.

The Wikipedia article on Mobile Web Analytics provides a succinct introduction to these and other challenges -- see the section on "Problems with tracking visitors, visits and clickpaths in the Mobile Web". Because of these limitations, the tagging methods used by traditional Web analytics tools do not work on most mobile devices.

While researching this subject, I discovered Kaizen Analytics, an excellent blog by Michael Notté. In a recent post, Michael provides a useful overview -- see Mobile Analytics: vertical-specific vs. traditional Web Analytics solutions. Michael points out that there are other ways to collect data about mobile Web sites. He outlines four solution approaches, illustrating each with a helpful diagram, reproduced below:

Mobile analytics: log-server based solution
Mobile analytics: packet sniffing based solution
Mobile analytics: tag-based solution
Mobile analytics: server script solution
  • Server log-based: Process the raw data coming from your web server. Michael recommends ignoring this one unless you are a geek with spare time on your hands.
  • Packet sniffing: A device is added between your server and the Web. It listens and analyzes requests sent and received by the server. This method does not require adding any tags to the page content, but does involve installing additional hardware and/or software in (all) datacenter(s) serving content to be tracked. This may present scalability challenges for large sites.
  • Image tag-based: Each time a page is rendered, the client device sends tracking data to a collector server using an image request that has parameters in the querystring. These tracking parameters can be set dynamically by the server when the content is generated. This method is easy to implement, but has limitations, and can not be used for event tracking.
  • Server-side script: A script is added on server side that sends data directly to a collector server when requests are processed. By extracting data from requests and content received at the server, this method avoids the complications of sending tracking data from diverse mobile device platforms. According to Michael, this solution is gaining popularity among traditional Web Analytics vendors.

These helpful diagrams caught my eye, because they are closely related to another current project of mine -- organizing a Webinar on June 30 for the Apdex Alliance on Choosing Application Performance Management (APM) Tools [details; register].

In the Webinar, Peter Sevcik will describe a systematic framework for evaluating performance management tools--and a key component of that framework is the location of measurement devices. There are at least half a dozen places to collect data these days, extending all the way from server logs in the backend to scripts on the client device.

Will performance and analytics tools converge?

Performance management tools have traditionally been regarded as a separate market segment from from Web Analytics tools, because they were purchased by different people -- namely IT staff and Marketing/Business Management respectively. But as business becomes increasingly mobile, and tool vendors continue to merge and expand their portfolios, I expect to see these two separate segments evolve and become closer.

Both tool segments face a similar--in some cases, identical--challenge of needing to collect and consolidate data about customers in a networked world. And while site performance and customer behavior are different issues, which focus on different metrics, the ultimate reasons for tracking those metrics are closely related. I have written about that here before, see The Dimensions of Usability.

In preparation for his Webinar (and a report on performance management tools), Peter will be surveying tools vendors. I am interested to find out whether Peter sees any signs of performance management and analytics tools converging.

PrintView Printer Friendly Version

EmailEmail Article to Friend

Reader Comments (3)

I'm glad to see some folks finally writing about this. Of course I'm a bit sad (but not surprised) to see that things really haven't advanced very much since Chris Walters and I wrote on the topic in 2002 (Automated Testing for Embedded Devices).

I guess that's why I'm still having a lot of success with projects in the wild world of wireless & mobile applications. Lucky for me, but my expectations for how far we'd have come by now in this area were far greater than what I see, so I'm not sure it's so lucky for the typical mobile web user.

--

Scott Barber

President & Chief Technologist, PerfTestPlus, Inc.
www.perftestplus.com
sbarber@perftestplus.com

Executive Director, Association for Software Testing
www.associationforsoftwaretesting.org
executive.director@associationforsoftwaretesting.org


"If you can see it in your mind...
you will find it in your life."

June 15, 2010 | Unregistered CommenterScott Barber

Hi Chris,

Just found out your post and I wanted to thank you for the comments about my blog and the post. Really happy to see it is helpful and that it is shared by others.

Cheers,

Michael

June 16, 2010 | Unregistered CommenterMichael Notté

Another way to track your page loading and performance is with this great $5 app I use as a supplement to all my development efforts is this iPhone app called "Are We Up?". Like I said, it's $5 annually and provides you with unlimited texts/emails notifications about your site. I have been really happy with this service. You can purchase it on iTunes.com or on fiverr.com: http://www.fiverr.com/users/johndesp/gigs/monitor-you-web-site-for-a-year-and-send-you-email-text-message-and-phone-call-alerts-if-its-down

October 22, 2010 | Unregistered CommenterChristina

PostPost a New Comment

Enter your information below to add a new comment.
Author Email (optional):
Author URL (optional):
Post:
 
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>