Web Performance Matters

Most bloggers, especially those who work in high tech, want to like Technorati. Its mission (and even its name) appeals to our vanity. When Technorati launched in November 2002, that mission was simple and easy to understand. By tracking links among blogs, and finding the most popular, it would determine which blogs were the most authoritative.

Over the years, that mission expanded from simply tracking the 100 most popular blogs to include periodic reports on the state of the blogosphere.

For some history of this evolution, Niall Kennedy's Weblog contains a fascinating reminder of the original Technorati site, and interesting unofficial histories of Technorati's first two years (2002-2004), and later, of its the third year (2004-2005).

Technorati launches tags

Early in 2005, Technorati added support for tagging. Citing an article by Clay Shirky about the advantages of folksonomies vs. controlled metadata vocabularies, Dave Sifry, Technorati's founder and CEO wrote:

Tags are a simple, yet powerful, social software innovation. Today millions of people are freely and openly assigning metadata to content and conversations. Unlike rigid taxonomy schemes that people dislike, the ease of tagging for personal organization with social incentives leads to a rich and discoverable folksonomy. Intelligence is provided by real people from the bottom-up to aid social discovery.

-- Dave Sifry, Sifry's Alerts, January 17, 2005

People can debate the pros and cons of folksonomies, and the effectiveness of uncontrolled tagging as searchable metadata. (Is Everything really Miscellaneous?) But that's a subject for another post. Technorati deserves credit for their initiative in creating mechanisms to support systematic tagging by bloggers, and integrating that support as an extension of their existing services for discovering, tracking and counting inter-blog links. But this service has been buggy from the outset. In the original blog post announcing support for tagging, Sifry had to admit:

This is still beta software. It has bugs. There are kinks to be worked out, and missing features, too. For example, we still miss some tags here and there, especially for certain weblogs, due to some post detection issues. We're working on that.

-- Dave Sifry, Sifry's Alerts, January 17, 2005

The problems multiply

He was certainly right about that. As the number of blogs to be tracked continued its explosive growth, and as more and more people began using tags during 2005, Technorati's continuing tag tracking problems became a serious concern for the company. Since then, despite Sifry's periodic public assertions that their quality problems were now being tackled head on, Technorati's problems have become notorious among bloggers.

In the world of high tech, rapid and evolutionary development, open source software, and the Web, we don't expect perfection. We tend to be far more tolerant of quality glitches in Web software than we are in other products, like cars. When we find a really useful product or service, we will forgive a few bugs, especially if the company:

• is working on fixing the problems,
• keeps us informed of their progress,
• and has a track record of responding to customers.

Sadly, Technorati has consistently struck out in all three areas. When the same problems happen again and again and again, and the company does not respond to customer concerns, even those people who believe in the value of the service eventually give up hope.

Dave Sifry certainly seems to care about bloggers' perceptions of Technorati. I have read quite a few stories of him getting problems resolved for bloggers who complained, either in their blogs, or by emailing him directly. But depending on CEO intervention is not an effective or scalable way to organize customer support.

My own experience

I, like many others, have found dealing with Technorati a frustrating experience. Two common problems bloggers seem to face is not being able to "claim" their blogs (i.e. authenticate their ownership of, or association with, a specific blog), and Technorati not updating their blogs' statistics promptly, regularly, or ever. The support forum is full of complaints like "I've been blogging every day, yet Technorati says my blog has not been updated for 57 days."

I had the opposite problem -- duplicate listings, as "Web Performance Matters - Home" and "Web Performance Matters - Journal". This happened because when I moved my blog from its old home on Blogger (which Technorati also lists) I made my new site's home page separate from the front page of the journal. This confused Technorati's counting mechanisms. So every post got listed twice, and inbound links might get counted to one or the other of my "two" blogs. I had tried claiming both versions of my blogs, thinking I could then find a way to merge them. But no such feature exists.

After poring through the poorly organized morass of help information in the support forum, I finally discovered that I had a known problem. Technorati's advice (from memory, since their site is now down again) was either to fix it by implementing some kind of redirection on my own site, or to email customer support and ask them to use an internal aliasing mechanism to merge statistics for the two URLs into a single reported blog.

Maybe I should have known better, but I chose the latter, getting their standard email response, which amounts to: Thanks; remind us next week. Two weeks later I reminded them, receiving an identical response, but a new ticket number. Ten days later, the same. I tried unclaiming one of my "two" blogs in Technorati, but while this made my profile look right, it did not actually make the duplicate blog (with all its duplicate post listings) disappear. They were still listed, but now as an "unclaimed" blog with no known owner, right alongside the same posts from my "claimed" blog.

Tired of this silliness, I decided I just had to fix it myself. After a few hours studying Squarespace's site templates, I figured out how to code some custom logic in a header extension file, which injects an extra CSS class statement into every generated page. Now I can distinguish the Journal's front page from all other journal pages. This has allowed me to format that page differently, make it the site's front page, and rename the former front page as "Recent Posts".

I also had to code a couple of redirections, so that external links either to "home" or "journal" now all go to the same place. I hope this action will force Technorati to finally recognize that my two blogs are really just one.

That will be a relief -- but I shouldn't have to go through all these gyrations in order to remain a happy customer of Technorati. I don't make my living programming, but I've been programming off and on since 1964, and I can usually figure stuff out if I work on it for a while. But many other customers will just give up. It seems as if Technorati's implementations are not able to keep up with their good ideas.

Another mission change

But before they have fixed their existing services, they've morphed their mission once again:

As I've blogged about before, the world has changed. Whereas folks using Technorati a couple of years ago were predominantly coming to us to search the blogosphere to surface the conversations that were most interesting to them, today they are increasingly coming to our site to get the 360 degree context of the Live Web - blogs of course, but also user-generated video, photos, podcasts, music, games and more. They want all the good stuff out there, all in real-time, and we're using the power of 80 million bloggers to help organize it and make it fun to browse; using the wisdom of crowds as a mirror on ourselves. Have a look at the new homepage ...

-- Dave Sifry, Sifry's Alerts, May 23, 2007

And once again, just as when tagging was announced in 2005, we get the familiar message from Dave that ...:

...there's lots more to do. This is only scratching the surface, and we're going to be rolling out lots more updates and innovations throughout the rest of the year. We're going to continue to work on performance, making things faster and more stable. Every now and then searches still fail for our users or take too long to respond, and that's a top priority for our backend team to fix. We're going to keep listening to you, our users - It is an honor that you give us your time and attention - over 11.7 Million people last month visited Technorati. We've tripled in visitors and traffic in under 5 months. Thanks for telling your friends about us. Thanks for putting your faith in us.

-- Dave Sifry, Sifry's Alerts, May 23, 2007 [emphasis added].

Well, Dave, we are telling our friends. Bloggers are experts at that. But we don't just talk about the good things you're doing. Because some of us are losing that faith in you. As I type this paragraph, the entire Technorati site is down and the staff is (once again) scouring the blogosphere to find the Technorati Monster.

Taming the monster ...

Dave, we've seen your periodic reports on The State of Technorati. We know you want to fix the problems and grow the company. So maybe after you have found a new CEO, you'll have more time to figure out how organize an effective support team to tame that Technorati Monster and keep it chained in its place.

We'd really like you to succeed.

Tags: Technorati, Dave Sifry, tagging, folksonomy, bloggers, support, customer support, problems, quality, bugs, Web performance, Performance Matters, site maintenance