Web Performance Matters

When stock markets everywhere took a big hit last Tuesday, volume on the New York Stock Exchange nearly doubled its typical level, topping 4 billion shares for the first time ever.

Barron's Online is a weekly magazine that covers U.S. financial information, tracks company statistics, and reports on market developments. Last week, Theresa W. Carey of Barron's decided that Tuesday's trading surge created "the perfect opportunity to see how some of the online brokers ... coped with extremely heavy traffic". The result was an interesting story -- Online Stress Test -- about the performance of leading brokerage sites during the trading peak. To sum up it up, the bigger the broker, the slower the site:

Major brokers affected

Most large brokers' infrastructure can only allow a small fraction of their customer base to log on at one time. I poked around Schwab.com, Etrade.com, TDAmeritrade, Fidelity, Scottrade, optionsXpress, OptionsHouse, TradeKing and thinkorswim's Websites. I looked at quotes, searched for a covered call to trade, and tried to enter an order.

Schwab was running extremely slowly, and actually tossed me out when I attempted to enter an order. When I sought to enter a covered call order on E*Trade, the option quote wouldn't come up. TD Ameritrade shifted to delayed quotes (instead of real-time), but I was able to enter an order. Fidelity was sluggish yet it too let me enter an order. Although they seemed a little slower than they did the previous week, OptionsHouse, TradeKing and optionsxpress weren't running at glacially slow speeds All three let me enter an order. Thinkorswim was plugging along as though there was nothing extraordinary happening.

I brought up software from Schwab's CyberTrader, MB Trading, Interactive Brokers and Terra Nova Trading. Cyber was OK even though the web-based Schwab offerings (including Street Smart Pro) were slow. The others looked fine, too. Quotes, especially from the NYSE, seemed to be loading more slowly than usual, but subsequent reports have said the exchanges were swamped, so the problem might not have been endemic to the brokers.

-- Theresa W. Carey, Barron's Online

Measurements confirm problems

During the busiest trading period, Keynote Systems, which has a worldwide network of computers that measure the response times of Web pages and online transactions, observed significant performance slowdowns for some of the nation's leading online stock trading sites:

According to their tests, many of the most popular Web sites suffered speed and availability hits. Keynote's stock trading performance tool measures the time it takes to conduct a stock transaction from 10 geographic locations across the U.S.

Sites included in the Keynote web performance index like Ameritrade, Fidelity, Firstrade, Muriel Siebert, Schwab, ShareBuilder, TD Waterhouse and Wells Fargo, began slowing down Tuesday at 10:30 a.m. Pacific. Many of the sites took up to two times the normal time to complete an online trade, which is a big deal during a huge selloff. On a typical day, Keynote takes an average of 10-15 seconds to log into a broker's site, get a quote, execute a transaction, and log out. While the markets went haywire, these measurements increased to 20-25 seconds, and up to one minute in some cases.

-- Theresa W. Carey, Barron's Online [emphasis added]

Maybe the worst ever ...

Keynote publishes its measurement results weekly, in its Broker Web Transaction Performance Index. Keynote's Abel Gonzales, who manages this service, compared Tuesday's incident with the history of online brokerage services:

"Keynote has never seen such significant slowdowns and limited availability among online trading sites since the index began in 2000." Gonzales says that there was a 25% drop in the overall number of successful trades his firm was able to undertake on the broker sites that make up the firm's index. That means it's possible that upwards of 25% of investors attempting online trades between 1:30 p.m. and 4:30 p.m EST may not have been successful.

"I would imagine that this time was frustrating for online brokerage users," he says.

By Wednesday morning, all the sites were back to normal, says Gonzales.

-- Theresa W. Carey, Barron's Online [emphasis added]

It appears that even the most well-prepared online businesses can be surprised by flash crowds. Last year I reported on the need for online retail sites to conduct load tests to prepare for the holiday shopping season. This is yet more evidence that any site whose business depends on handling a lot of online traffic should plan for sudden and dramatic increases in load, and conduct regular load tests that simulate traffic volumes well in excess of their normal levels.