In the last two elections, the patterns was similar -- Election Day traffic was three times that of the preceding days. This was our chart in 2006:
Given our daily traffic was in the 3 million pageviews per day in the weeks leading up to this election night, we wondered if that trend would hold. Our tech guru Jeremy (aka "ct") literally spent the entire year preparing our infrastructure for the Big Show. We had the primaries and debates to fine tune the backend, but would we actually serve 9 million pageviews? And if so, could our servers survive such a crush?
Well, here's what we served: First of all, the main site:
And our kick-ass Electoral Scoreboard (which we agree, was the best of the web):
Add them up, and we get ... 9 million pageviews Election Day. Funny how these trends have held three elections in a row.
Early in the night we had some scoreboard bugs to deal with -- it is, after all, new technology being debuted for its first election. But Jeremy and the widget's developer had squashed those bugs by 7 p.m. ET (before the big crush of results started coming in), and after that, it was pretty much smooth sailing. A year's of hard work by Jeremy, as well as efforts by our data center Voxel, and our widget's developer Free Range Studios, all conspired to deliver a fantastically smooth night for you guys.
And for me, it was a chance to focus on the results, rather than on keeping the site alive. My champagne tasted sweeter knowing my tech team had it all under control.
And you don't want to know what Jeremy did all year.
Hardware is part of the solution, Bingham says. “To handle the traffic better, we moved to a cluster of six quad core Xeons with 8GB RAM for webheads that all boot off a central NFS (Network File System) root, with the capability of adding more webheads as needed,” he said. Daily Kos also added two 16GB eight-core Xeons and a 6×73GB RAID-10 array for database files running a MySQL master/slave setup [...]
Many database-driven sites facing scalability challenges have looked to the distributed caching system memcached, which helps speed dynamic web applications. Bingham says memcached has played an important role in scaling the Daily Kos site. “I greatly expanded memcached usage with 1GB instances memcached running on each webhead, which they all share,” said Bingham. “The backend also places fully rendered pages into memcached, which a hacked up lighttpd running as the front end proxy then serves these pages from memcached directly to anonymous users. This has helped the sites performance immensely, since not only does it spread the work of rendering these pages around to the all of the webheads, but it greatly reduces the amount of work the backend has to do.”
I told you that you didn't want to know...