On Saturday at 8 AM PT, the tech team will be moving Daily Kos from its current web host to Amazon's Web Services. We'll be taking the site down for an estimated one to three hours while we move our data over to the new servers and update the DNS (the system that tells your browser where our servers are when you ask for a page from dailykos.com). The DNS changes could take different amounts of time for people in different regions of the country (or parts of the world).
Why the big move? Daily Kos has long relied heavily on "real iron" servers, which are what you imagine when you think of big machines hosting big websites: boxes in a rack in a data center somewhere that have our name on it. By moving to Amazon, we'll instead exclusively inhabit so-called "virtual machines" in the famous "cloud." In other words, Daily Kos will live in a slice of Amazon's vast network of servers, without being tied to any one specific box as we are now.
Under our old setup, Daily Kos was vulnerable during events like Hurricane Sandy (which actually knocked the site partly offline for a time); with Amazon, we'll have a much easier time dealing with such events. Even better, we'll be taking advantage the flexibility of virtual machines and a world-class data center that offers robust safeguards against failure and will allow us to quickly grow (or "scale," in industry parlance) to meet our traffic needs with ease. This should mean better uptime, faster page loads, quicker response to traffic spikes and outages, speedier deployment of new features, and better-rested, happier developers.
We are also doing a large number of major software system upgrades in the process, covering everything from better "visibility tools" that allow us to chart the site's performance, to a faster, more modern version of Ruby, the programming language that powers all the upcoming new features at Daily Kos. This is the last major planned outage before we upgrade our site's own software, but you'll have to wait to hear more from Markos on that!
Tonight, starting at 9 PM ET / 6 PM PT and running for about 15 hours, we will be taking Daily Kos offline to implement a major upgrade to the infrastructure behind the site. There should be no changes in user experience when the site comes back online, but this work will put us in a good position for major improvements to our platform in the near future.
Actually, I am going to try to upend the AMA formula and instead ask you lots of questions. We're still deep in development on a major release that has taken on the somewhat misleading name of "The Port"- replacing the front page, blog pages, the story page, comments, and the story editor. We've talked a lot about that in the past, and many of you will remember that we were trying our hardest to get that all done, tested and scaling in time for the upcoming elections. We're still pushing hard, coding as quickly as possible while maintaing high technical standards, but I do want to say that that goal is looking more and more difficult the closer we get to it- and we're awfully close. Our plan is to get beta testers on the our testing servers soon to try out everything but the story editor- yes 'soon' is a nebulous non-date. You know I've got real developer roots because I'll talk to scope and complexity, but I won't give you a date. This is where we are- close but nothing ready to be revealed right now. So, instead, I have some questions for you:
- What searches do you run on dailykos.com. Please tell me the simple boring searches as well as the more complicated ones, as well as the motivation behind those searches.
- What sort of information about DailyKos usership, traffic, social network sharing, etc. would best help you focus your writing, organizing and activism efforts on DailyKos?
As you can probably tell, next up are search and public metrics. The machine behind searching the site is largely in place- we'll be reusing the same components we use to power the search of the image library, but I want to be sure we tailor the search interface to how its used and how it could best be used. As for the public metrics the motivation is simple- we use all sorts of metrics to help guide our work- google analytics, quantcast, chartbeat.com, lots of server metrics- and we'd love to know if and how you could see getting benefit from these types of tools for how you use DailyKos.
Pull up m.dailykos.com on your smartphone and you’ll see we’ve brought some big changes to our mobile interface. The tech team has been working hard to bring you the ability to create comments and to recommend both comments and stories from the Daily Kos mobile interface.
Markos has talked about our increasing mobile traffic in the past. Our traffic (make sure you adjust the time range) is growing overall, and the percentage of our traffic that is mobile is on the rise as well. We’re long over due for a mobile interface to Daily Kos that allows for user contribution. I’m excited to see how the features will be used. Already there is a strong indication that smart phone use is categorically different from traditional desktop browser usage of the site. We see an increase in the percentage of mobile traffic for stories that go viral, and a similar pattern during big news events where we all want to stay appraised of the news, even when we aren’t in front of a computer. I have sincere hopes that the mobile site will be used by people at protests and other events as they happen.
And there are other great uses for these features. If you have a bad internet connection, I think you will find the mobile site is lighter- smaller images, less auxiliary content, less complex client side scripts- and runs faster. For those following the tech teams progress in porting Daily Kos from Perl to Ruby on Rails, a lot of the work that went into the mobile site will be reused when we port the front page, story page, and comments to Rails, and this should be a good, early scaling exercise of some of the Rails codebase. And, we are launching the ability to login and signup on mobile to accompany mobile recommends and comments, so perhaps we will see the community grow as well.
So, give it a spin, and let us know what you think.
This the part of the diary, right after the title that says you can ask me anything, where I tell you what you might ask about. As the tech person around here, maybe you want to ask me about the recent launch of the newly updated Image Library? Or mobile campaigns (to check it out, point your phone here)? Or maybe you’ve noticed that signing a campaign is twice as fast now as it used to be? You could ask about new work coming soon, like the ability to recommend and create comments on our mobile site. And there’s always the perennial tech questions, like how to get rid of that one misbehaving ad you hate, how to get images from the image library into the comments, the secret formula that sorts the recommended list. But really, the title is ask me anything, so do your best. Go ahead and ask me anything.
Today we are releasing a number of updates for the Image Library.
The big news is that we’re now making the shared images in the image library available for everybody’s use. This means anybody can dress up their post by pulling from the tens of thousands of images in the library. There are some really wonderful images there, like all these images of wildlife I found in the image library. Putting an image in your story is an easy and very effective way to get more readership. We find that stories with images share much better and grab users’ attention better once they find your story. If you see an image in the library, it's available for your use.
We’ve also overhauled the search so that you can find the image you are looking for quickly. Now that we have more than 70,000 images in the library, many are feeling the need to search on specific attributes, say who uploaded the image. Or, try filtering on staff_uploaded to find quality images purchased for use on Daily Kos. Open the image library and click into the search box for a list of attributes you can use to refine your search.
Also part of this release:
- You can now edit the attribution on multiple images at once. So, if you are uploading a batch of images from one source you can get proper attribution on them all quickly.
- Deleting an image works again.
- Few other small fixes.
- Image Library is now linked from everyone's welcome back box.
This is a rocking horse I built.
It has nothing to do with the deploy.
This deploy is a big one, but consists mostly of behind the scenes performance optimizations. Avatars have been ported to the new rails system. Most users won't notice a change here. A small handful of users will find that their avatar is missing- we no longer support some older image formats like bmp. If you find yourself in this situation, and you would like to keep your old avatar, please file a ticket with help desk. We have added a few new features for you to play with, though. There have been a few changes to the Image Library- we've moved tags up in the property panel so now you can see the tag autocomplete list, we now show you who uploaded an image and if that person is staff, you can search for a user's uploaded images by search for owner:username. And the spambot got a big upgrade- it will now be looking for diary spam.
I'm Jason Libsch, VP of Product and Technology. You can yell at me about pretty much anything software related- the design of new pages, new moderation rules and robots, application performance- or make feature request. Or ask me about anything- they tell me that's the rules. My goal since starting here two years ago has been to make the design of DailyKos speak to the community and content, and to guide the continuing development effort so that the site is a more stable and efficient platform for building and refining user friendly features. From day to day the job varies greatly. This is the most stressful job I've ever loved- trolls, bot swarms, hurricanes hitting our datacenter just days before the presidential election, pie fights, and… reddit users (No, really, I love Reddit, but hey guys, maybe stick around long enough to actually read
the article. Just a thought).
Recently we launched new Sign Up and Log In pages. There are some great stories here, my favorite are the formation and flourishing of the Mentors group who welcome first time commenters to DailyKos, and the 5x (or so) increase in first time comments per day. Looks like the community is growing.
Coming up: comments and recs to the mobile version of DailyKos. We often roll out new features to mobile first as there is about 5x less traffic there, so its a good place to test server performance. We've been working steadily on a new Front Page, Diary, Comment Page, but we won't be able to launch this work until the Story Editor is also rebuilt. Design of the Story Page is in the works, so now is a great time to get your design criticism of the current editor and feature requests in.
In a previous life, I built exhibits at the Exploratorium, and still spend a lot of time building things in my small garage woodshop. At the moment it seems to be frames of one sort or another.
New Front page, Diary Page Design
New Comment Interface
Tonight the tech team released some major site enhancements, mostly centered around an overhaul of the new user process. We've rebuilt the interface for sign up and login, as well as auxiliary pages like lost password, etc. More importantly we have made a suite of changes so that we can bring on new users immediately, with no waiting period before a new users can post or comment. Read on for more details.
Its been a while since tech has released new user-facing features. In the run up to the election we bolstered our infrastructure so as to be able to handle the drastically increased election-season load, and following that epic struggle we felt we deserved a really big nap. Actually, we engaged in, well, quite a lot of housekeeping tasks and reved up the design and development efforts, and now have quite a lot of progress to report.
This is the first of what I hope will be a monthly series where I give an overview of development progress. A lot of the work we do is behind the scenes stuff, so this will be a place to talk about it. Also, I'll try to give preview of features that are in the process of getting built. Feel free to ask questions and give feedback. Be sure to check out links to new designs- they are hosted at a site which allows you to make notations on the designs directly.
We have two site updates to announce. First is a minor mobile site enhancement- in the list of stories on the frontpage, recent list and recommended list (mobile only) you can now scroll back in time... FOREVER. Or, at least for ten years worth of diaries, which in internet time is pretty much the same.
The second change is to the the [New] decorations you see next to comments you have yet to read, as well as the 12/33 New text in stories listed in the side bar telling the number of unread comments for that story. We have implemented a major shift in where the records behind this feature are kept- this data is now stored in the browser, not on the server. This should go a long way in reducing the load on our database servers. It will have the downside that different browsers you use will no longer be in sync regarding unread comments. However, it will have the upside that the new comment text and counts will be available to anonymous and logged out users.