Netflix Outage Sparks Commitment To Improvement By Company

    January 2, 2013
    Chris Crum
    Comments are off for this post.

On Christmas Eve, Netflix experienced a widespread outage of its streaming services due to issues with Amazon Web Services. If you tried using Netflix that day, you may have noticed, though not everyone was affected (trust me, I did plenty of Netflix watching over my holiday break). If you were online at all that day, chances are you probably at least saw someone complaining about it or reports about the problem.

Customers were affected in the U.S., Canada and Latin America. Subscribers in the UK, Ireland and Nordic countries were not affected at all.

On Monday, Netflix put a post up on its Tech Blog to explain what happened. Essentially, the disruption in streaming services was impacted by problems with Amazon’s Elastic Load Balancer (ELB), which routes network traffic to Netflix.

Both companies have apologized for the outage, which began about 12:30 PM (Pacific), but “grew in scope” later in the day.

“Netflix uses hundreds of ELBs,” explains Netflix’s Adrian Cockcroft in the blog post. “Each one supports a distinct service or a different version of a service and provides a network address that your Web browser or streaming device calls. Netflix streaming has been implemented on over a thousand different streaming devices over the last few years, and groups of similar devices tend to depend on specific ELBs. Requests from devices are passed by the ELB to the individual servers that run the many parts of the Netflix application. Out of hundreds of ELBs in use by Netflix, a handful failed, losing their ability to pass requests to the servers behind them. None of the other AWS services failed, so our applications continued to respond normally whenever the requests were able to get through.”

The Netflix site itself was up and running throughout the incident, so customers could still access some functionality, and new customers could sign up (Update: Some claim they have been unable to use various site functionalities). According to the company, streaming from Macs and PCs was only slightly reduced from normal.

“Even though Netflix streaming for many devices was impacted, this wasn’t an immediate blackout,” says Cockcroft. “Those devices that were already running Netflix when the ELB problems started were in many cases able to continue playing additional content.”

The company says it is working on ways to extend its resiliency to handle regional outages.

“It is still early days for cloud innovation and there is certainly more to do in terms of building resiliency in the cloud,” says Cockcroft. “In 2012 we started to investigate running Netflix in more than one AWS region and got a better gauge on the complexity and investment needed to make these changes. We have plans to work on this in 2013.”

On a related note, Netflix says it sees higher usage on Christmas Day, with increasing streaming rates until people go back to school and work.

  • David Tripp

    Bull! My DVD management, queue, and ordering have been out completely since early New Year’s Eve.

  • Rho

    Has to wonder what people did before Netflix. Really not the end of the world here people. Put on your big boy/girl pants and get over it. Sh!t happens. Technology is not perfect and neither are you. They will fix it and try and make it better so people like you will quit btchin’….The End.

    • Thomas Lytle

      Boy!… 1. why are YOU so angry? 2. Perhaps you don’t care cuz you have high priced cable (good for you), but the fact is that Netflix has TONS of problems. And since we pay every month out of our bank accounts for the service, it should be at least as reliable as cable. Netflix however, even bigger than the outages, has constant issues playing video and connecting, and major speed problems to boot. If there were a better option than high priced cable, we would do it. Perhaps YOU should quit bitchin about other people’s complaints which are not intended for you anyway, and get a life! I normally don’t comment but you pissed me off.

      • LT

        Wow! That guy you commented on didn’t seem angry AT ALL. He’s right on all counts…technology isn’t perfect, and sh!t happens from time to time. I use Netflix, having fired Bell from my home ages ago. Sometimes it doesn’t work perfectly. No big deal. I just go for a walk or read a book.

  • Josephine

    Are you kidding? I haven’t had access to my queue since last week! I called because I thought I was having trouble on my end, but they said they would have it up and running shortly. I would like a few extra DVDs (more than two at a time) free of charge instead of the two at a time for a month to make up for this. This is really outrageous for it to happen over the holidays when so many people are relaxing with family or on vacation and want to watch movies. Why would Netflix decide to work on something like this during this time of year is not a professional decision. Next time you want to improve something please choose a better timeframe. –disgruntled customer…

  • http://yahoo fred n

    read a book!!!!

  • LT

    Anybody have an LG TV and just got the latest firmware update? My Netflix has been offline since then…but only on the TV