Comcast is the nation’s largest Internet service provider. In 2015 the company had more than 23.76 million high-speed data subscribers and 22.4 million video subscribers. To serve these customers, Comcast deals with a massive amount of data.
Let me give you an idea of how data usage is growing on Comcast’s WiFi network: In 2012, Comcast recorded 1.2 million GB of data traffic. In 2013, it more than doubled to 6.9 million GB of data traffic. It skyrocketed in 2014 when the company recorded 74.8 million GB of data. And in 2015, Comcast recorded 445.8 million GB (or 445.8 PB) of traffic on its Xfinity WiFi network, according to FierceWireless.
To put it in perspective: Comcast’s WiFi data usage grew from modest 1.2 million GB in 2012 to 445.8 million GB in a mere 4 years! And things are going to get even more intense with virtual reality, 4K Xbox and PS4 gaming, YouTube, Netflix, HBO Now, and Hulu all piping their data through network of ISPs like Comcast. Can you imagine how much data Comcast will be dealing with in 2017?
“We’re spending a lot of time and energy figuring out how to make our networks smarter. To extend the capabilities of our network.” Mark Muehl, SVP Platform Technologies at Comcast told me in an interview.
And open source is playing a very big role in Comcast’s empire.
“Today, open source plays an important role throughout that ecosystem and the types of problems that we’re looking at for the foreseeable future, and a lot of them in fact are around the need for us to be more nimble in networking,” said Muehl.
Open Source at Comcast
Comcast is a heavy user of Linux, and it touches everything: from back-end servers to customer facing devices like X1 products. Muehl said. “Comcast, like so many others, is a very Linux-heavy operating system company.”
Comcast’s choice of Linux flavors is interesting. “Generally speaking, we’re more on the open-source side of those Linux distributions than the commercial side of those Linux distributions,” said Muehl. Comcast is using Ubuntu and CentOS.
In addition to Linux, Comcast is a heavy user of OpenStack. They use a KVM hypervisor, and then a lot of data center orchestration is done through OpenStack for the coordination of storage and networking resources with compute and memory resources. Muehl said that Comcast has roughly a petabyte of memory and around a million virtual CPU cores that they are running under the OpenStack umbrella. As an operator, Comcast does a lot of things around operations, and they use Ansible to deploy and manage OpenStack at scale. They also use Cloud Foundry, but according to Muehl that work is in the very early stages at Comcast.
Containers, thanks to Docker, are emerging as the most exciting open source technology. Muehl pointed out that containerization is an up-and-coming area for them.
In Muehl’s own words, “I think it’s fair to say that everything we do in some way touches open source in a pretty significant way.”
Why Open Source?
There are many reasons why giants like Comcast go with open source technologies. Muehl said that there are some defensive reasons, like avoiding vendor lock-in to use open source. But there are actually a lot of positive reasons to choose open source.
If Comcast has a problem to solve, there are three possible approaches: solve it themselves by making an investment in teams and resources; solve it through a commercial vendor that could build a product for them; or work with the open source community.
The last option is where most companies go. As Jim Zemlin, the executive director of The Linux Foundation once told me in an interview, “Organizations have discovered that they want to shed what is essentially commodity R&D and software development that isn’t core to their customers and build all of that software in open source.”
For a company like Comcast, the third option also means lower investment in R&D and much wider support compared to single vendor.
Sometimes Comcast creates a project internally and then open sources that work so that others can contribute to it and use it, said Muehl. He compared open source with biodiversity: the more diverse the technological landscape the healthier it will be. Almost like a biological organism that’s healthier and more resistant to infection.
Be a good citizen
Comcast is not just a consumer of open source; they also contribute heavily to many open source projects. ”To speak generally about open source, OpenStack specifically is one of the larger open source projects that we participate in,” said Muehl.
Comcast has been involved with OpenStack since 2012. “We did a lot of early work around networking because we needed to get IPv6 working. We needed to do some traffic shaping and marking capabilities within the OpenStack infrastructure. All of those have now been upstreamed,” said Muehl.
Stackalytics gives a comprehensive view of the contribution Comcast has made to OpenStack; they have contributed more than 65,000 lines of code — with Neutron being the top project.
In addition to OpenStack, Comcast also participates in many other open source projects such as Apache Traffic Server CDN. Comcast has a GitHub repository where you can see the projects they’ve initiated, but it does not include all of the company’s open source work.
Muehl said that OpenStack is a great demonstration of Comcast’s overall philosophy around open source in that it doesn’t want to just take open source software from the community and use it. “We also believe very strongly in giving back to the open source community.”
