Despite being a library that most people outside of the technology industry have never heard of, the Heartbleed bug in OpenSSL caught the attention of the mainstream press when it was uncovered in April 2014 because so many websites were vulnerable to theft of sensitive server and user data. At LinuxCon Europe, Rich Salz and Tim Hudson from the OpenSSL team did a deep dive into what happened with Heartbleed and the steps the OpenSSL team are taking to improve the project.
The bug, itself was a simple one where the code didn’t check a buffer length, Hudson said. The bug had been in OpenSSL unnoticed for three years by the team member that checked in the code, the other team members, external security reviewers, and users, even though the commit was public and could be viewed by anyone. Hudson pointed out that “one thing that was really important is all of the existing tools that you run for static code analysis, none of them reported Heartbleed.”
Salz talked about how overworked and overcommitted the lead OpenSSL developers were, which was one of contributing factors to this issue, since at the time of HeartBleed, there were basically two developers, barely making enough money to live. OpenSSL was an open source project that barely got $2000 a year to keep going, so the developers had to do consulting work to make money, which made it difficult to find the time for them to address bugs and patches coming in from other people.
Hudson described Heartbleed as “a wake up to the industry and those commercial companies that were effectively getting a free ride on OpenSSL,” which led companies and organizations to realize that they needed to do something about it, instead of relying on just a couple of people who are too poorly funded to maintain such a critical piece of infrastructure.
As a result, The Linux Foundation set up the Core Infrastructure Initiative (CII) and effectively got a group of a dozen or so commercial companies together to be able to offer funding for not only OpenSSL, but other critical projects that are underresourced. One of the goals was to get more infrastructure, more support, and more ability to address the issues so that better processes can be followed.
As of December 2014, six months after HeartBleed, there were 15 project team members. Two people who are fully funded by the Core Infrastructure Initiative to work on OpenSSL as their day job, and two people funded to do the work full-time based on the donations that came in from people who were concerned, Hudson said.
Today, they have policies for security fixes and a release schedule with alpha and beta releases for people to test, which has worked reasonably well according to Salz. They have a code of conduct, and mailing list traffic has increased and become more useful. Salz says that “there are other members of the community now contributing answers to questions; members of the team are responding more quickly and rapidly; and we seem to be more engaged in having a more virtuous cycle of feedback.”
Downloading releases, submitting or fixing bugs, and answering questions on the mailing list are great ways to get involved in the project now.
Hudson described a couple of lessons learned. You can’t rely on any one individual, no matter how good they are, to not make mistakes. Also, people really need to take time to understand the code in detail when doing code reviews, and everything going into the project needs to be scrutinized.
For more lessons learned and other details about the OpenSSL project both before and after Heartbleed, watch the video below.
Interested in speaking at Open Source Summit North America (formerly LinuxCon) on September 11 – 13? Submit your proposal by May 6, 2017. Submit now>>
Not interested in speaking but want to attend? Linux.com readers can register now with the discount code, LINUXRD5, for 5% off the all-access attendee registration price. Register now to save over $300!
