Greg Kroah-Hartman is a superstar in the open source world. He is a Linux Foundation Fellow and the maintainer of the stable branch of the Linux kernel, the staging subsystem, USB, Linux drivers, userspace I/O, TTY layer…the list of his work is quite long. He was also the creator of openSUSE Tumbleweed, a rolling release distribution.
He is one of the friendliest faces of the Linux kernel community; always ready to talk and help. I met him again at CoreOS Fest this week in Berlin. (But only after he was done releasing the next Linux kernel 4.6 release candidate while everyone else was listening to the first day’s keynote.) We talked about Linux, security, the upcoming 4.6 release planned for May 15, and more. Below is an edited version of the discussion.
Linux.com: What is the hard truth about Linux kernel security that many people don’t want to hear?
Greg Kroah-Hartman: You have to be able to run a system that can upgrade itself. Lots of people think if they stick with the kernel and if nothing changes, it’s good. That’s not true. We’re fixing about ten bugs in the kernel every day. Not all of them are security issues, but sometimes the big problem is we don’t know if an issue is a security issue or not.
There’s an infamous bug with a fix in the TTY layer. We made it and we merged it and everything was fine. Three years later, somebody realized, ‘Hey! I can use this and get root!’ All of a sudden, this bug that we have fixed years ago had to be backported to really old enterprise kernels, because if you’re running those, all of a sudden you had a root, you had to exploit those a long time. We had a really hard time, when we fix bugs, getting those bug fixes to users. That’s a hard problem.
I’ll pick on Android the opposite way. My phone is running a 3.10-based kernel. That was a long-term stable kernel, but they never updated it. There’s some well-known easy ways to get root on my phone …which is great, because I like getting root on my phone, but that’s already been fixed. Fixes are pushed publicly, but they’re not being updated. We have to be able to update it. That’s something that you really have to do.
Linux.com: What is the possible solution for Linux so that users can easily keep it updated?
Kroah-Hartman: You have to design your system so it can update itself. The Chrome OS and then the CoreOS teams adopted the same mentality. You have two system images. You’re going to update one. Once you know it works, it can switch over to the other one. You have to be able to update it in a secure way. This technology’s been proven. It’s solved. People just need to use it, and build it into their systems. The kernel is not going to go around updating itself on its own. It’s up to the infrastructure you built for your product.
If you make a product with Linux and you can’t update it, or any piece of software, it’s dead. The environment changes. We’re in a world and the joke is “The only thing that’s constant is change.”
Many companies and countries are switching to Linux. Japan is switching a lot of its infrastructure to Linux. All the power plants, all the streetlights, are going to be running Linux. They want to support that for 20 to 30 years. They’re building in that ability to securely update them from the very beginning. They know they’re going to be able to have to do that. That’s good. That’s the way you need to do it. The products are out there. The base technology’s been long-time proven on how to do this. You just need to do it. Even Android can do it. They just need to spend the time to actually push their updates out and pay attention to it.
Linux.com: There is this mentality in the server space that once you install and set up your server and it’s working, don’t touch it…
Kroah-Hartman: That mentality works really good when you have a server that doesn’t talk to anybody else. But in the real world you have to talk to somebody else. If you’re going to take a server and put it over in a corner and it’s not going to ever change anything, not going to talk to the world, great. That’s fine. That’s a static environment. Once you get into a dynamic environment, you have to be able to update. People need to embrace change. They need to get over that fear of change doesn’t work.
Linux runs the world. Our rate of change is unheard of. We’re running almost eight changes an hour, 24 hours a day, to our kernel. It’s one thing to take these bug fixes, ten bug fixes a day, but you also need to take advantage of the new features. We’re adding new features for security reasons. We’re adding these airbags to the kernel.
The new release that came out May 15 has write-only protection to all the data structures. If a bug happens where you would normally be able to overwrite a portion of memory, now with the added protections in place, you aren’t allowed to do that so the bug does not cause any additional “harm.” All of a sudden we took out a whole class of exploits that a bug could turn into an exploit. If you don’t update your kernel to a new one, and you’re just trying to do bug fixes, all of a sudden you’re not going to get that new feature. You need to be able to embrace that and update for these new features at the same time.
Linux.com: How do you reverse that mentality that change is bad, and encourage software vendors to embrace it?
Kroah-Hartman: Most vendors know that today. Red Hat, SUSE, Canonical all offer these services to their users. They all offer the ability to update on time, update on the fly, on CoreOS. It’s there. They’re pushing out the security updates. People need to use them, to take advantage of them. The big problem is Linux and the Internet. They need to be able to build a system in an easy way that can be updated instantly. They just have to do it.
Linux.com: What are The Linux Foundation and the kernel community doing to address security issues?
Kroah-Hartman: The Core Infrastructure Initiative is a great thing from The Linux Foundation. Lots of companies around the world are sponsoring it, and helping improve internet security. They are also sponsoring kernel security work. At the Kernel Summit last year, Kees Cook did a presentation about all the things we need to do better.
One of things we need to do better is, we need airbags for the kernel. Konstantin Ryabitsev gave a big presentation a year ago at the Linux Security Summit and said that we need ways to protect ourselves. There’s a whole bunch of things out there that we need to do better for the kernel. Some of them make things harder for developers of the kernel. We need to be able to accept that and make that change and move on, because our real users of the kernel are not the developers of the kernel.
That happened at the kernel summit last year. Since then, we’ve been going through and doing a lot of work. We have people working on a lot of things: taking bits and pieces of the GRSec, the large security patch set, taking them and merging them into the kernel as needed and doing some other work. CII is helping fund that. A number of the developers working on that are just being funded by the companies that they work for such as Google, Red Hat and Intel. They are doing a lot of work to improve kernel security.
Editor’s Note: This article has been updated to clarify security changes in the 4.6 release.
