In an ideal world, everyone would implicitly understand that it just makes good business sense to upstream some of the modifications made when creating your Linux powered devices. Unfortunately, this is a long way from being common knowledge.
By Martyn Welch, Senior Software Engineer at Collabora.
In an ideal world, everyone would implicitly understand that it just makes good business sense to upstream some of the modifications made when creating your Linux powered devices. Unfortunately, this is a long way from being common knowledge, and many managers still need convincing that this is, in fact, in their best interests.
Just so that we are clear, I’m not suggesting here that your next Linux powered device should be an entirely open design. We live in the real world and unless your explicit aim is to produce a completely open platform, doing so is unlikely to be good for your companies’ profitabilty. What does make sense however is to protect the parts of your product that drive your value proposition, while looking for ways to reduce costs in places which don’t drive the value add or unique selling point. This is where upstreaming and open source can offer you a massive advantage, if done right.
Say you have a new product in development, with a number of cool features to implement that you hope will drive customers to your door. You also have a new hardware design, thanks to the hardware guys that have discovered some funky new devices that optimise and improve this new design. You’ve also picked up the SoC vendors’ slightly outdated kernel tree and discovered that a number of these devices already have some support in the kernel, awesome. For others there is no support, either in the vendors tree or in the mainline tree, so backporting isn’t an option, and you’re looking to write some drivers. You’ve heard something about upstreaming and would like to give it a go, but you’re wondering if this is a good idea. Is this going to help my company? Well, the answer is generally “Yes”.
Uptreaming is the process of submitting the changes that you have made, typically, to existing open source projects so that they become part of the main (or upstream) codebase. This may be changes to support specific hardware (usually kernel level changes), changes to fix bugs that you’ve exposed via your specific use case or additional features that may extend existing libraries that you use in your project.
Upstreaming provides you with a number of tangible advantages which can be used as rationale to help convince your management:
- You gain at least one 3rd party review, by a domain expert, giving you confidence in the quality of your changes.
- You decrease your delta with the upstream codebase, reducing the maintenace burden of your product (you do security updates, right?), providing product updates and potentialy when creating the next version of your product.
- Community suggested improvements, providing you with ways to reduce your code size whilst simultanously increasing available features.
Let’s use the Linux kernel as an example (one which many product developers will likely need to modify) of how these benefits manifest, as this is the project that I am familiar with.
Changes submitted to open source projects are not blindly accepted. Projects need to take some care that the changes are not going to negatively impact other users of the project that may have other use cases, and must also ensure that the changes are both sensible and done in a way that safeguards how the project can be maintained in the future. As a result, changes may need to be altered before being accepted, but such changes are likely to have a positive impact on your modifications.
The reviewer (who is very likely to be an expert in the area in which you are making changes) may be able to point out existing infrastructure that can be used to reduce code length and increase code reuse, or recommend changes that may remove potential race conditions or fix bugs that may not have been triggered during your testing. As the kernel (like most projects) expects a specific code style, there may be requests to change code to meet these requirements, as a consitent code style makes maintenance of the code easier. Once merged, the maintainer will be taking on the burden of maintaining this code, so he will want to ensure this can be done efficiently.
The upstream Linux kernel code base is being modified at a very fast pace, with a change being merged at a rate of one every 7 minutes. Different parts of the kernel develop at different rates however, some seeing a higher rate of change while others undergo little to no change at all. Should you have local patches, there is an increasing likelihood over time that these will be incompatible with the ever-evolving kernel.
This means your developers will need to spend time making modifications to the local patches when updating the software stack on an existing product, or when attempting to re-use these patches on a new product. Conversely, when local patches are applied upstream, existing code will be changed when APIs change, generally resulting in the modifications continuing to function as required in your use case without any engineering effort on your behalf.
Once a driver is added to the kernel, for example, others may add features to the driver that weren’t of immediate use to you. As your requirements change and grow for updates and subsequent revisions however, such changes may prove very useful to you and would be available with minimal work. A well documented example of this is Microsoft’s submission of hyper-V support. This large body of work was initially added to the “staging” area, an area where drivers that aren’t ready for full inclusion in the kernel can be put to enable them to be modified and improved with the help of the community. Whilst in the staging area the drivers were greatly improved, the drivers were modified to comply with the Linux Driver Model, reducing the code line count by 60% whilst simultaneously significantly improving performance and stability.
Of course, there are also less tangible reasons for contributing upstream. As a company, if you are planning to utilise Linux and other free software in your products, it is likely that you will want to hire talented, experienced developers to help you create your products. Contributions made to open source projects relevant to you are likely to be noticed by the very same developers that you hope to atract to your company, and will also reflect well on your company should they be looking for new opportunies or should these developers be asked if they have any recommendations for good places to work.
Submitting upstream, and contributing to an open source project, can also be a very rewarding experience for existing employees. By actively participating in a project on which your products are based, they not only gain direct access to the community behind the project, but also get a better understanding of the projects’ inner workings, enabling them to build future products more efficiently and confidently.
