When it comes to infrastructure for your open source project, you are never done, said Amye Scavarda, Gluster Community Lead at Red Hat, and Nigel Babu, Gluster CI/Automation Engineer at Red Hat. One theme during their LinuxCon Europe talk, “Making More Open: Creating Open Source Infrastructure for Your Open Source Project,” is that you can get closer to being done, but there is no such thing as “done” when it comes to infrastructure. Momentum is important – things are always moving, changing, and evolving. The work never ends as you figure out what can be left behind, what should be upgraded and how you can move into the future to incorporate new technologies.
Amye and Nigel talked about how when you start an open source project, you tend to focus on shipping and releasing your code. You don’t necessarily worry too much about how you got there and what you did to get it shipped. In the early days of Gluster, almost everyone had root access to the build machine, since it was only a few people working closely together. Fast forward a few years now that Red Hat has acquired Gluster, and there are many people across a wide variety of time zones working on the project. How to manage communication across a large, growing open source project became a big challenge.
When Nigel first started working on the project, he talked to many people to find the pain points and prioritized his time to work on the issues that were causing the most pain for others. He also had quite a few thoughts about, “How is this even working? It looks like it shouldn’t work.” In some cases, there was no real access control, and people were using some shared accounts. While this may be fine for a small team, it can go horribly wrong as the project continues to grow. In the early days of Gluster, there were plenty of firefighters working to solve immediate issues based on tribal knowledge, but there was very little documentation, which was making it hard to debug problems for new project members. It also created a culture of too many quick fixes, like restarting services regularly, instead of understanding why they failed and fixing the root cause. Rather than just talking to someone about each issue, they are encouraging people to file a bug, instead. With bug reports, you’ll be able to see the patterns and track issues over time.
Amye talked about the “Church of the Shaven Yak” where there is so much to do, but sometimes little progress being made. You pick a piece of the yak to shave every day, and you continue to make good progress, but while you are shaving one piece of the yak, the hair is growing back in another area. Nigel compares this to a marathon where you continue to run, but sometimes those miles seem to pass more slowly, and you don’t feel like you are making progress.
Much of this sounds a bit critical, but Amye stressed that you should be careful not to be too hard on the people who came before you. You should remember that the people working in the project when it was smaller were working in a different environment. They made the choices that seemed right for the project at the time, and the people who come into the project later when the project has continued to evolve probably won’t like the way you did some things. You will meet resistance because you are shaking up people’s worlds, and introducing new things and new processes, but sometimes you need to shake things up as your project grows and evolves.
