May 18, 2009, 3:53 am
A lot has been written on the intertubes about Open Source software development. A tiny bit by me, a lot more than others. What typically isn’t written though are about some of the misconceptions — A lot of folks contributors appear overnight out of the woodwork, that users grow on trees, and that it’s possible to direct community members as if they were employees — not so, of course, and folks get disappointed or discouraged when it doesn’t happen. While I do believe OSS is very rewarding and a great way to make technology move quickly, and having that kind of relationship with users is invaluable, it’s worth taking some time to dispell some misconceptions.. or at least make folks aware of some of the stumbling blocks and how to get around them.
So, a few misconceptions:
(A) Built it and they will come. If you have a niche idea or a solution to an uncommon problem, chances are your userbase will be small. It may be true that your contributor base is 1/50 to 1/500th the size of your userbase, depending on the application — so in small communities, finding people to help you can be difficult. Marketing is actually really hard in most cases, as it’s pretty easy to preach to the choir of your existing user base, but addressing new userbases can be a challenge — especially if you don’t know where they are. Know your userbase and make sure you are planning your app to grow your userbase — work on features that are good for a wide swath of folks when given the chose between breadth-first and depth-first. Ask folks often what they want, weigh that with what others have asked for, etc. Figure out how to reach new audiences and ask your users to help spread your project — LUGs are probably a decent start, as are BarCamps. Similarly, use adoption as a metric — if you have no audience, or a really small one, this may be a sign you need to change your direction (unless it is a just-for-fun project, which is ok too).
(B) Your community will help you build HARD DIFFICULT FEATURE X. Contributors are more likely to work on components that show quick wins. If a large amount of code changes are required to visualize any progress, your average contributor … unless he is heavily vested in the technology, really really likes seeing computers on weekends, or has a very tolerant and awesome boss, is unlikely to work on any such features for long. Even small bugfixes may not be something many users attempt if they get confused by trying to trace a difficult code path. Furthermore, if the feature is “enterprisey”, and not directly useful to them, they are very unlikely to help work on it. There is a light at the end of the tunnel though — maybe a codebase modular and simple to work on, and you can get often get folks to knock out a lot of small items. (Can a module be written to an API without needing to understand the codebase at all? Great!) Not to say there aren’t some rockstar contributors that code large features — but they are generally rare. In order to solve this problem, continually fight to keep things simple and keep the code extensible enough so that no addition becomes unduly burdensome. This is easier said than done.
(C) Directional fallacies. Likely your potential contributors will have lots of different ideas where they can take a project. Your ideas may differ and you’ll be unlikely to manage six directions at once and make sure all those feature directions are getting the right levels of attention. Learning how to bend is important, but so is remembering the reason you are doing the project as well. Figure out how to accomodate and support these directions, but also learn when to not go down rabbit-holes that will prove more of a distraction. If you are strictly after adoption, some of that may be a good idea — but then again, if you are working on a project because it’s interesting to you, or you need it for other purposes, you may have to concentrate more on those core tenets than the new ideas. This is a hard one to manage.
(D) Contributions are free. Each contribution a maintainer gets involves a initial testing cost and a perpetual support cost. If a feature is complex or the maintainer does not understand it, the price of accepting the contribution may be higher than not having the feature. I’m a big fan of the software being 80% good enough, rather than coding to cover everyone’s use cases. A feature that is not used by a large percentage of the userbase is subject to bitrot — a year later it may no longer work, and then you may have to spend time on debugging why it didn’t work. In those cases, sometimes it’s best to leave the feature out. In other cases, it’s better to write a ton of automated tests. For network applications, this is often easier said than done. A lot of tests folks write are more at the “unit test” level than the “functional test” level, and interaction between a wide variety of systems over a network, with a lot of variables, can be one heck of a complicated thing to automate. Anyway, realize the cost of a feature before you accept it, lest you have to deprecate it later or have to spend a lot of time supporting it.
(E) Profit models. I think the OSS consulting model seems to do pretty well, though if your product is free, and folks do not need support, they are less likely to pay you. If you are starting a small project, coming up with an OSS business model may be a challenge. This is not insurmountable, but is something to figure out.
(F) Folks can understand your code. There needs to be a constant battle in any OSS project to squash complexity from the eyes of contributors. There’s a good quote that I like that says “if debugging is harder than programming, if you write code to the best of your ability, you won’t be able to debug it”. True. Assuming you know your domain best, think of your potential contributor pool. Avoid smart, clever, or magic code of all kinds. Code as if you’d want anyone to be able to read and understand it. As cliched as it is, comment well. Never lose sight of this — the code is not for you, it’s for everyone. This is why I generally like the Python philosophy, but in many cases, it too can tend towards magic. Fight for boring code.
(G) Documentation updates. As much as I’d like to see community folks update docs, in general, as the author of the application, you’re the one who’s going to have to make sure the documentation remains in the best state. This is very important for both acquiring contributors and users, for if someone is looking for something and it’s not there, they are more likely to go elsewhere. As you create more documentation, that’s more documentation to keep updated. Documentation is often un-exciting to contributors, and while it is an area where lots of folks have the ability to contribute, it may not always result in that. Taking some round-up to your Wiki is important, though more often than not I see projects erring on the side of having insufficient documentation as opposed to too much of it. A good practice to me is that for every feature that is completed, make sure there is some mention of it somewhere (other than the code) before going on to the next one. While we like to say there are many kinds of contribution and room for everyone, realize that documentation may be so incredibly boring that no one wants to do it. Step up and do it yourself, in that case, it’s important 
(H) Everyone using App X will give back freely and without your asking. While we’d like to think of bug reports as contributions, and everyone’s ideas as equally valuable, it’s also possible to get too many feature requests, and a lot of bug reports to the extent of “I want a pony” — when code contributions and pre-release testing aren’t coming in at the rates you expect. Realize that you benefit a lot for everyone’s ideas, but also realize that you can’t be perfect. Some resistance to patching everything immediately can actually help grow a contributor community in some regards. If you try to do everything, you’re going to get burned out. Learn to like the feature requests and user communition though — this is a huge leg up on proprietary software development, which tends to be bad at establishing these relationships. Ask for help often. It’s ok to let your users know you aren’t superman. If you ask enough, you’ll probably get more help. Folks do not assume all communities are welcoming, or that there is room to join a team. It’s also important to remember that all the bugs aren’t your fault, either. One of the worst things you can do is become a maintainer that says “it’s supposed to be broken like that”, as those don’t seem to build very strong communities. Then again, that too needs to be done occasionally.
(I) Contributors are like coworkers. Often you’ll get a series of patches from someone and not hear from them for a long while. This is ok, this is how the system works. Often this means your application is good enough to satisfy their needs and you should be happy with that. Another problem is that lots of organizations (especially companies not centered around OSS), have policies that keep their employees from working on OSS projects during their free time. We’d all love for this to change, but until it does, realize your contributors time is limited. Often you may see very good features started, but the final rounds of sanding/polishing will probably be up to you. In a similar vein, don’t just reject patches because they are wrong — it’s also important to be able to teach potential contributors how to improve so they can become a frequent contributor to an effort. This can be hard to do, especially over textual communitication media — I often really find myself wanting a holodeck instead of email/IRC. This is just another point towards needing small/simple architectures that are quick to dive in and hack around with. If you are building a more complex system, make it out of other projects that are also easy to understand, at least at a black box level.
(J) Resources grow on trees. In many cases your project is going to depend on community testing, hardware, and so forth if your project isn’t bankrolled by a large corporation. Therefore, community size == strength. Realize that of your potential userbase, those that will actually get involved in pre-release testing is quite low… so ask for help often. Most of your testing will probably still happen after a release, making production bugs a likely reality — plan for this by releasing often. While some distributions offer testing branches, in many cases, depending on the application, they may not be widely leveraged.
(H) Cross distribution support is easy. Cross distribution support and getting folks to package and maintain an app is hard — it’s a decently sized commitment, and often requires someone fluent in a couple of distributions. I like focusing on upstream, but close attachment and packaging for a distribution is really a good way to find an audience.
If cross-distro inclusion is really important to you, you are probably going to have to do it yourself. Seeing a community Windows port is probably the most unlikely thing to happen. If you really want the wider community, step up and put in the time yourself. I’ve seen a lot of projects that say “support for X is partial/TBD”, including my own, and these things are often rather hard to make as first-class-good as the original OS support.
(I) Users help users and it’s ok to just be a developer. Oh they do, a lot. This is great to see. But it’s not true that you can simply work on your applications and expect the users to help themselves solve all of their problems. It’s important to make sure that questions asked get answered, in a decently timely manner, in order to build a userbase that CAN help other users. If you answer a question once, a user can help answer that question for someone else. In this sense, running a project does involve a ton of customer support. Folks can be very willing to put up with complexity or problems as long as there is someone to turn to, in other cases, a project may not have a good user community and they may look for other applications that are simpler or do have those support networks. Bottom line: the userbase builds the contributor base, so don’t neglect the userbase.
(J) Cross-project collaboration is easy. This is just plain hard. Often different projects have different release schedules. They often have different goals. There are often a lot of folks really interested in building technology rather than apps (nothing wrong with that in many cases), so there can be a lot of NIH or reinveted wheels. How many text editors do we have? In general, it may be very possible to work with other products via web services APIs and the like, or use libraries. This is all great. However if you see a project that is similar to your project, it can be a real challenge to join forces and become one project. It may often seem that projects are copying each other feature by feature. I suppose in this way competition is good, but it can be a bit weird to think of open source and competition in the same sentence, as opposed to collaboration. I think this is more of a reason for looking to build user/developer collaboration… though I think there may be some room for cross project sharing. Making more things available as components and libraries in your own project may be a good start, so that other projects can share those components. In many cases this is a good thing though — if your project gets slow/lazy/bad, there’s something to replace it on the horizon When that doesn’t exist, say, in the case of web browsers, we often see things trend to sluggish or lack of innovation. I’d say take a look at all the clones and find the best ways to borrow ideas from them as best as possible, even if appeals to join forces are not met with interest.
This competition is a good thing.
Anyway, not trying to be doom-and-gloom or scare anyone, OSS is fantastic. Although don’t expect your external community to cure all your problems overnight. Realize where the roadblocks are, and instead focus on what OSS is particularly good at — namely, building a strong community of users and developers, establishing powerful feedback loops, pulling in ideas, gathering small and mid size contributions (modular code, low hanging fruit), and taking advantage of some supplementary community testing. In other areas, you can still achieve great positive results with your community, but understand their limitations and how you can work to move beyond them. Some of the community augmentation you’ll get will be small. The community network multiplier effect may double or triple your power some day, but initially it may only feel like a 5% boost (and even when it’s running on full stream, some days, it’s a power drain). Recognize what the community is good at — and what you need to do to enable that — and you’ll probably get more mileage from your project.
