After a year of open source, Second Life looks ahead

A year has passed since Linden Lab, maker of the popular 3-D virtual world Second Life, released the source code to its Second Life viewer under the GNU GPL. In that time, a viable community of developers has grown up around the SL code, and the company is pleased enough with the success that it has branched out further into open source and open standards.

The SL viewer code was released on January 8, 2007, and has spawned multiple unofficial offshoots. Most of the unofficial viewers listed at the SL wiki are individually maintained and focus on adding specific features. Some add or enhance existing features such as the particle system, some integrate bug fixes not yet rolled out to the official viewer, and some even roll back changes made in the official viewer.

But according to Linden’s Liana Holmberg, building alternate clients is just one facet of SL’s open source community. At SCALE last month, she listed three other growth areas from the past year. Open source developers have taken on the task of porting and packaging the official SL viewer for different Linux distributions. They have also contributed heavily to the development of the Windlight add-on, an in-world atmospheric renderer recently integrated into the official viewer. And finally, volunteers have contributed bug reports and patches to the official Linux SL viewer, accelerating its move from alpha to beta status. At the end of 2007, Holmberg says, there were 719 subscribers to the development mailing list, and 64 outsiders had contributed patches.

Holmberg also emphasized that Linden’s belief in open source extends beyond the enhancement of its own viewer code. It has invested developer time and upstream contributions in third-party projects such as Mercurial, SDL, OpenAL, and OpenJPEG, and it has released source code to several utilities developed in-house, such as the networking library Eventlet and the Web service framework Mulib.

By the summer of 2007, Linden decided it would recognize outstanding contributions by the open source community with its Linden Lab Innovation Awards. Winners were announced in August, with the winner of top prize Contributor of the Year receiving a MacBook Pro.

In September, Linden’s Architecture Working Group held a meeting to map out how to formalize, stabilize, and refine the protocols that run SL — both on the server-side and between clients and servers.

Linden looks back

We asked Linden’s Director of Open Source Development Rob Lanphier for his thoughts on the progress made in 2007. “As individuals, I think all of us here have learned a lot of different things,” Lanphier said. “Many of us are new to open development, so a lot of the learning over the past year has been the basics of community engagement and building relationships with outside developers. One really cool thing we confirmed is that open sourcing from a market-leading position is especially effective. Since most corporate open source is instigated as some form of underdog or comeback strategy, most corporate contributors have to bend over backwards to attract a community. Coming from a market-leading position means the community is more inclined to meet you halfway.

“But that leads to another big lesson: there seems to be a big gulf in expectations between corporate open source project leaders and the community attracted to those projects. It takes a lot of give and take on both sides of the equation to build a really robust and functional community. We know we have a lot of work to do to facilitate better collaboration (e.g. live version control, better build tools, better bug processes). As I alluded to, many of our staff are still learning the basics about working in the fishbowl. We’ve learned that, if nothing else, we can and should get design feedback when possible.”

Linden’s open source viewer work has helped the company in other areas, Lanphier says. “A great technology choice that our community helped us make was moving to CMake, a cross-platform build tool, which makes it much simpler to maintain makefiles for many different platforms.” The developer community found Linden’s old configuration system limited and frustrating, and Lanphier credits the persistence of community member Callum Lerwick with leading to the CMake solution.

In addition, Lanphier says that the availability of the SL viewer source code “reduced a lot of friction” in Linden’s business dealings. It accelerated the testing and integration of Windlight, which began as a proprietary product. And the accessibility of the code makes it more approachable even to third-party companies who are interested in commercial licensing.

Asked about the development of the SL viewer itself, Lanphier expressed satisfaction. “The personal-bugfix viewers are currently the most popular of the alternative viewers because they’re of the most immediate utility. There are more radical experimental viewers like the stereoscopic viewer from University of Michigan or the brain tracking viewer from Tokyo University that are too experimental at this stage for widespread use.

“We think we’re in a good place. We know there’s a lot more we can do to help make the community and ourselves more productive, and we could have waited until we had a big staff or even all the tools developed and in place before releasing our code. However, because perfection is the enemy of the good, we didn’t wait…. In the long term, we’re not hoping for a lot of alternate viewers so much as a flexible general-purpose viewer with better capabilities for extensions.”

Looking toward year two

Linden plans to repeat its Linden Lab Innovation Awards in 2008, making the recognition an annual affair, and the Linux SL viewer is expected to graduate from alpha to beta status “very, very soon,” according to Lanphier.

But client-side development will not be the big story in 2008. Holmberg and Lanphier both say that developing open protocols is the most important facet of growing 3-D virtual worlds.

The 3-D Web, as Holmberg puts it, has to be as open as the existing Web in order to be useful. SL is one virtual world, she says, but as others appear — such as internal corporate spaces used for product development — moving from one world to another raises the same sorts of identity, security, and synchronization problems faced on today’s Web. Developing SL’s protocols in the open will help ensure interoperability and sustained growth.

It is to that end, Lanphier says, that Linden formed the Architecture Working Group. “We have a strong interest in opening large portions of our server infrastructure, because providing our insight into our existing implementation will help guide the creation of the most accurate and complete documentation.”

But don’t expect to see the SL Grid server source code released any time soon. “There are a number of things about the architecture of our current server infrastructure that inhibit a source code release. There are a lot of cases of monolithic design and improper trust relationships between components that would need to be addressed.”

“We’ve been putting a lot of effort into re-engineering the back end systems and the way in which they expose formal APIs,” Lanphier says. “This is largely based upon using Mulib and Eventlet to turn our internal and external APIs into highly modular and secure Web services. We’ve designed a security model based on granting capabilities to use particular APIs, and we’re separating out the management of ‘agents’ (resident information such as appearance and inventory) from the management of ‘regions.’ This breakup will allow residents to log in once and move between regions operated by different hosting providers, and the use of fine-grained capabilities will allow for servers in our agent domain to grant different levels of information access/manipulation to region simulators depending on which region provider the agent is visiting.”

The Architecture Working Group plans to eventually hold quarterly meetings, but at this time has not scheduled its next event.

Second Life wants you

Holmberg and Lanphier both recommend visiting Second Life before getting involved in the open source community that surrounds it. Gaining familiarity with the project is important, Lanphier says, but the in-world environment offers a community of its own. “Second Life is a platform for expression, and a big part of that is scripting in-world objects. We have a fantastic in-world open source community using just that. Our scripting community is far larger than the open source community around our infrastructure.”

Holmberg encourages open source developers interested in getting their feet wet with SL development to visit secondlifegrid.net/programs/open_source. The usual entry points of bug reporting and triage are valid for Second Life, she says, but suggested particular projects as well, such joining the porting effort to bring the SL viewer to your favorite distro, or finding a proprietary component in the viewer code and replacing it with an open source equivalent.

Lanphier emphasizes that SL is still very new, and as such stands to benefit from the active involvement of open source developers.

“Right now, virtual worlds look a lot like the Web in 1994. People often have a difficult time imagining what this will be like in five to 10 years when a lot of our current problems are distant memories. The technology will improve and new business models and enterprises will drive investment in this area. As that happens, virtual worlds will encroach on more and more of a person’s online experience and will affect and interact with all kinds of industries, activities, and occupations.

“If you are a software developer, you have a vested interest in making sure that we get this right. If you’re really passionate about ensuring software freedom, you’ll want to make sure that virtual world technology remains rooted in a foundation of open standards and open source. It’s an incredibly exciting time, where each one of us has the opportunity to shape what the future of being online will look like.”