Good code is cheap; it’s operational knowledge that’s holding back big data from solving the great problems of our time.
Solving those operational difficulties with a modular, easy-to-use system was the solution Mark Shuttleworth laid out in his keynote entitled “More Fun, Less Friction” at Apache Big Data in Vancouver in May.
“If we take the friction out, we can unleash all sorts of creativity,” Shuttleworth said.
Shuttleworth is the founder of Canonical and is known for his work on the Linux distribution Ubuntu, which was created for the purpose of making the open operating system as easy as possible to use.
Now Canonical is working on a system called Juju, which models applications and services together to be deployed on any cloud database system. Those models are called charms, and companies from IBM to Cloudera to Couchbase to any number of Apache projects are creating them to run on Canonical’s system.
During his keynote, Shuttleworth spent the first half of his talk — about 8 minutes — spinning up a containerized Linux server on a laptop, an Amazon Web Services server, and a physical server rack using Juju, demonstrating how each environment handled the application cluster with the exact same commands.
“We’re using a common modeling language to describe these different applications, and then essentially binding all of those to the underlying resources needed in each of those environments,” Shuttleworth said. “Why is this interesting? It’s interesting because free software is becoming expensive… Scarcity, what’s rare, is not the code anymore. What’s scarce is the knowledge on how to operate that code.”
By reducing the complexity of spinning up servers to run dozens of combinations of different types of big data technology, and then getting those systems to play nice with each other, Shuttleworth and Canonical are trying to enable people working on big data to focus effort on making money, doing groundbreaking research into things like AI, or just making cool stuff.
“There are 33 big data projects in Apache. How many people know how to operate all of those?” Shuttleworth said. “And the complexity of those architectures is growing. It’s not two apps across two machines, it’s many apps across many machines. And architecture today is really a discussion about how we map those things together.
“We call this a phase change … because all of these pieces of software … they all have that same line crossing — the software is becoming free, but the operation is becoming expensive.”
Reducing complexity means reducing expense, and that means increased adoption and — hopefully — increased results.
Watch the complete presentation below:
