Ubuntu 11.10 is an important step forward in cloud computing for the Ubuntu Project. With this release, we’ve introduced some new features that will help our users set up and manage cloud services. One of the most important features in this release is juju. Formally known as Ensemble, juju packages the intelligence of installing, configuring, and managing services. Let’s take a look at what juju is, and what it can do for you.
Virtualization and more recently Infrastructure cloud computing have been major buzz words in the IT field the past few years. Cloud believers see a world where the cloud changes everything, and there is some truth in that. The reason why many IT people see a lot of value in the cloud is because it layers an automation layer (with an API) on top of a data-center. This alone enables rapid access to various compute resources. With an API call, a click of button or executing a command, servers can be created or destroyed, storage can spring up to existence and various virtual networks are created to service the virtual datacenter just created. The speed and low entry barrier of cloud computing help businesses and even kids in a garage experiment with new software technologies. The automation ensures a repeatable well controlled deployment that IT professionals like, and since almost everything is automated it is cost effective as well. Indeed such cloud solutions are a great “first step”!
Having resolved the Infrastructure on-demand problem, as well as having a rock solid server platform providing timely updates of well integrated components such as Ubuntu Server. DevOps are more and more caring about higher levels of the stack. Having rapid and automated access to servers and storage is a needed first step, but an IT professional deploying a large scale Web application cares more about “Rapidly deploying a MySQL server with multiple read slaves,” “Tried and true tuning of mysql configuration for optimal efficiency,” “A working backup and monitoring strategy for the database and Web server fleet” and so on. A potential solution to that is PaaS cloud environments, that try to solve the issue by offering runtime environments that contain standard configurations for common components and frameworks (databases, application servers… and so on).
The Ubuntu server team however had a far reaching vision of creating a software that would enable users to very rapidly assemble their own software-services (databases, caching, app servers..etc). One way to look at it, is users would be rapidly creating their own PaaS environments that fit their needs, from off-the-shelf components! Want that tuned scalable MySQL server, grab that “service” and deploy it, fancy application servers, caching, monitoring or backup services? You simply deploy such services and hook them up together.
Enter juju
Juju goes “beyond packaging”. While traditionally software packaging focuses on packing software, the actual binary bits, juju packages the intelligence of installing, configuring, connecting and managing the services you are deploying. Juju formulas are where authors embed and codify their expertise in order to share it with the rest of the community.
Juju is all about reuse and composition — you can re-use other people’s juju formulas with no modifications, and you can (and most likely will) couple multiple services together composing a bigger deployment that works as a single unit.
Juju formulas can express deployment best-practices, and through sharing and collaboration, juju allows those best practices to spread through a community very efficiently. Turn HOWTO’s into formulas and see your application deployed faster and more effectively.
Core to the juju vision is a public repository of re-usable juju formulas for popular application deployment scenarios. Those formulas are produced collaboratively, like a traditional Linux distribution, and follow the same pattern of governance as Ubuntu. Whatever you need to deploy, start by looking for a formula you can improve on, or start your own!
With juju you can deploy services across multiple cloud or physical servers and manage those services via a handful of commands, built on the strong foundation of Ubuntu Server.
Check It Out Now
Enough talk, let’s what juju can do today. Let’s deploy a multi-tier LAMP application to the cloud, connect all the pieces together and see how it fares. In this article (the first of a series) I will not try to delve into too many details, we’ll keep things simple and only mention what you need to know, however you can ask detailed questions on our mailing list. If you’d like to actually run the examples provided here, you’ll need to install juju (I recommend the latest version from PPA) and you’ll need to give it your Amazon AWS key. I won’t repeat the docs, you can find all the getting started info in the juju documentation.
Bootstrapping
The first thing an juju environment needs is a bootstrap node. This node is a utility node that is used to manage the environment. Juju is rapidly progressing, and in the future this node’s role might not need its own dedicated node, however for now it is needed. Once you have your environment.yaml file set (as per the getting started section) you can go ahead and bootstrap:
juju bootstrap
Let’s also download the needed formulas. In the future juju will be able to search and download formulas on its own. But, for now, this step is needed.
bzr branch lp:principia/mysql
bzr branch lp:principia/mediawiki
bzr branch lp:principia/memcached
bzr branch lp:principia/haproxy
Deploying an Initial Set of Service
Juju operates at the level of a “service”. You deploy services to the cloud, and you manage them. The low-level “steps” needed to create a mysql service are defined in the mysql formula. Let’s see an example of deploying services.
juju deploy --repository=. mysql wiki-db
juju deploy --repository=. mediawiki demo-wiki
juju deploy --repository=. memcached wiki-cache
juju deploy --repository=. haproxy wiki-balancer
The first line deploys a mysql service, from the formula called mysql. The resulting service name is “wiki-db,” this is an arbitrary name you assign to your service. The formula repository is the current directory. The same happens in the next three lines where mediawiki, memcached, and haproxy services are deployed and assigned service names.
Deploying each of those services results in spinning up a new Ubuntu server cloud instance (since we’re choosing to deploy to the Amazon EC2 cloud in this case). The Ubuntu server instance is a plain vanilla instance, nothing special is embedded in that image. The instance picks up instructions passed to it from the user data, and starts executing formula instructions to become the intended service (for example, mysql)
Scale It Up
For a more practical deployment, let us increase the number of media wiki servers to two, let’s also do the same for memcached.
juju add-unit wiki-cache
juju add-unit demo-wiki
That is exactly how simple it is to scale-up a service (even in production!), since juju already knows everything it needs to know about the demo-wiki service, you only need to ask it to add or remove instances from it.
Connect the Pieces
Up till now, we have deployed disparate systems. Each node has a set of packages installed, and is “ready” however it is unaware of its environment, it is not connected to the rest of the services. It’s usually a eureka moment for a first time juju user when she connects together all the pieces, and sees how services start configuring themselves to fit into the environment. Let’s see how easily that is done.
juju add-relation wiki-db:db demo-wiki:db
juju add-relation wiki-cache demo-wiki
juju add-relation wiki-balancer:reverseproxy demo-wiki:website
Let’s try to understand exactly what is happening here, and why this is very interesting! The first line connect our database (mysql) service, to the two nodes of the mediawiki service. Once connected, mysql will realize the mediawiki service has no DB created, thus a DB will be created for it, and the username/password/URL for connecting to that DB will be sent to the mediawiki service over juju provided communication channels. Mediawiki then rewrites its configuration to make use of the newly created database.
The second line connects mediawiki to the caching service. Again similar things start happening, where memcached configures itself to be a backend caching service for mediawiki, and mediawiki reconfigures itself to make use of the memcached service.
Finally the third line causes the mediawiki to publish its details to the haproxy service. Accordingly haproxy starts reconfiguring itself to serve requests to the (currently two) nodes in the mediawiki service. All the magic under the hood is handled by juju formulas that can speak a common language resulting in composable services that you the ops person, can start connecting together to create larger systems.
What Have We Now?
Let’s run juju status to see what we’ve got now. Here’s the response:
machines:
0: {dns-name: ec2-67-202-29-40.compute-1.amazonaws.com, instance-id: i-90d616f0}
1: {dns-name: ec2-75-101-233-145.compute-1.amazonaws.com, instance-id: i-22dd1d42}
2: {dns-name: ec2-50-17-119-109.compute-1.amazonaws.com, instance-id: i-dcdd1dbc}
3: {dns-name: ec2-50-17-143-48.compute-1.amazonaws.com, instance-id: i-8edd1dee}
4: {dns-name: ec2-67-202-57-106.compute-1.amazonaws.com, instance-id: i-5ade1e3a}
5: {dns-name: ec2-107-20-73-192.compute-1.amazonaws.com, instance-id: i-fade1e9a}
6: {dns-name: ec2-50-19-21-19.compute-1.amazonaws.com, instance-id: i-a2de1ec2}
services:
demo-wiki:
formula: local:mediawiki-68
relations: {cache: wiki-cache, db: wiki-db, website: wiki-balancer}
units:
demo-wiki/0:
machine: 2
relations:
cache: {state: up}
db: {state: up}
website: {state: up}
state: started
demo-wiki/1:
machine: 6
relations:
cache: {state: up}
db: {state: up}
website: {state: up}
state: started
wiki-balancer:
formula: local:haproxy-14
relations: {reverseproxy: demo-wiki}
units:
wiki-balancer/0:
machine: 4
relations:
reverseproxy: {state: up}
state: started
wiki-cache:
formula: local:memcached-11
relations: {cache: demo-wiki}
units:
wiki-cache/0:
machine: 3
relations:
cache: {state: up}
state: started
wiki-cache/1:
machine: 5
relations:
cache: {state: up}
state: started
wiki-db:
formula: local:mysql-98
relations: {db: demo-wiki}
units:
wiki-db/0:
machine: 1
relations:
db: {state: up}
state: started
2011-09-07 16:25:34,574 INFO 'status' command finished successfully
Here is a graphical representation of the environment, helping visualize services and how they relate. This graphic was generated using.
juju status --format svg --output image.svg
Why We Call juju “DevOps Distilled”
Juju is all about distilling infrastructure best practices into “code,” and sharing that code with peers. That allows for many benefits such as having a repeatable, reliable, auditable, and automated infrastructure-as-code procedures, as well as being able to stand on the shoulders of giants, reusing the devops experience previously created. Juju excels at distilling devops experience into intelligent formulas that are instantly reusable and can be combined to create larger systems. Any customizations needed for those formulas to fit into your own environment is done cleanly via passing configuration parameters, not hacking on the formulas directly (although that is still an available option).
When you choose to build an infrastructure modeled as juju services, you are reusing the infrastructure experience of the Juju community. What’s even more interesting perhaps, is that juju does not force you to learn yet another domain specific language to model your infrastructure. Rather, juju provides you with command-line tools that are callable from any programming or scripting language. Very soon juju will also provide launchpad integration enabling the community to search for and leverage formulas written by other community members. You can think of it as PPAs for Infrastructure services!
I hope you have enjoyed this first article as an introduction to juju. In the next tutorial I will be diving into more details into the exact hows and whys of juju. While this article only scratches the surface of what’s possible, I hope it helped whet your appetite to learn more about juju. You can find more information on the juju site and you can join juju’s community at #juju on Freenode IRC.
