Creating a project dashboard with RSS and rawdog

One of the challenges of managing a development project is keeping all parties abreast of key events or milestones. It’s not just the team members — marketing personnel need to plan events, business teams need to do forecasting, even the operations people need to know whether to lay in extra caffeinated beverages. Here’s how you can use the rawdog RSS aggregator to build a project dashboard that will keep your project’s people aware of its status. A project dashboard is an aggregation of the key ongoing information about a project. Anyone monitoring the dashboard will see the current status of the project.

Rawdog is a Python-based open source program written by Adam Sampson, based on Mark Pilgrim’s Feed Parser. It supports virtually all flavors of site syndication: RSS 1 and 2, RDF, and Atom. Unlike other aggregators, many of which are full-blown applications with user interfaces not unlike an email client, rawdog reduces feed aggregation to its essentials. A simple configuration file tells rawdog what feeds to read and how often to check them. Then, when rawdog is run (typically from a cron job) it assembles those feeds into a single well-formatted HTML page that can be read in any browser.

Installing rawdog is straightforward. First, be sure you have Python version 2.2 or greater. Download the latest release tarball, unpack it into a convenient location, and install it using the standard Python distutils facility. Typically if you have privilege to write into your site’s Python site-install directory you can simply run this command in the directory in which you’ve unpacked rawdog:

python setup.py install

This will install rawdog in the standard locations on your system. If you need to modify or customize the installation, examine the distutils documentation.

Now you can configure rawdog, by creating a .rawdog directory in your home directory and copying the config file that came in the rawdog tarball into it. For our purposes, the rawdog config file defaults are OK, but change the outputfile parameter in the config file to something sensible. You’re going to want to make rawdog’s output available via a Web server, so you may wish to point directly to your Web server’s document root. While you’re at it, copy the style.css file that came with the rawdog distribution to the same location.

Now it’s time to configure the feeds themselves. Start by deleting all of the “feed” lines at the end of the configuration file (or comment them out for later reference).

The heart and soul of most development projects is the check-in stream in the source control system. Depending upon the system you use, you may have to do a little scripting or hunt for a solution that will turn the ongoing commit logs into an RSS feed. CVS users can try cvs2rss or this script. Subversion users should look at SubveRSSed. Subversion also has an --xml argument for its log command that generates the Subversion log in an XML format that could be converted to an feed using XSLT. Norman Walsh has produced just such a stylesheet. If you’re using a different source control system, you could use the techniques described in the NewsForge article “Building a command-line generator for RSS feeds.”

Once you’ve generated your feed, you need to tell rawdog about it. If you’ve arranged for the feed to be placed in /home/release/feeds/checkin.xml, you’d add the following line to the rawdog config file:

feed 1h file:///home/release/feeds/svn.xml

This line tells rawdog to aggregate the specified feed but to check the feed only once every hour (1h). This constraint is probably important only when you’re using rawdog’s traditional role of aggregating newsfeeds from a variety of servers, to avoid abusing the remote server’s bandwidth. You should set all of your values to something less than the amount of time between cron runs of rawdog to insure that the aggregator always checks your feeds. The feed itself is just a standard URL. Typically a feed is an http: URL, but file: URLs work just fine.

Before you go any further, check that everything is working. Save the config file you just edited and execute the following command:

rawdog --update --write /home/release/feeds/feed.xml

This should cause rawdog to read the feed you specified (--update) and generate an HTML (--write) page at the location you specified with the outputfile config setting. Load the HTML file into a browser and review it. If it looks right, go ahead and start adding more feeds.

Once you’ve taken care of the development stream, you should turn to the the other clear indication of a project’s status: the bug stream. If you’re using the current development branch (2.19) of Bugzilla, you can build an RSS feed out of any Bugzilla query. If you’re using the current stable release, you can request queries be presented in CSV (comma-separated values) format, which you can easily convert to RSS.

You’ll want to consider exactly what you’d like to show on your dashboard. In the early stages of development a fairly restricted query makes sense: priority 1 and severity 1 logged bugs or bug fixes might be a place to start. After you’ve reached a code freeze, you might open the query up a bit to log most incoming and closed bugs. The idea is to let folks know when that really thorny memory leak finally gets fixed, but not to overwhelm them with the minutiae of dialog box misspellings. You might also consider putting together a simple script to generate a small feed of summary information — number of bugs found, fixed, and confirmed — over some useful period of time.

If your team does nightly builds or uses continuous integration, you’ll want to track their results. Using the techniques in the NewsForge article mentioned earlier, it’s fairly simple to convert the build logs into an RSS feed. Consider your audience and focus on key events rather than flood them with information when you’re constructing your feed.

Finally, consider including some of the non-automated information related to your project for your dashboard — things such as upcoming milestones, milestones achieved, marketing events, and vacation schedules. Probably the easiest way to inject this sort of information into your feed is to set up a weblog for just that purpose. Virtually all of the standard blogging tools provide some form of RSS feed, which can add to rawdog’s configuration file. If you’re looking for the simplest possible weblog tool, check out Bloxsom. It’s a simple Perl script that can run standalone or as a CGI script to produce a weblog (and associated feed) based simply on a directory hierarchy of text files. Once you’ve set up a weblog, you’ll find it easy to inject information into your dashboard as the need arises.

Of course it’s up to you to craft the most useful summary for your team. You might want to include information that is unique to your project or industry. In any case, setting up a project dashboard will give your project members a sense of the state of your project and will eliminate unpleasant surprises.