On the Release Train

matt — Sun, 22 Apr 2007 20:13:35 GMT

The engineering team at Jive Software is growing fast (btw, we're hiring!). I thought it might be interesting to talk about some of the engineering process changes we're making to cope with that growth, especially since they directly affect product releases. First, a bit of history. We've always had a fairly agile development process -- lots of iterations, tools like unit tests and continuous integration, etc. But, we've consistently had a lot of pain around our current process:

* Not enough time for QA* . It's always scheduled, but gets squeezed due to lack of time. For example, the "official" QA time period for the Spark 2.5.0 release got crunched down to almost nothing. * Stress when we don't need it* . The stress is caused by having to cram a ton of work into a short period of time to make internally set product release dates. It's also caused by scheduling extra features into releases at the last minute and by having to do emergency patch releases due to bugs we missed. We like working really hard, but there should be a way to do that without excessive stress. * Unpredictable release dates* . The bigger a release is, the harder it is to make accurate work effort estimates. That means that dates slip and that it's hard to communicate to the outside world exactly when a release will ship.

So, there are some clear problems that we want to fix. The trick for us was to come up with a process that fixes those problems but that's still light-weight enough to avoid a huge administrative burden. We're now experimenting with a strict release train model for Spark and Smack. Assuming it goes well, the same model will be applied across all our products, including Clearspace. A visual summary of the model is below (click the image for a larger version):

[Clearspace|http://www.igniterealtime.org/blog/wp-content/uploads/2007/04/train.png|train_sm all.png]

The key points to this model:

We put out a new release every three weeks (although each release will have gone through a nine week process total).
Three weeks before each development cycle is reserved for product management; three weeks after each development cycle is reserved for QA. But notice that all three processes are all happening at the same time.

There are all sorts of subtleties to the system like how to deal with new features that take more than three weeks of engineering time, but so far everything is going quite well.

The start of the three week cycle is always on a Monday, and we have a big meeting to determine exactly which features will go into the release. Product management has already done the work of prioritizing the features so we just need to choose exactly what we're going to work on and come up with the plan about how to get it done over the next three weeks. On the second and third Mondays during each cycle we go through more detailed updates than in our SCRUM meetings to see what's on track and what's not.

At the end of the cycle, we:

Do the product release coming out of QA on Thursday.
On Friday, finish all development and create a branch ("spark_2_5_2_branch" for example).
Switch the builds in our continuous integration environment, TeamCity. The new branch becomes the "stable" release of the product and trunk becomes "development".
Release a beta version out of the new branch. In other words, the world generally always sees a stable official product release and a beta release that will become official in three weeks.

There's way more that I could talk about with this process (including some of the potential drawbacks), but I'll save that for future blog entries. Things we're jazzed about so far include solid release dates, having a better process in place to deal with new feature requests and bug reports, more stable quality in each release, and always having a place to check in new feature development (trunk). We're still early on in the new system, but we'll all know how exactly how well this works in the next two months. Of course, we'd love your feedback as time goes on about how well you think we're doing.

Releases: Second Wave

matt — Tue, 06 Feb 2007 23:27:37 GMT

Just one week later, I'm pleased to announce the second wave of releases:

Wildfire 3.2.0

The production release of Wildfire 3.2 is now available, including updated versions of the Enterprise plugin and the connection manager module. The 3.2 release is a major milestone for the project. We've massively increased scalability and started branching out beyond instant messaging by diving into VoIP support. View the changelog for details.

Spark 2.5.0 Beta 2

We've incorporated a number of bug fixes and minor improvements since the first beta. Grab the download from the new beta area of the site.

Smack 3.0.0 Beta 1

After a long wait, Smack 3.0.0 is now in beta. See the blog entry for full details.

Gateway Plugin 1.0.0 Beta 7

The gateway plugin, which provides connectivity to AOL, ICQ, Yahoo and MSN, gets a step closer to the final 1.0 release with the beta 7 version. View the changelog, or visit the download page. Also check out Daniel's post for details on the release.

SparkWeb 1.0.0 Beta 1

The first downloadable release of SparkWeb is now ready. Over the last week, we fixed support for Safari, did many UI tweaks and added minor new features like emoticon support.

Ignite Realtime Blog

On the Release Train

Releases: Second Wave