Taking Stock of 2013: Trends in Release Frequency, Code Quality, Uptime and Client Satisfaction


Since we’ve wrapped up 2013 and announced the last VitalSite CMS release of the year, it’s an opportune time to reflect on how far we’ve come these last twelve months. Let’s start with some metrics related to release frequency.

If you remember, last March we revealed a strategic shift in how we were developing software. Influenced by innovative practices in software development like continuous delivery, we developed our own deployment automation capabilities. This allows us to deliver more features, faster, to the clients who request them.

How did it play out? If we look back on the year, we can see that we’ve had a record number of releases: eleven since the start of 2013, and nine since we started automating our deployments.

A timeline showing the releases of VitalSite CMS in 2013

While this is a significant increase (almost 40%) over the previous year, it’s important to remember that these are not releases we’ve forced on users. VitalSite clients remain in control and decide if and when they want to accept a new release. Most want enhancements as soon as they are available, but for those times when they just aren’t ready to test on their dev and stage sites, we’ve built our deployment capabilities to accommodate each client’s unique preferences. If a client waits for a release or two before deciding to update their site, that’s OK. They get all updates whenever they are ready.

Of course, pushing updates is all for naught if it’s causing problems for clients. Here’s where our investments in deployment automation dovetails with our investments in quality.

We’re able to improve our release frequency without incurring additional risk because of our investments in Behavioral Driven Development. This means that we can catch bugs in our codebase sooner — sometimes just minutes after they are created, instead of weeks or months later, as is often the case with traditional software development.

We can measure the resulting code quality in a number of ways. For instance, the number of priority one (P1) bugs is down by nearly 30%, year over year. And when we do find a bug, the time it takes to deploy a fix is down to hours or days instead of weeks.

These are tremendous improvements, but I also like to look at where the rubber meets the road: client uptime statistics. Looking at these over the course of the year, we see some pretty remarkable numbers:

Graph showing hospital website uptime in 2013

That’s right, most clients are experiencing 99.99% uptime or higher, even though we’re releasing more often than we ever have before.

When considering this distribution, it’s important to remember that the highest and lowest uptime numbers are separated by just a few hundredths of a percent. Just to keep this in perspective, here’s the same graph zoomed out to full scale:

A full scale image of hospital website uptime in 2013

Of course, uptime isn’t just a measure of code quality. It’s also a measure of the investments we’ve made in building out redundant cloud hosting for hospital websites. In 2013, we’ve brought online additional capabilities that help make sure graphs like the above are the norm, and not the exception.

And by all accounts, our approach is working and our clients are noticing the change. Going into Q4, our overall client satisfaction scores were higher than they’ve been in two years. While total client satisfaction measures more than just VitalSite and hosting satisfaction, these two components are strong influencers of a number that ultimately reflects our investments across the board in making sure that Geonetric clients have the industry’s top performing websites.

Graph of Geonetric client satisfaction over multiple years

Any organization with a year like this deserves a moment to celebrate, but we’re already hard at work on some of the things we’ve planned for 2014. On top of that, we’re trying to push our uptime numbers, code quality, and client satisfaction scores to new highs.

Think we can do it?

Stay tuned, buckle up and get ready. We’re going for it.

Plusone Twitter Facebook Email Stumbleupon Pinterest Linkedin Digg Delicious Reddit
This entry was posted in Hosting, Value, VitalSite by Michael O'Neill. Bookmark the permalink.
Michael O'Neill

About Michael O'Neill

It’s not often you find a communications professional who is an expert writer, understands the power of social media and has the technical capabilities to embed on and contribute to software development teams. But that’s exactly the background Michael brought with him to Geonetric as the technical communications strategist. From writing eBooks to managing Geonetric’s digital presence, Michael uses his software know how and his marketing savvy to help tell Geonetric’s story through a variety of platforms. This former adjunct professor holds a bachelor’s degree in English literature from Worcester State College in Massachusetts and completed graduate level coursework at the University of Connecticut. In addition, Michael is also a Certified ScrumMaster, a contributing writer at iBusiness Magazine and a member of the Board of Directors at Gems of Hope. This new dad is known for his high coffee standards and has quite the following around the office when he brings in his favorite craft-roasted beans.

Leave a Reply

Your email address will not be published. Required fields are marked *

Time limit is exhausted. Please reload CAPTCHA.