But they can be really deceptive.

The 2011 Healthcare Internet Conference is in full swing in Orlando and there’s more push than ever to demonstrate success with real numbers.  The trick to working with analytics is this:  the tools report the numbers but they don’t know what they mean.  Before you present your numbers, and I’m more concerned about presenting them to your leadership than to a conference, you need to understand them.

I attended a session yesterday about a physician directory overhaul and how they used metrics to demonstrate success.  The numbers that they shared were the following:

• Page views:  before: 320,000, after: 300,000
• Page views per visit:  before: 3.76 , after: 8.83
• Bounce rate: before: 48.12, after: 7.78

Understand that these were numbers that they shared with their executives.  I’ll admit that, looking at these numbers, I’m really confused.  Doing a little division, we find that they had around 85,000 visits per week before the redesign and only 34,000 after.  Sure, those new visitors are more engaged (lower bounce rate, higher pv/visit), but that’s a big shift!
After much discussion, the disconnects were teased out:

• They didn’t measure the same thing – both sets of numbers were for an overall URL, but the old site included a health library that was dropped in the overhaul without recasting the old numbers to remove the irrelevant page activity.
• They changed tools – the old site used WebTrends while the new site uses Google Analytics.  Past experience tells me that you really can’t compare numbers between these platforms.
• They really didn’t dig into the numbers to understand what was going on.  Why did the bounce rate go down?  It could be that large numbers of people were coming to the health library and then leaving, but it could also be that physician profiles aren’t indexed well in the search engines.  You just can’t know without further exploration.

The presenters did do one thing very well from a metrics perspective – they looked at conversions !  It’s possible that the activity metrics could lead them to diagnose real, serious issues with their tools or it could be that the numbers are simply meaningless. But the true story lies in the fact the presenters noted that the number of appointment requests being made from the directory increased. This is a meaningful goal that they can promote, especially if the numbers continue to grow.  In the end, that’s  what really matters!