This post discusses the differences between monitoring user experience and actually managing user experience to a certain level.  The difference between the two lies mainly in our ability to figure out where user experience problems lie.

A recent EMA study found that 52% of user experience problems (problems with things like front-ends to web sites) were reported by customers. In other words, the majority of  user experience problems were found by customers and not by IT.  Not good - this is probably a statistic we might want to keep hidden from the business!

How do we get to a situation where we're not using our customers as very expensive monitoring devices, particularly in these days when each piece of  customer business is important?

We use "user experience monitoring" or EUM. There are two flavors or EUM - synthetic and real-user.

 

1. Synthetic monitoring uses scripts to simulate customer activity. Let's imagine we want to ensure our online check-in web user interface performs well at all time. After all, if it doesn't, there is a good chance that customers will not choose our airline again. We record a script that retrieves a dummy booking, chooses a seat, and opts to print a boarding pass. We run this three step script every ten minutes from three different locations around the world. We can now proactively and automatically know when customers are having problems with our online check-in. And because we are monitoring from three different points, we can determine if it's the actual business service or something on the way to the business service that is causing any problems.
    
2. Real user monitoring is like a network probe. It listens to http network traffic (or tcp/ip traffic) and builds up a picture of the performance of a web user interaction. We would use it to monitor our online check-in process. Real user monitoring gives us a very rich source of diagnostic information should there be a problem.

 

Which should we use, synthetic or real-user monitoring? Both. Synthetic gives you the proactive notification - we have one customer who runs 30 different scripts at 7:30am every morning before its customers come on line so it can proactively fix problems before the customers notice. But real user monitoring gives a richer source of diagnostic information.

And that brings me to the title of this blog posting .. "from user experience monitoring to user experience management".

Everything I've described right now is all about telling you you have a problem before your irate customers ring in and tell you first. This is important, but it's not the whole story. Imagine I could tell you your car was having problems before you noticed.  If I told you of a problem, your first question would most certainly be, "and what's causing the problem?" 

And that's a very important question to ask. Forrester estimates that 80% of the time spent fixing a problem lies in finding where the problem is.  And the typical performance problem of the sort we're talking about ping-pongs between different expert groups as they try to determine whose fault it really is. Going back to your car, it would go to the electrics group first. They would say, "not us". Then to the transmission group. "Not us". And then to the fuel system group. "Not us". Then the wheels and tyres group. "OK - it's us. You need new tyres and a re-alignment".

And why does this allocation ping-pong occur? Because we don't give the appropriate tools to first-line support - to the Operations Bridge. What they need is a tool that allows them to figure what is causing a user experience performance problem.

We have such a tool (which probably doesn't surprise you - I probably wouldn't write about a problem we most certainly couldn't solve and had no plans to solve!) And this tool moves us from User Experience Monitoring to User Experience Management - to detecting a problem early and then fixing it quickly and efficiently. This tool does the following:

 

1. It takes you step by step thru a workflow looking at different potential sources of problem cause. Once I've talked about the areas the tool considers, you'll see why a guided workflow is a helpful thing to have.
    
2. It re-runs any synthetic scripts against the problem business service to see if the problem is still occurring
    
3. It looks for patterns in the behaviour of the problem business service's performance. Does the problem occur weekly? Is it only from one location?
    
4. It looks to see if a dependent service is causing the problem. These critical business services are complex beasts - they can depend on a lot of "stuff" underneath - and we always under-estimate just how much "stuff" there is under a business service. A colleague of mine was recently at a customer site. They were estimating how many IT artefacts were under their claims processing system. The consensus was "about 12 systems". They looked in the CMDB and found it was 42 systems, all the way down to network paths, DNS servers and the like.  Anyway - our tool looks at the performance of all these dependent services and uses some clever statistical analysis tools to determine the most likely suspect - the dependent service that most highly correlates with the business service performance problem.
    
5. It looks to see what changes have occurred under the problem business service.  It gets this information from the discovered CMDB - discovery notices when a service has changed and flags this. There is an IDC statistic that says that if a problem has occurred under a problem service, there is a probably of 80% that the change will have caused the problem.
    
6. It looks at incidents against this problem service in the service desk. This allows us to get the service desk's view of what is happening.

 

These guided analyses allow us to get a much better idea of what is causing a user experience performance problem, thus allowing us to "cut into the 80%" - cutting into the most time-consuming part of solving a complex performance problem and stopping that inefficient "allocation ping-pong".