DIY case studies and end-user references

Regular readers may know about our weekly Spotlight service, in which we point out the new research that interests us the most. This week we came across some interesting Seybold research which I just had too comment on.

In this CRM case study, Mitchell Kramer explains his own experience of using a bank’s website. Let me assure those of you who think that this is just one more analyst venting steam over bad experiences, in the style of Andy Bitterer’s 2005 experience of KLM. In fact, Kramer has made a long series of these studies. You can see some similar research here. Other things being equal, I think this is a pretty rare approach towards research, and one that we need to see more often.

Too many case studies are sugar-sweet. Because some analysts rely on vendors to give them customer references and, because some analysts feel that the supply of references from the vendor will dry up if their case studies are not positive, it’s a difficult kind of research. Taking the references you’re given is highly unrepresentative, even though it’s always a surprise to see how frequently reference customers are shockingly frank to analysts about bad experiences. However, most analysts have to either get references through their own clients or do some primary research…. or find another alternative.

Of course, a number of analysts have taking an alternative route. In the same way that Web 2.0 businesses often sell content that their readers have created, some analysts simply surf the wave of web comments. Filtering, editing and concentrating the chat can produce credible-sounding, if eclectic, conclusions – and all before lunch. Other firms are outsourcing more and more of their research, which can work well or badly depending on how it’s done.

In the face of all that non-research, it’s not only reassuring to see analysts conducting primary research: it’s even a little shocking.

To see what other research we liked this week, join the free mailing list for the Spotlights. Use the form on the right.

Duncan Chapple

Duncan Chapple is the preeminent consultant on optimising international analyst relations and the value created by analyst firms. As SageCircle research director, Chapple directs programs that assess and increase the business value of relationships with industry analysts and sourcing advisors.