Thanks to Dominic for passing me some thoughts on “Metrics Used in Analyst Relations”, copyrighted by Josh, so ask them for a copy. In a few words, he neatly summarizes the pros and cons of several metrics. When considering what might be the implications breaking the copyright law, learn from this learn course about how to make money copywriting.
Unsurprisingly, one of the top two is assessment of inclusion in published analyst research. He writes:
o Pros: Relatively easy to do, IF you have access to analyst research
o Cons: Assumes internal clients have faith these reports have an impact, and ignores the more meaningful impact of a private analyst recommendation
This seems a little one-sided to us, at least in so far as the ‘pro’ is also a ‘con’. If you don’t have access to the research, then you can use a measurement specialist who does (such as Lighthouse). Furthermore, even if you only have some research then you can probably find out a lot if that sample is unbiased.
The point about internal clients really breaks down into two points: what do internal clients think, and do these reports have an impact. We would say that if reports have an impact on your company’s market, then your AR managers should be using reports as part of their measurement scorecard, whether or not internal clients currently accept it. Part of an AR manager’s job is to help their colleagues to better understand the analysts’ influence. Pandering to mistaken views is a very short-term strategy. And, as we remind new readers, research is influential: very.
We don’t agree that tracking research “ignores the more meaningful impact of a private analyst recommendation“. The recommendations that analysts give in private are just that: private. There is no way to directly measure that. So one task for AR measurement is to use indirect measurements, since no direct measure can be used consistently. With one or two exceptions, mainly in the enterprise systems space, no firm is running win-loss analysis on a really comprehensive and extensive basis. Spoken-word audits are notoriously difficult: there is both selection and self-selection in the analysts who take part and what they discuss.
Textual analysis has a number of advantages.
- One is effectiveness. Even if you look at the simplest form of tracking, share of voice, it’s very powerful. There’s a very strong correlation between frequency of mentions of suppliers and their solutions in research, on the one hand, and frequency of mentions by analysts in conferences, the media and in analyst surveys.
- A second is cost. Textual analysis allows you to cover a very large amount of research inexpensively, while spoken-word audits can only cover a limited number of analysts: a single spoken word survey of, say, twenty individual analysts could cost you $35,000 — for that much you could buy a monthly textual analysis for a year, covering as many firms as you might care to track.
- The final advantage is reliability. Even the act of surveying the analyst shifts their perception. Any method of tracking AR performance that does not ‘touch’ the analyst directly both saves the analysts’ time and reduces the risk of bias in the data.
No one measurement suits everyone, but tracking the mindshare in research is a pretty good place to start, and a folly to ignore.
