You told us which analyst firms were most, and least, independent

Respondents to the Analyst Value survey in September and October 2013 commented on analyst independence

Respondents to the Analyst Value survey in September and October 2013 commented on analyst independence

Analysts firms vary massively in their independence. That’s one finding of our 2013 Analyst Value Survey, which shows the opinions of 352 users of analyst research.
Ovum, NelsonHall and Redmonk stand out as firms widely seen as highly independent, with strong scores for 451 Group, Pierre Audoin Consultants, Saugatuck and TBR.
Perhaps unsurprisingly, many of the firms that are most seen as independent are also seen as the least independent, which draws down firms like Gartner, Forrester and IDC in our chart. Thanks go to Ludovic Leforestier, who suggested this way of charting the data for us last year. It’s taken a while to get around to it; we’ve been spurred on by the forthcoming IIAR forum discussion on ethics.

No study is perfect, and it’s worth making some points about this chart. The more respondents scored a firm as being independent or not independent of its commercial relationships, the further its data point is over to the right. Don’t pay too much attention to the data over on the left: it’s there for entertainment value only, and we’ve blanked out a few firms where the number of responses was low.

The higher up on the chart a firm is, the more it’s seen, on balance, as producing research that’s independent of its commercial relationships. We’ve shown that number as the percentage of people scoring that firm as independent, divided by the total of the numbers that scored in either independent or not independent. {For example, Gartner was selected as most independent by 67 people but as least independent by 62. Gartner gets a score of 52% [that is 67/(67+62)] and is the furthest to the right because the total number of people selecting it in either of those questions [129, which is 67 plus 62]  was the highest out of the firms in the survey.}

One of the options was “Other…”, which allowed people to write in the names of smaller firms. It’s the data point without a label that’s between Everest and IDC. There were lots of them and, as you can see, as a group they are averagely independent. That point is unlabelled, since it doesn’t add much.

The black line rising across the middle of the chart is a trend line automatically added by Excel. It suggests that the more people there are commenting on a firm, that the more independent it is. Perhaps that means that the more eyes there are on a firm, the core careful it is about its independence.

Some of the firm had scores that were so close that the labels went on top of each other. So that the names are legible, I’d dragged a few down so the labels underneath can be seen. I’ve done that by picking up the label that’s longer (since the shorter one is hidden under it) and pulling it down a little. When you see labels that are very close, that’s what happening. So, for example, Saugatuck and TBR had essentially identical scores. So did GigaOM and Current Analysis. BARC and Canalys were also evenly matched… you get the picture. Where there’s one dot and more than one label, their dots are on top of each other. One day we’ll all have 3D monitors.

We’ll conduct this survey again in September: with more responses, we’ll have a more accurate picture of users’ perception of analyst independence.

P.S. One useful caveat about this chart came from an analyst who gave feedback on it: this chart is “all about perception”.  Needless to say, that is exactly what surveys do show. Analysts’ clients are the people who are best able to make an informed judgement about the relative independence of these leading analyst firms, and there’s no more effective alternative to a survey. We don’t want to be complacent about the research, so we remain open for suggestions on how to improve the study.

0.00 avg. rating (0% score) - 0 votes