Because there are no commercially-available databases of analysts’ users, the Analyst Value Survey relies on Panalyst, Kea’s in-house database, and invitations which are shared by our staff and clients. Several analyst firms also share links to the survey. The upside is that if companies share the link then the survey gets much more data about them and the other firms which their clients use. On the downside, it produces some concerns that the results could become imbalanced.
I’ve been running the survey since 2001 and take those concerns seriously. When we compare the responses, we see that respondents who come from invites distributed by analyst firms are not notably different in their thoughts about those firms than those who do respond in other ways. What it does do, however, is push up the share of voice of those firms and their competitors. For example respondents who use the firms that have shared links to the survey are more likely to use IDC than are those who are not. So those analysts are also boosting IDC’s profile, although there’s no notable difference in how those respondents feel about IDC.
After consulting with many people, including members of the Market Research Society, and have designed our survey accordingly. We focus on average scores, so that changing the share of voice doesn’t impact on the outcomes.
Because of this design, we believe that it’s on balance a good thing that supportive analyst firms help us gather more data.