Who will police the police? AllTheAnalysts.com?

In an episode of The Simpsons when Homer creates a vigilante group to track down a cat burglar, Lisa, concerned over their abuse of power asks: “If you’re the police, who will police the police?” to which Homer replies: “I dunno, Coast Guard?”

Lisa’s question is that of Juvenal: qui custodiet ipsos custodes? And I am reminded of that by the news that AllTheAnalysts.com now allows users to score analyst research found by the Google search engine.

It’s not a new idea. If you look at Forrester.com you’ll see a similar system. However, on the Forrester site, which has a lot of users, the actual number of scores is pretty few. Not many people are volunteering scores, even though those users have access to the full text. On may papers, just one or two votes sets a very high or low score on reports (but perhaps that’s a starting-off sort of problem).

My bet is that the user-scoring of AllTheAnalysts will develop a systematic bias. The system identifies research, but users can click a button to see only free research. That free research will not only get more readers, it will also produce a tendency for readers to score free research more highly (since they can read the free stuff, but not the subscription content). In this system, free research will get a high score and paid-for research will get a low score.

But there’s a further bias: what sort of research is free? Often it’s vendor-sponsored research, either hosted on vendors websites or vendors have funded the research, so the reports are free on the analysts’ websites. The end result may be that AllTheAnalysts becomes a method for vendors to push their point of view into the market, with the gloss of web 2.0 community.

In the meanwhile, I am just waiting for free research firms to get higher scores, and then start to claim they have higher quality.

Duncan Chapple

Duncan Chapple is the preeminent consultant on optimising international analyst relations and the value created by analyst firms. As the head of CCgroup's analyst relations team, Chapple directs programs that increase the value of relationships with industry analysts and sourcing advisors.

There are 3 comments on this post
  1. April 23, 2008, 7:49 am

    Perhaps it should be called NotAllTheAnalysts.com? Although the list indexes around 100 sites, that’s a small fraction of the 750 or so analyst firms. Some pretty important firms, including NelsonHall, are missing.

  2. April 23, 2008, 12:35 pm

    Thanks Duncan. I like that. It made me laugh.

  3. July 26, 2011, 1:17 am

    Appreciating your questioning the scoring of analysts at AllTheAnalysts.com, I’m surprised nobody has measured and then placed a number of senior analysts at their appropriate relative positions within a drawn box; let’s call it a Magic Quadrant! What the heck,a diagram is worth a thousand words, no?