IIAR’s Tragic Quadrant spotlights overrated analysts

http://analystrelations.org/2015/06/08/the-iiar-tragic-quadrant/

Analyst firms should pay serious attention to the Institute of Industry Analyst Relations‘ (IIAR) new Tragic Quadrant. The TQ spotlights some upstart firms that have successfully won much greater mindshare with many AR managers than their firms have in the market. Analyst firms can boost their position in future editions of the TQ through being easier to work with, through increasing their profile on social media, and by appearing more open to discussing with and learning from vendors.

About the TQ

On June 8th, the IIAR published its first Tragic Quadrant. The TQ shows the perception, by around 60 vendors’ analyst relations professionals, of the sales impact, relevance and ease of interaction with 14 analyst firms. The results confirm two worrying findings from the 1,100 respondents to the Analyst Value Survey. First, vendors mistakenly give some companies credit for much more impact on sales than buyers tell us they have. Second, other companies with much greater influence on the market are absent.

My Take

The Tragic Quadrant is a modest and, in the IIAR’s words, “slightly cheeky” attempt to chart the most influential analyst firms. It fails to do that. As a comparison, the 2014 Analyst Value Survey (AVS) collected the responses of 450 users of analyst research in demand-side organisations, those which buy IT. When we asked those end-users that firms influence buying, they mention both companies that get into the TQ’s top 14 and many which do not. Everest and NelsonHall, for example, are highly rated by the demand-side in the AAS but are absent from the TQ. Also ignored are CXP Group, Aberdeen and ISG although end-users tell us those firms are influencing buyers notably. The IIAR’s survey, however, places analyst firms like Constellation, ESG, SMB and Ventana in their top 14, when they fail to get into the top 20 in the AAS question.

The principal reason the TQ fails to show accurately influence that AR people over-rate particular firms. While the survey design and execution could be improved, our warnings about IIAR surveys has been consistent over several years: don’t forget who took the survey. AR people will always tend to equal being easy to influence with being impactful, and they will tend to overlook analyst firms that do not actively develop business with AR managers.

It succeeds, however, in showing those analyst firms that have high mindshare with the AR managers that took the survey. The value for vendors in this chart is the size of the bubbles, which shows how easy the firms are to interact with: use this as a guide to understand which firms are harder to crack.

The actual audience for the TQ should be the analyst firms. Everest and NelsonHall have every right to snigger over the inclusion of SMB and ESG, but they should be asking instead why they don’t have a higher profile in the AR community, especially in North America.

The context

While the TQ is new, the ideas have been brewing for a while. The data some from a survey conducted irregularly over the last several years by the IIAR. The primary output is the Analyst of the Year award. The term Tragic Quadrant has been coined previously to refer to the Magic Quadrant ironically.

The article announcing the TQ credits Helen Chantry. Back in 2013, Chantry and I discussed the Analyst Value Survey and the KCG Mystical Box, which the TQ seems also to pay homage to. Helen wasn’t able to answer some questions I emailed her about the survey last year, but I’ve picked a few things up. The base of respondents is rather small: many IIAR members were invited to take part, but not all, and the survey was open for only a day or two. Some of the criteria are hard to grasp. The TQ segments firms into being ‘global’ and being ‘independent‘: only one firm is described as both although, as far as I can see, these words seem to have no identifiable meaning.

For the IIAR team that produced the survey, a key factor was the ease of interacting with analyst firms. They wrote that “analyst firms should monitor the ‘transactional tax’ they impose on AR people: if they raise the ‘interaction barrier’ too high while not providing sufficient coverage and not showing impact, their vendor information source might soon provide them only a partial view of the market (raising exhaustivity and fairness issues) or their vendor revenues might suffer too.

Analyst firms need to pay attention to that notion, even if they don’t accept it. Partly this is a side-swipe at firms like Gartner whose requests for ever-increasing volumes of information don’t lead to any improvement in research quality or favourability. But it is in some way a representation of what is creating the unusual vision of the market shown in the TQ. How is is possible to leave off Everest and ISG and yet include Ventura and SMB? The answer must surely be in the ease of interaction. Probably those two firms are reaching more frequently to AR people and are adding value to the relationships they are building with AR people.

The bottom line.
  • The TQ repeats a finding of the AVS: the vendor community greatly misunderstands which firms are impacting on sales.
  • AR professionals should not use this diagram to guide them on the impact or relevance of these firms. The Tragic Quadrant is misleading, incomplete and inaccurate.
  • AR professionals should pay attention to the estimates that this chart makes of the relative ease of interacting with firms. Pad your plan for time to get through to these companies, and recover from them. Most importantly, please appreciate that a firm that is hard to work with is often much more influential and has demanding communications because of robust methods.
  • If ESG can get into the IIAR’s top ten, then any firm with a couple of dozen analysts on Twitter can. Analyst firms should consider if they want to be a top performer on this chart’s future editions.
  • Analyst Firms should understand that this shows the opinions of 60 or so AR pros. If they are an important group of stakeholders, you need to be adding more value to AR people and lubricating the interactions with them.
The Tragic Quadrant is misleading, incomplete and inaccurate Click To Tweet
5.00 avg. rating (87% score) - 2 votes

7 Comments

  1. Ludovic Leforestier July 10, 2015 Reply

    Duncan, I find your conclusion a bit pathetic. It’s OK to try sell your own survey, but trying to rubbish the tragic quadrant is like Hormel suing the Monty Pythons for the use of the word spam: pointless and prone to backfire. Relax, take a breath and smile.

    • Author
      Duncan Chapple July 11, 2015 Reply

      If the TQ is only humour, then they would publish it on April Fools’ Day. Otherwise, people will take it as it is presented: a tongue-in-cheek visual homage to the MQ that aims to show the results of IIAR survey in which AR professionals rate the impact of the analysts firms. That is, in fact, exactly that it is.

      From that starting point, AR professionals are now sharing and discussing the TQ and asking how to use it. I think it can be used usefully in some ways, and not usefully in other means. It can be used by analyst firms to see what profile they have in the IIAR’s end of the AR community. It can also be used by AR pros to see which firms they should anticipate more time-consuming or low-value interactions. There are other, more mistaken, ways to use the survey.

      To ignore the TQ would be mistaken, since readers want to know what to do with it. Neither is the TQ in some way Pythonesque humour: it is a real survey, and unsurprisingly it is being consumed in such a way.

      Hormel, of course, did not sue Monty Python: it helped their brand. I don’t think the TQ undermines helps or harms the IIAR much, but I am sure it gets a lot of clicks. The primary task for this blog is clear. I aim to help professionals to focus on the influencers with the most impact on sales. Clearly, my role is to explain why the TQ is not a tool that does that, but still retains some value. If you want to engage with the arguments, or if you can explain — for example — how the survey calculated ‘Influence’ and ‘Relevance’ then you would add value to the conversation.

      • Ludovic Leforestier July 11, 2015 Reply

        Duncan, cheer up: there are some real, actionable findings. We just don’t pretend it to be more than it is. #meditate.

  2. brilliant!

  3. Ludovic Leforestier November 13, 2015 Reply

    Duncan, you should have read till the end. We suggest precisely the opposite.

    AR professionals may use this as a subjective analysis tool to look at their target audience engagement strategies. They should balance ‘ease to do business with’ against ‘relevance’ and ‘impact’. Or, in other words, they shouldn’t brief analysts just because they’re easy to deal with (or conversely they should look at analysts which are less of a pain depending on the type of impact the AR professional is looking for (see the AR SOSM model);

    Here’s the link again to refresh your memory > http://analystrelations.org/2015/06/08/the-iiar-tragic-quadrant/

    I guess you should re-write your blog post by saying you agree with us. Maybe not a catchy headline, but being nice will make you a better person ?

  4. Author
    Duncan Chapple February 24, 2016 Reply

    Sorry Ludovic, I only just noticed your post. I’m sorry that you can’t grasp that, in fact, we do seem to disagree. In my opinion AR professionals should not be trying to find the optimal balance between the scores shown in the TQ for “ease of doing business with”, “relevance” and “impact”.

Leave a Reply