Why CIOs Rely on IT Industry Analysts

How reliable is analyst research?

CIO and Darwin Magazines conducted a survey to gage CIOs’ use of IT analyst firms’ services to make IT purchase decisions.
More than half of the executives we spoke with said that IT market research is important to overall IT strategy. Research is used most commonly to educate on a technology or an IT issue.
Likewise, education was most frequently cited as the most valuable use of IT market research.
The vast majority (84%) of respondents feel the information they receive from IT industry analysts is reliable. Additionally, the majority of respondents feel that the information is objective.
However, they expressed concerns that the same companies that they report on are also clients, which leads CIO to believe that IT professionals may be unaware that most analyst firms have a mix of both vendor and users in their client base.

Methodology

Screen Shot 2015-05-29 at 10.43.27CIO Magazine and Darwin Magazine conducted a telephone survey of 200 CIOs on their reliance on IT analyst market research in making IT decisions. We defined IT analyst firms companies that are not allied with a financial services company and that exclusively track information technology related products and services and offer written reports and consultative services on a subscription basis. Respondents were randomly selected from CIO’s circulation file. Surveys were conducted between November 15 and November 28, 2000.
In terms of company size, 24% of respondents worked in companies with less than 1,000 employees, 30% had between 1,000 and 4,999 employees. 9% said their company employed between 5,000 and 9,999 employees while 38% worked in organizations with 10,000 employees or more. (3% did not answer or did not know.)
When asked to estimate the company’s gross sales or revenue for 2000, 18% of respondents estimated their company’s revenue at less than $50 million. 28% said their company’s revenue would be between $50 and $100 million. 27% estimated their annual revenue to be greater than $1 billion. 7% of the companies surveyed were non-profit. 23% either did not know or did not
answer.

Key Findings

Use and Value of IT Analyst Market Research

53% of our survey base believe that IT analyst market research is
somewhat important to their overall IT strategy. 33% indicated
that it was very important or extremely important while 14% felt it
was not at all important to their IT strategy. (1% said don’t know
or refused to answer.)
The most common use of IT market research among the IT
executives we surveyed was to educate themselves or their staff
on a technology or topic (67%). 55% of respondents indicated that
they used market research to make a technology purchase
decision and 42% used IT market research to compare their
company’s metrics to industry averages. 40% relied on IT market
research to make a vendor decision while 29% used it to make a
technology integrator decision. 27% said they relied on IT market
research to determine the strategic direction of their company’s
IT. On average, companies included in our survey spent $552,000
on IT market research in 2000.
When asked which areas market research firms provided the
most value, IT executives listed educating themselves or their
staff on a topic (41%) most frequently, followed by making a
technology purchase decision (22%), determining strategic
direction of IT (21%), comparing company metrics to the industry
average (16%) and making a vendor decision (14%).

Reliance for decisions/specific decisions/level of satisfaction

35%, or 69 out of 200, respondents said they had relied on IT
analyst market research for a major IT purchase decision. The
following chart is based on these 69 responses:

Reliance for decisions

Reliance for decisions

Accountability/objectivity/concern about conflicts of interest

When asked whether or not analyst firms have adequate accountability
to their organization if their recommendations are
incorrect, 56% of our respondents said no. 33% felt there was
adequate accountability. (11% said don’t know or did not answer.)
Recommendations to improve accountability most frequently
included tying compensation to accuracy and results and writing
contingencies into contracts. (See verbatim comment for question
4.)
Our survey asked IT executives to rate the reliability of the
information they receive from IT analysts, using a scale of 1 to 7.
(7indicated completely reliable and 1 indicated not at all reliable.)
56% of respondents indicated that the information was reliable or
fairly reliable (answered 4 or 5). 28% said the information was
very or completely reliable (answered 6 or 7) while 11% indicated
they were not reliable.
Our survey asked respondents to rate the objectivity of
information they receive from IT industry analysts on a scale of 1
to 7, where 1 meant not at all objective and 7 meant completely
objective. In response, 28% of respondents felt the information
they receive from IT analysts was very or completely objective
(answered 6 or 7) while 15% said the information was not
objective (answered 1,2 or 3). 53% said the information was
objective or fairly objective (answered 4 or 5).
When asked how concerned they were that the vendors covered
by analysts were also the analysts’ clients, 40% said they were
very to extremely concerned (answered 6 or 7) and an additional
40% were concerned or somewhat concerned (answered 4 or 5).
16% (answered 1, 2,or 3) said they weren’t very concerned or not
at all concerned.

Feb 27, 2001

0.00 avg. rating (0% score) - 0 votes