Slashdot Log In
IDC Analyst Dan Kusnetzky Explains the Numbers
from the pulling-aside-the-veil dept.
First, Dan Kusnetzky says,
Now, on to the questions...I'm responsible for research on access-point device management software, operating environment software (embedded, client and server), clustering and high availability software, web-centric computing software, and some portions of storage software. You'll note that PCs, workstations, and servers were not listed anywhere. My team follows software, not hardware.Other groups within IDC conduct extensive research into hardware (PCs, handheld devices, appliance servers, servers, storage systems, etc.). My team and I are part of IDC's software research group.
1) I just have to ask...
by Ded Bob
Might he have the stats on the BSD's. People from the BSD community are curious. :)
Kusnetzky:
BSD revenue or paid copies are tracked as part of IDC's research on Unix. In 1999, just over 8,000 paid copies of BSD server software were shipped. This would give BSD about 0.9% share of worldwide Unix server operating environment shipments.
The 2000 figures have not yet made it through IDC's publication process and so I will not pre-release them here. IDC has a policy that subscribers see research first.
2) Funding
by ritlane
In studies of market share (or studies in general), we often hear quotes about who funded them. This seems to somehow imply that those who funded the study had some influence in how the data was gathered/interpreted.
My question is: Do those who fund a study influence how the study turns out (i.e. Microsoft studies show higher MS market share). Or is it that these corporations only decide to fund groups who they know will most likely return results in their favor.
Kusnetzky:
Over 95% of the research which IDC produces has not been sponsored. Therefore, our research studies are completely independent. In the few cases where IDC performs specific research for a client, IDC has strict guidelines and review processes in place to ensure the objectivity of our research. IDC never does market share studies on a sponsored basis.
3) More breakdown needed
by BillyGoatThree
When People magazine does an issue devoted to "what's hot" in fashion, do they interview Jane Doe from Des Moines, Iowa? No.
So why are OS numbers reported with equal rating? Not all users are equally suited to *choose* an OS, therefore not all users *choices* are equally interesting. I'd really like to see a breakdown of OS by user-type (levels of education, field of degree if applicable, occupation, etc). Keep in mind this applies just as much to business. A technology company presumably put more informed thought into their choice of server than an art supply house or whatever.
Kusnetzky:
IDC conducts research which will help its subscribers make better business decisions. This often means surveying decision-makers who make the purchasing decision rather than the people who actually use the products. This also tends to mean that academically interesting, but not commercially viable, research may not be done at all.
The complete findings of IDC's research are supplied only to subscribers. So, the snippets of data that Slashdot community members may have seen in magazines only includes information IDC chose to make public. Quite often, IDC's intent is to interest companies in purchasing the entire study. 8^) Another point is that it is quite possible that the findings which are mentioned in the press were taken out of context or mentioned incorrectly.
4)Self-fulfilling analysis?
by dvk
Do you think that there exists a possibility (or can even provide examples of) self-fulfilling analysis, such as "analysis says X is losing market share=people get skeptical about X=X loses market share although it may not have done so otherwise"?
If it is possible or already happened, do analysts in general (and you in particular) find it a worrisom possibility, and if so, are there any attempts/ideas to deal with the issue?
Kusnetzky:
Having studied a great deal of physics in college, I understand the Heisenberg uncertainty principle. That theory indicates that it is impossible to observe a system without interacting with it in some way.
While I suspect that studies have been done to detail how analysts (and journalists for that matter) influence the creation of products and their success in the market, I have not read them and, thus, can not comment on what they may or may not say.
5)Data origins
by m2
Do you base your data mostly on marketing analysis or do you actually go a pay a consultor to scan machines on the net? If there are scans involved, how do you pick the IP blocks to be scanned and what's the uncertainty associated with such a method (and how is this uncertainty guessed)? If there are no scans involved, why not? If this is "maket analysis", can you defined that for me? Which factors are involved? And a different question: who's the target market for this kind of study? How much does such a thing cost?
Kusnetzky:
IDC conducts two types of research -- demand-side and supply-side. Demand-side (end-user) research is designed to shine light on who is buying and using information technology products, how these products are being used, and what plans decision-makers have for future purchases. A scan of the network is not involved in this process. Contacting people and working with them is the basis of IDC's demand-side research. Network scans, while interesting, are not part of IDC's current research methodology.
Another important point is that a network scan would only show systems which were up and available at the time of the scan. Any systems protected by a firewall wouldn't appear. This means that the study would not be able to shine light on what organizations were using inside of their firewalls.
The cost of demand-side studies is driven by how many respondents are needed, how many questions must be asked, and how many countries are included. A small study conducted in North America which asks a few questions can be conducted for tens of thousands of dollars. The data gathering phase of such studies might be completed in a few weeks. A large, multi-country study which must be segmented by company size, market type, etc. might cost many hundreds of thousands of dollars. The data gathering phase of this type of project might six or more months to complete.
Supply-side research shines light on revenues, shipments, technology trends, etc. IDC gathers information from the financial reports of public companies and from public statements made by executives of privately held companies. This data is then segmented into 102 different software markets, 9 different operating environment platforms, and 6 different geographical areas. The software markets, by the way, are defined by a software taxonomy which attempts to define a list of markets which is both exhaustive and mutually exclusive.
The company data segmented using this model is offered to representatives of each of the companies. After the company representatives have had a chance to review the models, an IDC analyst then contacts the representative to verify the segmentation.
Once IDC has completed this process, the company models are examined by internal and external reviewers. Any questions which arise during the review process are brought back to the company representatives and resolved to IDC's satisfaction.
IDC goes beyond determining the revenues to modeling revenue or paid shipments in some markets. In this case, the revenues produced by the first process are segmented using demand-side data which shows the average shipment value. The modeled shipment data is, as with revenue data, examined by both internal and external reviewers. As before, any questions are resolved before data is published.
IDC, by the way, is watching over 1,200 companies worldwide. Revenue data from these companies is collected on a quarterly basis, segmented into IDC's taxonomic software markets and then stored in the IDC software research group's software forecaster database for analysis.
6) *what* constitutes a Linux server?
by Anonymous Coward
At what point does a computer become a server? Many Linux desktops have ftp, telnet & http ports open, so do they count as severs too?
Kusnetzky:
IDC's software group is using a very simple definition. If it serves the needs of a single person, it is called a client. If it serves the needs of many people it is a server, regardless of the system configuration. IDC's enterprise server research team has detailed definitions of different server configurations, but I won't repeat them here.
Linux, as observed in a recent study (December 2000, N=1583 North American and Western European respondents), appears to have a different usage pattern now than seen in earlier studies. This recent survey showed that just over a third of Linux shipments were used as client operating environments, roughly a third of Linux shipments were used as server operating environments, and just under 30% of Linux shipments were used on a workstation which supported the workload of a single person and provided some service to a workgroup. We're calling that new category "Serverstations." For historical purposes, Serverstations are currently allocated across client and server operating environment shipment totals. IDC will be publishing data on each of these categories in the near future.
7) Polling questions
by cavemanf16
I have often wondered how biased polls are based on the questions asked, the demographics of the people polled, etc. When results about polls are made public, is it also possible to obtain information about how the poll was conducted in a simple, by request method? Now if the answer to that question is, no, how much can we rely on polls, since we have no way of verifying if the questions asked and the people interviewed were heavily biased to favor one outcome over another? (Such as in the recent large discrepancies of the 8% vs. 24% use of Linux as a server results that we've seen on Slashdot recently).
Kusnetzky:
I believe that I've already covered this area in previous answers. It is clear that IDC's server operating environments data is being compared to another research firm's server adoption data. This comparison really isn't valid for several reasons including the fact that supply-side research is being compared to demand-side research and the fact that software research is being compared to hardware research.
IDC's extensive hardware research shows that Linux only holds a small share of the market when one examines shipments of servers and then segments it by the operating system which was installed at the factory. IDC's software research is showing that Linux server software is being installed on both older and new systems and that the configurations being used as servers include PCs, workstations, appliance servers, and more traditional server configurations.
8) What about the so-called "third world"?
by Kareem Abdul-Lamarr
Do these analyses factor in the so-called third world? Most of these analyses are US-centric or some times do include the continent across the pond but what about Africa and Asia? Do these analyses *really* take inputs from these continents?
Kusnetzky:
At this point, IDC has offices in 43 countries. IDC's software research team is represented in North America, South America, Europe, Africa, and Asia. IDC's operating environments research included revenue and shipment data from over 140 different Linux distributions, and we have collaborative input from analysts in geographic areas around the world. I think that this means that the answer is yes, IDC is examining operating environment revenue and shipment data worldwide.
9) Who Keeps track of the Predictions
by dmccarty
I have a question on the area of predictions in general. For example, fellow IDC analyst Jill House has been severaly negative on Palm over the years, with regard to the Win CE operating system and devices. A sample quote from her in Feb. 2000 read, "If I was Palm, I would be beside myself with panic."
The issue is, that over the last 3 - 4 years she's been predicting the demise of Palm and the rise of Win CE, a claim that has never materialized. Who verifies the reliability of these predictions and keeps the analysts accountable. With the frequent sound bytes and one-liners that they give to the press, these analysts have significant influence over public perception of the issues. But how is policing done when the analysts don't analyze very well?
Kusnetzky:
I really can't speak to this issue directly. Jill is not part of the software research group.
I know that a prominent part of the software research group's annual forecast and analysis reports for each software market is reviewing last year's forecast, how close it came to this year's findings. If the findings differ from last year's forecast, the analyst(s) attempt to determine what changed and why. I'm told by subscribers that this analysis is often is very helpful.
10) Re:Question(ADDNUM)
by HeUnique
I just remembered an Idea that a friend of mine suggested:
What if IDC could work with the Linux distributions (RedHat, Mandrake, SuSE, TurboLinux, Debian) to add a small program which will run after the first internet connection has been succsessfull..
When this program runs - it will ask the user to "register" his copy of the distribution. If it has been purchased from one of the distributors then the user can add his serial number. Some other questions like will this distribution be used as a server, a workstation, or combination of the 2, or a development workstation. The survey SHOULD be anonymous (unless the user wants to give some details about himself)
By that way - the distributors can give the numbers back to IDC - and IDC can publish a report which will tell that the number of Linux installations - and that number is X. X is combined of Y free download version and Z purchased copies of Linux.
What do you think, Dan? (And what do Slashdot readers think about it?)
Kusnetzky:
I agree this would be interesting information. I don't believe that this would be a good business for IDC to be in. Our analysis is that many users of Linux don't want their organizations or their competitors knowing what they're running. They'd rather get kudos for a job well done than criticism for how they got the job done. Not only that, but some vendors that have added features to track activations have also faced a great deal of criticism. I doubt that Linux users would appreciate such tracking operations.
Clearly biased (Score:2)
What about Antarctica? After all, we know what OS all those penguins are using.
Re:Heavy questions, Light answers (Score:2)
Personally I didn't think that any of your answers were "fluffy," but I am glad that someone did :). Your answers to random posts in the forum have been nearly as enlightening as the original interview.
Thanks!
Re:Self-fullfilling? (Score:2)
The iPaq is extremely popular, and I'm starting to see them everywhere. Not as often as the Palm, but still...
Why the emphasis on revenue? (Score:5)
Which is great for your supply-side surveys. But surely demand-side clients are far more interested in actual deployments rather than paid for shipments. When making a decision about an OS, I don't care how much revenue it's generated for a given company, but I am interested in how widely it's being used, and in what areas. For example, of our 25 or so servers running Linux, Solaris and OpenBSD, only 5 or 6 have paid-for copies of the OS, and I know that plenty of others are in the same situation. Going purely by paid for shipments, your figures are going to be wildly inaccurate.
Methodologies and Trust (Score:5)
Could you imagine a scientific journal that only contained abstracts? It would be laughed at. The methodology is absolutely essential to understanding the trustworthiness of the data. There needs to be full disclosure of these procedures before they release their snippets into the media.
Spreading unfounded information is as good as lying.
David E. Weekly [weekly.org]
Re:Linux on the Desktop (Score:3)
If 50% of the linux market is desktop then it has 10% of the total, not 0.3%. Your mistake came from assuming that 150:1 win32 client:server ratio even applied to linux(you said 1:1). In fact, this shows that windows servers have 0.6% of the server market, whereas Linux has 10% of the server market or 1500 times more servers.
Client side it's 10%(linux) vs 39.4%(windows). So windows has 4 times the market share. I do consider 10% cracking the market.
I believe your largest mistake was lumping both segments together and comparing them.
NB. True number are closer to
Win32 - 90%
Mac - 5%
Linux - 5%
for client
Re:The important bits (Score:4)
Er, I should have said "doing both an analysis based on actual usage surveys, and one on market sales, so they should be (assuming their methodology is sound) producing numbers that reflect how Linux is actually used, as well as how it's being sold, assuming you look at the right sets of their numbers."
-
The important bits (Score:5)
IDC IS doing an analysis based on actual usage surveys, not just market sales, so they should be (assuming their methodology is sound) producing numbers that reflect how Linux is actually used.
Dan knows his mouse from a hole in the ground.
Dan knows the difference between Linux, BSD, and Unix.
IDC seperates Linux from Unix and BSD because they think that division matters to their customers. They don't seperate BSD from Unix because they think that division doesn't matter to their customers.
IDC gives a shit whether it's predictions turn out correct, and tries to improve their accuracy by examining their results.
In all, it sounds like Dan's a geek, knows what Linux is, and is concerned first and foremost with producing accurate, truthful, scientifically-gathered information.
Now watch me get modded down for not following the party line.
-
whom to ask next: (Score:2)
It sounds like there's a lot of stuff Mr. Kuznetsky can't bring up because that information is reserved to subscribers of their research. Maybe /. should ask a subscriber to share what they get out of the data and how they use it? I imagine they're not allowed to just release the whole report, but some info on the general conclusions that they reached from it and how those differ from the results of the public portions of the research might be interesting.
Caution: contents may be quarrelsome and meticulous!
Re:The important bits (Score:2)
this "interview" could have been summarized in
just a few words: "pay me and I'll tell you".
I respect IDC's attitude, but I wonder why
bothered with questions.
Re:Methodologies and Trust (Score:2)
OTOH, it seems obvious that he was saying we do the surveys that will bring us money. We try to avoid lying. If one of our people is wrong frequently, we try not to notice. And if you want any more information, we can sell it to you. (The info that you can see for free is really just an ad campaign that we don't need to pay for.)
I suppose that others will have formed a different summary of his answers, but that's mine.
Caution: Now approaching the (technological) singularity.
Registering users. (Score:2)
it would not be so bad to point them to the
Linux Counter [li.org]
Re:Exactly, revenue or paid OS share, is a bad met (Score:2)
You have the technical point of view. They have the business point of view. From a technical point of view, their metrics are wrong. But from a business point of view, they are not that bad. If they want to estimate the market share in number of copies sold, not in number of copies used then they are right.
Again, you have a technical point of view. The best business decision may be very different from the best technical decision. Often, a manager will have to choose a solution that is not optimal from a technical point of view, but is good for the company because it minimizes the risks or because it is considered better by the shareholders or board of directors or whoever decides the future of the company and the future of this particular manager. Note that it does not mean that the solution is better, only that the shareholders perceive it as being better.
Re:Would by zealots, for zealots be better? (Score:2)
True true. I confess to a bit of hyperbole, but in our experience, the "tests" we do on control machines for linux/unix upgrades are usually successful, as opposed to what happens when you run Service Pack X on your perfectly stable Exchange machine.
The "tests" we do typically end up just being technicalities on the linux/unix boxen.. plus the fact that we can COMPLETELY restore the state of unix box from a tar file (or tape backup) with 100% reliability means the IT dept. sleeps much easier when they tweak a unix box.
I know 2K/NT backup solutions are SUPPOSED to do this, but getting back a PERFECTLY tweaked Exchange server from tape is not exactly painless.
Re:Would by zealots, for zealots be better? (Score:3)
There you go again, confusing "good" with "really unbelievably crappy".
Exchange is NOT a very good product. It happens to have a TON of *features* that PHBs really like, but nobody in their right mind would consider it stable, let alone good.
We have both sendmail AND exchange running. They both serve their purpose, and they are both stable *in their current state*.
The difference is that I *can* reconfigure the sendmail servers. I *can* perform upgrades. I *can* back it all off, and restore them, and expect them to come up just as they were. I *can* add different virus scanners. I *can* run other services on them.
The Exchange servers are black boxes. You don't run upgrades (of Exchange OR Win2k/NT) without testing them thouroughly on a control machine. You don't run other services on them. You don't expect a restore of the server to come up correctly.
Sure, they are stable, as long as you don't so much as look at them sideways.
Re:Self-fullfilling? (Score:2)
It was announced this week that iPAQ will overtake the Palm very shortly
iPaq may eclipse palm in revenue (after all, it costs a lot more), not unit shipments, and is not even close in terms of installed base. Of course it may be revenue that counts to you - it depends on what you're looking for.
IMO Developer mindshare shouldn't necessarily swing to iPaq as fast as revenue has, since for a developer, Palm+Handspring looks just like one big market, cost of entry is lower, and the tools are good, and cheap (or free).
Interesting.. but.. (Score:2)
Well, I'd better get used to this, as the election in Norway is coming in the autumn.
Re:Would by zealots, for zealots be better? (Score:2)
Notes seems pretty decent. The Domino mail server runs on Solaris (not sure about linux) and seems pretty straightforward. I've worked with an older version of it a while ago, and it was definitely a lot more predictable than Exchange. I'd switch to it in a heartbeat -- Just have to get management approval first.
If anyone can come up with any alternatives (free?) to Exchange/Notes/Groupwise, I'm sure I'm not the only one who'd be interested in hearing about them. Anything that does what your typical "groupware" does -- email, scheduling, and contact management I think are the biggies.
- Sean
Re:Linux on the Desktop (Score:2)
Let's say Windows has about twice the market share of Linux in the server market (40% to 20%). Let's even assume that the client:server ratio on Linux is 1:1. Both of these are on the generous side to Linux (I believe).
Now, Windows clients outnumber Windows servers by about 150:1. This means Windows clients outnumber Linux clients by 300:1.
By anyone's books, 0.3% of the market is a LONG way from cracking it. Even if my numbers are a factor of 5 off (1.5%), Linux is still a long way away from cracking the market.
Re:Linux on the Desktop (Score:2)
If 50% of the Linux market is desktop and 50% is server then that means there are the same number of Linux servers as desktops.
If there are twice as many Windows servers as Linux servers, and 150 times as many Windows desktops as Windows servers, then it follows (from simple algebra that somehow eluded you) that there are 300 times as many Windows desktops as Linux desktops.
Or, mathematically:
Assume Ls = Ld.
Assume Ls * 2 = Ws.
Assume Ws * 150 = Wd.
Then Ld * 2 * 150 = Wd.
or Ld * 300 = Wd.
I have no idea where you got your 10% from because I was not lumping both market segments together in any way. Perhaps you need to reread my post??
(I know what the true numbers are - I'm just going to show how stats can tell all sorts of wonderful lies).
Re:Statistics & Methodology (Score:2)
Damn I wish I could mod this up. Computer science is black and white - things work, or they don't. Social science is grey - it depends on how you approach it. To assess the validity of the results, all you can do is evaluate the methodology chosen while considering the constraints it was done under. It's easy to say "it would have been better to portscan every computer on the Internet three times a day over a period three months", but even ignoring the cost factors in such an approach, servers running Linux may not even be connected to the Internet. His methodology sounds ok to me. There may have been other better ways to do it, but it doesn't look like there were any significant flaws in his approach.
People on /. have asked sampling questions - our normal explanations on how and why we structured our sampling go for 5+ pages of highly technical documentation (that relies on theory most people here probably wouldn't understand or be interested in, such as the benefits of 5 point likert scales vs 7 point, scaling and weighting difficulties, and techniques for accounting for measurement error). You can see why a complete answer to these questions won't be presented here. Our methodology section in our reports will often go for 15+ pages.
As an economist / statistician and part-time programmer / sys-admin, I like the comments about "all things being held equal". The reason we do that is to define the problem in such a way that we can answer it. Interacting with the real-world is a lot messier than coding a program. Stats and research is not easy. It's hard work coming up with something that's defensible and will give you quality outputs.
Re:Interesting... (Score:2)
Re:Explanation (Score:2)
Huh?
I think you misspelt "But we've corrected the error, and you can buy the new survey for $1,999.95!"
Re:Would by zealots, for zealots be better? (Score:3)
Dude, I hate to tell you this, but where I work if you don't do upgrades without testing them thouroughly on a control machine, you'll probably find yourself out of a job. It doesn't matter if the platform is windows, linux, mainframe, unix, or the doorlock on the bathroom. If it isn't tested before going into production, you are going to find trouble.
I am as big a linux zealot as they come. All of my home machines are linux-only machines, and I quickly repartition any work hard disk so that I can have a copy of linux on it. I publically hate microsoft. BUT I don't think that it's good advocacy to start telling people that our software can be used without extensive testing. Sure it's a worse plan for M$ software, but it's a bad plan for any software.
I do understand your point that Exchange is crappy software, especially when compared to some of the opensource/free alternatives. But that doesn't justify ignoring good change control and change management policy.
$.02
--
Re:Would by zealots, for zealots be better? (Score:2)
At the consulting firm I used to work for, I had quite a bit of exchange exposure. IMNSHO, any mail server that crashes the OS when a poorly formatted message arrives is crap.
This was not a one time incident. It happened at multiple clients of ours and was repeatable (by sending the suspect message again).
Microsoft's response was "That's weird. We have no idea why that's happening. We'll take a look at it"
We never heard back from them or noticed the problem in their KB.
Re:Paid Shipments? (Score:2)
Re:Footnote on data gathering (Score:2)
Re:The important bits (Score:2)
Re:The old adage... (Score:2)
"Not everything that counts can be counted and not everything that can be counted counts." Albert Einstein
Re:Why the emphasis on revenue? (Score:2)
As others have noted, only a small percentage of BSD is packaged software.
BSD shows up in IDC's operating environment surveys but, this data is not used to create supply side reports. Only revenue data and the resulting paid shipments are mentioned.
IDC also publishes customer profiles, vendor profiles, surveys and other types of reports. IDC subscribers see these documents.
Linux and BSD use often hidden (Score:2)
When I ask how it is possible for so much Linux (or BSD) to be installed without executives being aware of it, I'm often told something along the lines that within the organization it is much better to get congratulated for doing a great job and solving a problem than to listen to criticism about what tools were chosen to solve the problem.
I believe that a portion of Linux (and BSD) usage is hidden. This means that traditional surveys of IT decision-makers won't find it. Interviews of computer users in business units might turn up clues to its presence.
Re:Reading between the lines (Score:2)
IDC conducts many types of research. A great deal of this research is published use of subscribers of one or more of IDC's 300+ "continous information service" subscribers. It is correct to point out that the subscription fees pay for the research. No subscriber, however, controls the topics researched, the methodology, or the findings beyond the choice to subscribe or not to subscribe.
A small portion of the findings may (or may not) be released in a press release, in a media interview, or in a presentation at an event.
The press releases, interviews, presentations, etc. all serve to publicize IDC's publications and services. In some cases, this encourages individuals or companies to purchase the document or become a subscriber.
Yes, you are right. The resulting revenue allows the company to stay in business and continue conducting research for people.
Re:Heavy questions, Light answers (Score:5)
IDC's software research group has an editorial review board. All forecasts are reviewed before they are published. Since IDC's name and reputation are on the line, every attempt is made to make sure that the assumptions are reasonable and that they have been applied in a supportable way. They still, by the way, may prove to be wrong in interesting and highly public ways.
IDC analysts review previous year's forecasts each year when the report is published and discuss whether or not they were accurate. If they were not, the analyst tries to point out how market trends differed from those projected in the previous report. This is an attempt to help the subscriber gain some value from what the analyst learned from thet time the first forecast was published.
The bottom line, however, is the review provided by subscribers. If they renew their subscription, they found value in the program. If they don't, then they didn't.
Statistics & Methodology (Score:3)
As a former statistician, let me tell you that performing this type of analysis is not cookie-cutter stuff.
The methodology for gathering information, and the feedback mechanism he describes, seem prudent to me. I see little fault with his analysis, given the constraints on his access to the target data set. I think he did a good job explaining it to a group of non-statisticians. Nice interview.
Investing is different than choosing technology (Score:2)
IDC is paid to provide research to institutional investors (for the most part). They don't care what the best technology is, only if it will make them a bunch of money if they buy the stock.
I would like to see someone perform research that was of more use to a techie like myself, but I think we get that job ourselves. Since we aren't willing to pay millions for that research, they don't provide it. 8-)
Re:Self-fullfilling? (Score:2)
Well, I posted the above question you replied to, so I guess I'll take a crack at your reply. First let me say that I think you're taking the story you're quoting (available here [cnet.com]) out of context. The report states that Compaq may soon start shipping more iPaqs than Palm does Palm devices, not taking into account that Palm already has a lot of devices on the shelves, the closer-to-reality average device cost numbers of Palm, return rates of CE devices, etc. And that doesn't mean that Palm's install base of 10,000,000+ devices is going to evaporate if more iPaqs are sold, a la IE vs. Netscape.
Secondly, I don't think that she was right all along, although if Palm doesn't solve some short-term problems they will have made her right through no insight of her own. I do think that her predictions had some influence on the people that had buying power and very little technical knowledge.
When she first started posting pro-CE reports, anyone worth their salt would have laughed out loud. In 98 or 99, CE was absolutely horrible and Palm was nothing short of golden. Palm devices actually synced better with Windows than Pocket PC devices (back then they were called Palm-size PC's) did at the time. So we--some of the old school Palm developers--read her forecasts and had a good chuckle.
So to bring this post back around on-topic, now the situation is one where the IDC analyst may end up becoming right, even though for the wrong reasons. She'll probably proclaim, "I was right all along!" but there is no way she could have known that Palm would make several key missteps, more CE device makers would not jump ship (as they were when she was predicting) and that the corporate world would blindly buy inferior devices on the whim of an uninformed purchaser. Which brings this question [slashdot.org] around full-cirle: there doesn't seem to be a mechanism for analysts to be shown wrong, or say "oops, I screwed up--sorry" or be held accountable for poor predictions. And until there are, it's not prudent to make important decisions based on their reports.
Heavy questions, Light answers (Score:4)
So I was pretty surprised to read his interview. After thinking we were going to get some "real" answers, it was disheartening to read his corporate, fluffy answers to the questions. My question (question #9 [slashdot.org]) was glossed over with a "that's not in my group" answer, so the question of who keep the analysts honest is still a very open one. Someobody other than yourself has to keep you honest. Accountants have audits, programmers have code reviews and people in construction have inspectors. Reviewing your own data from last year isn't good enough without either a) an oversight organization to make sure that your mistakes won't happen again or b) a watchdog agency to make sure that your mistakes won't happen again. Apparently, IDC has neither.
Explanation (Score:3)
Re:Self-fullfilling? (Score:2)
But I have yet to notice one person using an iPaq where I work (or anywhere), whereas out of 100 people in my office, I'd guess there are at least 15 Palm users.
Re:Would by zealots, for zealots be better? (Score:2)
I agree that polling zealots will get you less useful results - regardless of which product they're zealous for.
My analysis (Score:5)
IDC: By PHBs, For PHBs (Score:5)
IDC conducts research which will help its subscribers make better business decisions. This often means surveying decision-makers who make the purchasing decision rather than the people who actually use the products. This also tends to mean that academically interesting, but not commercially viable, research may not be done at all.
To summarize: they make recommendations to the purchasers based on comments received from the purchasers, without regard to what the users think!
Now I know why I have to use Exchange.
Linux on the Desktop (Score:3)
Self-fullfilling? (Score:3)
It was announced this week that iPAQ will overtake the Palm very shortly (assuming you could Palm, inc seperate from Handspring, etc). Maybe she was right all along? Or did her predictions have some influence? Personally, I think the problem is that palms were great in 1997 but they haven't really gotten significantly better since then. But it is interesting to think about.
Footnote on data gathering (Score:2)
-- .sig are belong to us!
All your
Exactly, revenue or paid OS share, is a bad metric (Score:2)
The problem is this - let's look at Rogers Cable, up in Canada, who are putting out a set top box with a version of Linux. It could be BSD too, for all it matters.
The main problem is they may have only five paid copies and hundreds of thousands of boxes running Linux or BSD that they did not pay for a copy of the OS.
It's free software, open source. So long as the industry uses the wrong metrics, asks the wrong questions, they'll continue to underestimate the number of Open Source OS boxen out there.
Let's look at your average Fortune 500 company using Linux. They probably buy 10 licensed copies and a support agreement, but since they standardize on only three PC versions (laptop, desktop, workstation), that's all they need. And they have tens of thousands of PCs running Linux or BSD. They may have only one-third of their PCs running Windows (any flavor) and two-thirds of their PCs running Linux or BSD, but IDC and other industry groups will report that Linux and BSD have less than one percent of the OS share, because they measure only paid OS copies.
You must measure something correctly to be able to make correct decisions. In modern accounting, pollution has no cost, only trying to stop pollution has a cost. So a CEO will try to increase pollution, not decrease it.
Thanks IDC! (Score:4)
1. The consulting company brought in by our PHSM uses IDC demand-side research (for PURCHASERS by PURCHASERS)?
2. We (the IT group) and our users are getting tired of our Novell file servers and *nix web servers never crashing?
3. Microsoft invested $600 million in our company a year ago when we really, really needed it?
And the bonus question: is it still called demand-side when it's being shoved down our throats?
(BTW, Deloitte Consulting believes in Microsoft software and IT outsourcing.)
Re:Thanks IDC! (Score:2)
Is that your company running business burying ships deep into the sea [gcn.com], called US NAVY?
 _
Re:Interesting.. but.. (Score:2)
All I wanted to know was if and how we can see what types of questions were asked, and then draw our own conclusions of the analysis. I didn't want to know about software vs. hardware server numbers. Just say: "No, you're not allowed to see the questions we ask, no one is." Then I'll be able to say, fine, then I should be expected to trust your judgements of what the data means. But maybe if he had just said: "Yes, you can see how we 'poll' people, but you have to pay for access to it like our clients do.", then I would likewise understand his answer. Is it so hard for someone to take a stand these days and just give a straight answer? My question was simply meant to find out why analyses of polling data was so often quoted as fact these days, when it's possible that the way the poll was conducted was completely bunk. Yes you can independently verify our analysis, or no, no one is allowed to verify our conclusions, was all I was looking for out of him.
Re:Interesting.. but.. (Score:2)
OK, I understand what you're saying now. I guess my original question was more directed at how do we (or you) verify the data with which you are presented (be it actual sales figures, or subjective questions asked of individuals and/or companies). As you mentioned in response to my post to this discussion, some clients rely upon you to merely collect data for them, then they analyze it as they see fit. So I guess what your saying is that any 'result' that someone publishes should definitely be taken with several grains of salt, and merely used as a guide post, but not the only way to measure a conclusion. This seems fair, and really does answer my question in that, if I pay your company for all the collected data, in essence, I would have access to see all the sales figures with which you used to do the analysis. Thank you!