IDC Analyst Dan Kusnetzky Explains the Numbers 105
First, Dan Kusnetzky says,
Now, on to the questions...I'm responsible for research on access-point device management software, operating environment software (embedded, client and server), clustering and high availability software, web-centric computing software, and some portions of storage software. You'll note that PCs, workstations, and servers were not listed anywhere. My team follows software, not hardware.Other groups within IDC conduct extensive research into hardware (PCs, handheld devices, appliance servers, servers, storage systems, etc.). My team and I are part of IDC's software research group.
1) I just have to ask...
by Ded Bob
Might he have the stats on the BSD's. People from the BSD community are curious. :)
Kusnetzky:
BSD revenue or paid copies are tracked as part of IDC's research on Unix. In 1999, just over 8,000 paid copies of BSD server software were shipped. This would give BSD about 0.9% share of worldwide Unix server operating environment shipments.
The 2000 figures have not yet made it through IDC's publication process and so I will not pre-release them here. IDC has a policy that subscribers see research first.
2) Funding
by ritlane
In studies of market share (or studies in general), we often hear quotes about who funded them. This seems to somehow imply that those who funded the study had some influence in how the data was gathered/interpreted.
My question is: Do those who fund a study influence how the study turns out (i.e. Microsoft studies show higher MS market share). Or is it that these corporations only decide to fund groups who they know will most likely return results in their favor.
Kusnetzky:
Over 95% of the research which IDC produces has not been sponsored. Therefore, our research studies are completely independent. In the few cases where IDC performs specific research for a client, IDC has strict guidelines and review processes in place to ensure the objectivity of our research. IDC never does market share studies on a sponsored basis.
3) More breakdown needed
by BillyGoatThree
When People magazine does an issue devoted to "what's hot" in fashion, do they interview Jane Doe from Des Moines, Iowa? No.
So why are OS numbers reported with equal rating? Not all users are equally suited to *choose* an OS, therefore not all users *choices* are equally interesting. I'd really like to see a breakdown of OS by user-type (levels of education, field of degree if applicable, occupation, etc). Keep in mind this applies just as much to business. A technology company presumably put more informed thought into their choice of server than an art supply house or whatever.
Kusnetzky:
IDC conducts research which will help its subscribers make better business decisions. This often means surveying decision-makers who make the purchasing decision rather than the people who actually use the products. This also tends to mean that academically interesting, but not commercially viable, research may not be done at all.
The complete findings of IDC's research are supplied only to subscribers. So, the snippets of data that Slashdot community members may have seen in magazines only includes information IDC chose to make public. Quite often, IDC's intent is to interest companies in purchasing the entire study. 8^) Another point is that it is quite possible that the findings which are mentioned in the press were taken out of context or mentioned incorrectly.
4)Self-fulfilling analysis?
by dvk
Do you think that there exists a possibility (or can even provide examples of) self-fulfilling analysis, such as "analysis says X is losing market share=people get skeptical about X=X loses market share although it may not have done so otherwise"?
If it is possible or already happened, do analysts in general (and you in particular) find it a worrisom possibility, and if so, are there any attempts/ideas to deal with the issue?
Kusnetzky:
Having studied a great deal of physics in college, I understand the Heisenberg uncertainty principle. That theory indicates that it is impossible to observe a system without interacting with it in some way.
While I suspect that studies have been done to detail how analysts (and journalists for that matter) influence the creation of products and their success in the market, I have not read them and, thus, can not comment on what they may or may not say.
5)Data origins
by m2
Do you base your data mostly on marketing analysis or do you actually go a pay a consultor to scan machines on the net? If there are scans involved, how do you pick the IP blocks to be scanned and what's the uncertainty associated with such a method (and how is this uncertainty guessed)? If there are no scans involved, why not? If this is "maket analysis", can you defined that for me? Which factors are involved? And a different question: who's the target market for this kind of study? How much does such a thing cost?
Kusnetzky:
IDC conducts two types of research -- demand-side and supply-side. Demand-side (end-user) research is designed to shine light on who is buying and using information technology products, how these products are being used, and what plans decision-makers have for future purchases. A scan of the network is not involved in this process. Contacting people and working with them is the basis of IDC's demand-side research. Network scans, while interesting, are not part of IDC's current research methodology.
Another important point is that a network scan would only show systems which were up and available at the time of the scan. Any systems protected by a firewall wouldn't appear. This means that the study would not be able to shine light on what organizations were using inside of their firewalls.
The cost of demand-side studies is driven by how many respondents are needed, how many questions must be asked, and how many countries are included. A small study conducted in North America which asks a few questions can be conducted for tens of thousands of dollars. The data gathering phase of such studies might be completed in a few weeks. A large, multi-country study which must be segmented by company size, market type, etc. might cost many hundreds of thousands of dollars. The data gathering phase of this type of project might six or more months to complete.
Supply-side research shines light on revenues, shipments, technology trends, etc. IDC gathers information from the financial reports of public companies and from public statements made by executives of privately held companies. This data is then segmented into 102 different software markets, 9 different operating environment platforms, and 6 different geographical areas. The software markets, by the way, are defined by a software taxonomy which attempts to define a list of markets which is both exhaustive and mutually exclusive.
The company data segmented using this model is offered to representatives of each of the companies. After the company representatives have had a chance to review the models, an IDC analyst then contacts the representative to verify the segmentation.
Once IDC has completed this process, the company models are examined by internal and external reviewers. Any questions which arise during the review process are brought back to the company representatives and resolved to IDC's satisfaction.
IDC goes beyond determining the revenues to modeling revenue or paid shipments in some markets. In this case, the revenues produced by the first process are segmented using demand-side data which shows the average shipment value. The modeled shipment data is, as with revenue data, examined by both internal and external reviewers. As before, any questions are resolved before data is published.
IDC, by the way, is watching over 1,200 companies worldwide. Revenue data from these companies is collected on a quarterly basis, segmented into IDC's taxonomic software markets and then stored in the IDC software research group's software forecaster database for analysis.
6) *what* constitutes a Linux server?
by Anonymous Coward
At what point does a computer become a server? Many Linux desktops have ftp, telnet & http ports open, so do they count as severs too?
Kusnetzky:
IDC's software group is using a very simple definition. If it serves the needs of a single person, it is called a client. If it serves the needs of many people it is a server, regardless of the system configuration. IDC's enterprise server research team has detailed definitions of different server configurations, but I won't repeat them here.
Linux, as observed in a recent study (December 2000, N=1583 North American and Western European respondents), appears to have a different usage pattern now than seen in earlier studies. This recent survey showed that just over a third of Linux shipments were used as client operating environments, roughly a third of Linux shipments were used as server operating environments, and just under 30% of Linux shipments were used on a workstation which supported the workload of a single person and provided some service to a workgroup. We're calling that new category "Serverstations." For historical purposes, Serverstations are currently allocated across client and server operating environment shipment totals. IDC will be publishing data on each of these categories in the near future.
7) Polling questions
by cavemanf16
I have often wondered how biased polls are based on the questions asked, the demographics of the people polled, etc. When results about polls are made public, is it also possible to obtain information about how the poll was conducted in a simple, by request method? Now if the answer to that question is, no, how much can we rely on polls, since we have no way of verifying if the questions asked and the people interviewed were heavily biased to favor one outcome over another? (Such as in the recent large discrepancies of the 8% vs. 24% use of Linux as a server results that we've seen on Slashdot recently).
Kusnetzky:
I believe that I've already covered this area in previous answers. It is clear that IDC's server operating environments data is being compared to another research firm's server adoption data. This comparison really isn't valid for several reasons including the fact that supply-side research is being compared to demand-side research and the fact that software research is being compared to hardware research.
IDC's extensive hardware research shows that Linux only holds a small share of the market when one examines shipments of servers and then segments it by the operating system which was installed at the factory. IDC's software research is showing that Linux server software is being installed on both older and new systems and that the configurations being used as servers include PCs, workstations, appliance servers, and more traditional server configurations.
8) What about the so-called "third world"?
by Kareem Abdul-Lamarr
Do these analyses factor in the so-called third world? Most of these analyses are US-centric or some times do include the continent across the pond but what about Africa and Asia? Do these analyses *really* take inputs from these continents?
Kusnetzky:
At this point, IDC has offices in 43 countries. IDC's software research team is represented in North America, South America, Europe, Africa, and Asia. IDC's operating environments research included revenue and shipment data from over 140 different Linux distributions, and we have collaborative input from analysts in geographic areas around the world. I think that this means that the answer is yes, IDC is examining operating environment revenue and shipment data worldwide.
9) Who Keeps track of the Predictions
by dmccarty
I have a question on the area of predictions in general. For example, fellow IDC analyst Jill House has been severaly negative on Palm over the years, with regard to the Win CE operating system and devices. A sample quote from her in Feb. 2000 read, "If I was Palm, I would be beside myself with panic."
The issue is, that over the last 3 - 4 years she's been predicting the demise of Palm and the rise of Win CE, a claim that has never materialized. Who verifies the reliability of these predictions and keeps the analysts accountable. With the frequent sound bytes and one-liners that they give to the press, these analysts have significant influence over public perception of the issues. But how is policing done when the analysts don't analyze very well?
Kusnetzky:
I really can't speak to this issue directly. Jill is not part of the software research group.
I know that a prominent part of the software research group's annual forecast and analysis reports for each software market is reviewing last year's forecast, how close it came to this year's findings. If the findings differ from last year's forecast, the analyst(s) attempt to determine what changed and why. I'm told by subscribers that this analysis is often is very helpful.
10) Re:Question(ADDNUM)
by HeUnique
I just remembered an Idea that a friend of mine suggested:
What if IDC could work with the Linux distributions (RedHat, Mandrake, SuSE, TurboLinux, Debian) to add a small program which will run after the first internet connection has been succsessfull..
When this program runs - it will ask the user to "register" his copy of the distribution. If it has been purchased from one of the distributors then the user can add his serial number. Some other questions like will this distribution be used as a server, a workstation, or combination of the 2, or a development workstation. The survey SHOULD be anonymous (unless the user wants to give some details about himself)
By that way - the distributors can give the numbers back to IDC - and IDC can publish a report which will tell that the number of Linux installations - and that number is X. X is combined of Y free download version and Z purchased copies of Linux.
What do you think, Dan? (And what do Slashdot readers think about it?)
Kusnetzky:
I agree this would be interesting information. I don't believe that this would be a good business for IDC to be in. Our analysis is that many users of Linux don't want their organizations or their competitors knowing what they're running. They'd rather get kudos for a job well done than criticism for how they got the job done. Not only that, but some vendors that have added features to track activations have also faced a great deal of criticism. I doubt that Linux users would appreciate such tracking operations.
Well done -- up to a point (Score:1)
The real scoop.... (Score:1)
Stakeholders, Rqmts. Analysis & Sampling (Score:1)
The issue is not necessarily with analysts results, it's how the analyst's customer use the results :) And in defense of the analysts, they do NOT hold a gun to anyone's head or tell them what decisions to make- this is buyer beware folks, and read the small print. Analysts are typically very careful to fully disclose their methodologies to paying customers.
It was mentioned that the sampling population was mostly purchasing managers, or leadership / decision maker types.. here now is, potentially, your self fulfilling prophecy considering the egos of the leadership types - they feel better knowing that they are viewing data of other leadership types- important people you see, self confirmation, though not necessarily sensible process or adding value to the bottom line :) Fortunately, there really is little accountability anywhere, so no big deal there
We know that proper life cycle engineering consists of, among other things, a requirements analysis phase. In this phase, as many stakeholders as possible are engaged to make sure that a balanced system or service is the ultimate outcome and that, in particular, in house technical staff have appropriate input into the process. Multidisciplinary teams are the way to go.
It is unfortunate, and by no means the fault of IDC or other analysts, that many companies choose to forgoe sensible process and simply roll over to what analyst (fully disclosed) biased stakeholder sampling *implicitly* suggests as being an appropriate technical decision. The implicit underlying architectural issues, my god.. These same customers are the ones who sit about and spew best practices acronyms at the speed of light, and have NO CLUE what they are getting into.
The people making IT decisions are generally not well informed. I'll avoid characterizing in detail or disclosing why this is :)
Strong commitment to process though, that engages technical contribution upfront solves so many problems. Real life cycle technical strategy plays out in the standards groups w/ respect to interoperability, not around an Oak table of sheep led to the slaughter with a nice smelling bulk purchase agreement. What is a copy of windows actually worth? I wouldn't pay a dime for it and haven't run it in years. But many think it's valueable stuff?
I've been cleaning up after this cr*p for years. The life cycle costs are a killer- the rework, the planned obsolescense, the upgrade cycles, abandoning last years dev. tools for this years,.. sigh. I've also been trying to educate leadership (always a fresh batch of young "movers and shakers"), but they won't listen to people who don't dress like them or go to their health spa. In the end, who cares? There is plenty of work for technical people :) Just stay out of my cube- and don't encourage your children to study science or engineering.. it really doesn't pay well
We make the missiles that make the whole world scream ;)
Clearly biased (Score:2)
What about Antarctica? After all, we know what OS all those penguins are using.
Re:Heavy questions, Light answers (Score:2)
Personally I didn't think that any of your answers were "fluffy," but I am glad that someone did :). Your answers to random posts in the forum have been nearly as enlightening as the original interview.
Thanks!
Re:Self-fullfilling? (Score:2)
The iPaq is extremely popular, and I'm starting to see them everywhere. Not as often as the Palm, but still...
Why the emphasis on revenue? (Score:5)
Which is great for your supply-side surveys. But surely demand-side clients are far more interested in actual deployments rather than paid for shipments. When making a decision about an OS, I don't care how much revenue it's generated for a given company, but I am interested in how widely it's being used, and in what areas. For example, of our 25 or so servers running Linux, Solaris and OpenBSD, only 5 or 6 have paid-for copies of the OS, and I know that plenty of others are in the same situation. Going purely by paid for shipments, your figures are going to be wildly inaccurate.
OT: OpenBSD ISO images (Score:1)
Theo runs the OpenBSD project on a real shoestring and could use as many CD sales as he can get.
Cheers,
--Maynard
Methodologies and Trust (Score:5)
Could you imagine a scientific journal that only contained abstracts? It would be laughed at. The methodology is absolutely essential to understanding the trustworthiness of the data. There needs to be full disclosure of these procedures before they release their snippets into the media.
Spreading unfounded information is as good as lying.
David E. Weekly [weekly.org]
Re:Linux on the Desktop (Score:3)
If 50% of the linux market is desktop then it has 10% of the total, not 0.3%. Your mistake came from assuming that 150:1 win32 client:server ratio even applied to linux(you said 1:1). In fact, this shows that windows servers have 0.6% of the server market, whereas Linux has 10% of the server market or 1500 times more servers.
Client side it's 10%(linux) vs 39.4%(windows). So windows has 4 times the market share. I do consider 10% cracking the market.
I believe your largest mistake was lumping both segments together and comparing them.
NB. True number are closer to
Win32 - 90%
Mac - 5%
Linux - 5%
for client
Re:The important bits (Score:4)
Er, I should have said "doing both an analysis based on actual usage surveys, and one on market sales, so they should be (assuming their methodology is sound) producing numbers that reflect how Linux is actually used, as well as how it's being sold, assuming you look at the right sets of their numbers."
-
The important bits (Score:5)
IDC IS doing an analysis based on actual usage surveys, not just market sales, so they should be (assuming their methodology is sound) producing numbers that reflect how Linux is actually used.
Dan knows his mouse from a hole in the ground.
Dan knows the difference between Linux, BSD, and Unix.
IDC seperates Linux from Unix and BSD because they think that division matters to their customers. They don't seperate BSD from Unix because they think that division doesn't matter to their customers.
IDC gives a shit whether it's predictions turn out correct, and tries to improve their accuracy by examining their results.
In all, it sounds like Dan's a geek, knows what Linux is, and is concerned first and foremost with producing accurate, truthful, scientifically-gathered information.
Now watch me get modded down for not following the party line.
-
Meaningless number for BSD (Score:1)
Unfortunately, the number Dan quotes for BSD accounts only for commercial sales of BSD/OS, and totally disregards sales of FreeBSD, NetBSD and OpenBSD CDs (which are orders of magnitude greater than the figure Dan quotes), not to mention non-paid-for installations. In other words, it's a meaningless number - and not only is it meaningless, but it'll spawn masses of FUD about the BSDs that we could really do without. Way to go, Dan! I expect the penguinites will give you an award for this particular exercise in creative reporting.
DES
--
whom to ask next: (Score:2)
It sounds like there's a lot of stuff Mr. Kuznetsky can't bring up because that information is reserved to subscribers of their research. Maybe /. should ask a subscriber to share what they get out of the data and how they use it? I imagine they're not allowed to just release the whole report, but some info on the general conclusions that they reached from it and how those differ from the results of the public portions of the research might be interesting.
Caution: contents may be quarrelsome and meticulous!
Re:Would by zealots, for zealots be better? (Score:1)
Re:The important bits (Score:2)
this "interview" could have been summarized in
just a few words: "pay me and I'll tell you".
I respect IDC's attitude, but I wonder why
bothered with questions.
Re:Methodologies and Trust (Score:2)
OTOH, it seems obvious that he was saying we do the surveys that will bring us money. We try to avoid lying. If one of our people is wrong frequently, we try not to notice. And if you want any more information, we can sell it to you. (The info that you can see for free is really just an ad campaign that we don't need to pay for.)
I suppose that others will have formed a different summary of his answers, but that's mine.
Caution: Now approaching the (technological) singularity.
Registering users. (Score:2)
it would not be so bad to point them to the
Linux Counter [li.org]
Re:Would by zealots, for zealots be better? (Score:1)
My company has attempted to upgrade from Exchange 5.5 to Exchange 2000 twice now. Both times, the mailserver became so corrupted we had restore the whole server from backups. We've contacted MS, and they had NO CLUE why this keeps happening.
One of the Unix machines on our network was erroneously sending error messages to the Exchange server, which would then cause a NDR, which stick in the mail queue *permanently*. (Unlike sendmail or qmail, where the delivery will eventually fail, and an error will be returned to the sender or the server admin.) On top of that, the actual NDR message wasn't accurate, which wound up being a bug in Exchange. Not to mention the total lack of compliance with the SMTP RFCs.
Meanwhile, qmail has been on version 1.03 for quite some time, with no problems. And Sendmail, while you do need to upgrade it fairly often, the procedure is completely painless compared to what we have experienced with Exchange.
-Wintermute
Re:Exactly, revenue or paid OS share, is a bad met (Score:2)
You have the technical point of view. They have the business point of view. From a technical point of view, their metrics are wrong. But from a business point of view, they are not that bad. If they want to estimate the market share in number of copies sold, not in number of copies used then they are right.
Again, you have a technical point of view. The best business decision may be very different from the best technical decision. Often, a manager will have to choose a solution that is not optimal from a technical point of view, but is good for the company because it minimizes the risks or because it is considered better by the shareholders or board of directors or whoever decides the future of the company and the future of this particular manager. Note that it does not mean that the solution is better, only that the shareholders perceive it as being better.
Exchange is so good (Score:1)
that Microsoft uses it to server Hotmail.com. Oh wait a minute..
OpenBSD (Score:1)
The site openbsd.org is running Apache/1.3.19 (Unix) mod_ssl/2.8.2 OpenSSL/0.9.6 on NetBSD/OpenBSD [netcraft.com]
The site www.openbsd.org is running Apache/1.3.12 (Unix) PHP/3.0.15 mod_perl/1.21 on Solaris [netcraft.com]
Re:Would by zealots, for zealots be better? (Score:2)
True true. I confess to a bit of hyperbole, but in our experience, the "tests" we do on control machines for linux/unix upgrades are usually successful, as opposed to what happens when you run Service Pack X on your perfectly stable Exchange machine.
The "tests" we do typically end up just being technicalities on the linux/unix boxen.. plus the fact that we can COMPLETELY restore the state of unix box from a tar file (or tape backup) with 100% reliability means the IT dept. sleeps much easier when they tweak a unix box.
I know 2K/NT backup solutions are SUPPOSED to do this, but getting back a PERFECTLY tweaked Exchange server from tape is not exactly painless.
Re:Would by zealots, for zealots be better? (Score:3)
There you go again, confusing "good" with "really unbelievably crappy".
Exchange is NOT a very good product. It happens to have a TON of *features* that PHBs really like, but nobody in their right mind would consider it stable, let alone good.
We have both sendmail AND exchange running. They both serve their purpose, and they are both stable *in their current state*.
The difference is that I *can* reconfigure the sendmail servers. I *can* perform upgrades. I *can* back it all off, and restore them, and expect them to come up just as they were. I *can* add different virus scanners. I *can* run other services on them.
The Exchange servers are black boxes. You don't run upgrades (of Exchange OR Win2k/NT) without testing them thouroughly on a control machine. You don't run other services on them. You don't expect a restore of the server to come up correctly.
Sure, they are stable, as long as you don't so much as look at them sideways.
Re:Interesting.. (Score:1)
Thanks.
btw is there a place where the public can see older data or all of the publicly released data in one place?
Is it just me... (Score:1)
Of course it's not nearly as bad an interview as the one time
Re:Self-fullfilling? (Score:2)
It was announced this week that iPAQ will overtake the Palm very shortly
iPaq may eclipse palm in revenue (after all, it costs a lot more), not unit shipments, and is not even close in terms of installed base. Of course it may be revenue that counts to you - it depends on what you're looking for.
IMO Developer mindshare shouldn't necessarily swing to iPaq as fast as revenue has, since for a developer, Palm+Handspring looks just like one big market, cost of entry is lower, and the tools are good, and cheap (or free).
Interesting.. but.. (Score:2)
Well, I'd better get used to this, as the election in Norway is coming in the autumn.
Re:Would by zealots, for zealots be better? (Score:2)
Notes seems pretty decent. The Domino mail server runs on Solaris (not sure about linux) and seems pretty straightforward. I've worked with an older version of it a while ago, and it was definitely a lot more predictable than Exchange. I'd switch to it in a heartbeat -- Just have to get management approval first.
If anyone can come up with any alternatives (free?) to Exchange/Notes/Groupwise, I'm sure I'm not the only one who'd be interested in hearing about them. Anything that does what your typical "groupware" does -- email, scheduling, and contact management I think are the biggies.
- Sean
Re:Linux on the Desktop (Score:2)
Let's say Windows has about twice the market share of Linux in the server market (40% to 20%). Let's even assume that the client:server ratio on Linux is 1:1. Both of these are on the generous side to Linux (I believe).
Now, Windows clients outnumber Windows servers by about 150:1. This means Windows clients outnumber Linux clients by 300:1.
By anyone's books, 0.3% of the market is a LONG way from cracking it. Even if my numbers are a factor of 5 off (1.5%), Linux is still a long way away from cracking the market.
Re:Linux on the Desktop (Score:2)
If 50% of the Linux market is desktop and 50% is server then that means there are the same number of Linux servers as desktops.
If there are twice as many Windows servers as Linux servers, and 150 times as many Windows desktops as Windows servers, then it follows (from simple algebra that somehow eluded you) that there are 300 times as many Windows desktops as Linux desktops.
Or, mathematically:
Assume Ls = Ld.
Assume Ls * 2 = Ws.
Assume Ws * 150 = Wd.
Then Ld * 2 * 150 = Wd.
or Ld * 300 = Wd.
I have no idea where you got your 10% from because I was not lumping both market segments together in any way. Perhaps you need to reread my post??
(I know what the true numbers are - I'm just going to show how stats can tell all sorts of wonderful lies).
Re:Statistics & Methodology (Score:2)
Damn I wish I could mod this up. Computer science is black and white - things work, or they don't. Social science is grey - it depends on how you approach it. To assess the validity of the results, all you can do is evaluate the methodology chosen while considering the constraints it was done under. It's easy to say "it would have been better to portscan every computer on the Internet three times a day over a period three months", but even ignoring the cost factors in such an approach, servers running Linux may not even be connected to the Internet. His methodology sounds ok to me. There may have been other better ways to do it, but it doesn't look like there were any significant flaws in his approach.
People on /. have asked sampling questions - our normal explanations on how and why we structured our sampling go for 5+ pages of highly technical documentation (that relies on theory most people here probably wouldn't understand or be interested in, such as the benefits of 5 point likert scales vs 7 point, scaling and weighting difficulties, and techniques for accounting for measurement error). You can see why a complete answer to these questions won't be presented here. Our methodology section in our reports will often go for 15+ pages.
As an economist / statistician and part-time programmer / sys-admin, I like the comments about "all things being held equal". The reason we do that is to define the problem in such a way that we can answer it. Interacting with the real-world is a lot messier than coding a program. Stats and research is not easy. It's hard work coming up with something that's defensible and will give you quality outputs.
Re:Interesting... (Score:2)
Re:Explanation (Score:2)
Huh?
I think you misspelt "But we've corrected the error, and you can buy the new survey for $1,999.95!"
Re:OpenBSD (Score:1)
IIRC, Netcraft/NMap style identification relies on analising the response packes it gets from the server. This obviously may change due to modifications to the TCP/IP stack in different versions of an OS.
So it's reasonable to assume that OpenBSD may have had a packet signature very similar to Solaris in one version, and quite different in another.
Re:OpenBSD (Score:1)
Try ftp'ing to the machines and have a look at the motd banners.
I guess you can't choose what your mirrors run.. 8(
Re:Why the emphasis on revenue? (Score:1)
You should buy at least one copy, as the proceeds go to support the FreeBSD project. I own 4 copies of the CD sets, and I've only installed off of one of them. I generally do an ftp install, or a cvsup update, then buy the CD anyway. The CD's are not very expensive, and they support a good cause.
blah!
Re:Would by zealots, for zealots be better? (Score:3)
Dude, I hate to tell you this, but where I work if you don't do upgrades without testing them thouroughly on a control machine, you'll probably find yourself out of a job. It doesn't matter if the platform is windows, linux, mainframe, unix, or the doorlock on the bathroom. If it isn't tested before going into production, you are going to find trouble.
I am as big a linux zealot as they come. All of my home machines are linux-only machines, and I quickly repartition any work hard disk so that I can have a copy of linux on it. I publically hate microsoft. BUT I don't think that it's good advocacy to start telling people that our software can be used without extensive testing. Sure it's a worse plan for M$ software, but it's a bad plan for any software.
I do understand your point that Exchange is crappy software, especially when compared to some of the opensource/free alternatives. But that doesn't justify ignoring good change control and change management policy.
$.02
--
Re:Why the emphasis on revenue? (Score:1)
In 1999, just over 8,000 paid copies of BSD server software were shipped. This would give BSD about 0.9% share of worldwide Unix server operating environment shipments.
With how easy and convenient it is to install the free BSD's from the Internet, I am still surprised the numbers were so high. This is assuming the numbers are only of BSDi's version. Just imagine the number of the Free-Net-Open BSD's installed there are.
Re:Why the emphasis on revenue? (Score:1)
Re:Why the emphasis on revenue? (Score:1)
BTW, when you say 8000 copies of BSD sold, does that just mean BSDi's version or all of them (Free,Net,Open,BSDi)?
Re:Self-fullfilling? (Score:1)
Worse off for Palm, they absolutely BLEW the launch of the new m50x series. They announced the product, which dried up their sales as consumers awaited the new product. Then they delayed the actual launch of the product. By the time it finally came out, the color m505 got very mixed reviews.
While the other replies that the Palm OS user base is extremely healthy, the fact still remains that Palm's business has literally fallen off a cliff the past half year or so. With Palm & Handspring thoroughly owning the PDA market for a while, it's quite surprising that Compaq has gotten to this point so very quickly, even if it only reflects total revenue.
Re:Would by zealots, for zealots be better? (Score:2)
At the consulting firm I used to work for, I had quite a bit of exchange exposure. IMNSHO, any mail server that crashes the OS when a poorly formatted message arrives is crap.
This was not a one time incident. It happened at multiple clients of ours and was repeatable (by sending the suspect message again).
Microsoft's response was "That's weird. We have no idea why that's happening. We'll take a look at it"
We never heard back from them or noticed the problem in their KB.
Re:Would by zealots, for zealots be better? (Score:1)
I can personally attest to this fact. I got into ALOT of trouble when I forgot to test the doorlock on the bathroom before going into production....
"I'll take the red pill, no, blue. AAAHHHHHHHHHHHHHHHH........"
Possible alternative to Exchange (Score:1)
http://www.bynari.net/index.html
http://www.bynari.net/Products/products.html
The product in question is "Insite Server/Client". It offers compatibility with both Outlook and Exchange, and runs on *NIX based systems (I think). Anyway, I don't know if it is exactly what your after, but it looks close.
(Also, their tech support seems quite comprehensive from a casual examination.)
"I'll take the red pill, no, blue. AAAHHHHHHHHHHHHHHHH........"
Re:Interesting.. but.. (Score:1)
I did my best to answer the thoughtful questions posed by the Slashdot community without also making it impossible to sell the results of our research.
Re:Why the emphasis on revenue? (Score:1)
Re:Interesting.. but.. (Score:1)
On more than one ocassion I've been told by a subscriber that they think that my assumptions and analysis are useless to them. They purchase the reseach just for the raw data and then do their own analysis.
One even told me that they consider me a reverse bell weather. If I say suggest one thing in a report, they do the opposite!
My goal is to help subscribers by providing solid information for their planning processes not to be popular or to see my opinion win out.
Re:What??? (Score:1)
IDC software anaylsts stationed in every major geographic area obtain software revenue data from every major companies in the software market. Companies having over $10 million dollars/year and up are included This is done on a quarterly basis and includes data from both public and private companies.
IDC segments that revenue into many (102)different software markets using a well defined taxonomy. The segmentation is then reviewed by company representaives to make sure the segmentation is as accurate as possible.
Operating system shipments are modeled using operating system snd subsystem (one of IDC's taxonomic categories) revenue data and survey data on the average shipment value. This model is also reviewed by company representatives.
Reports are then written which review how the revenues and shipments in each operating environment segment stack up.
So, the revenues generated by Windows, UNIX, Linux, NetWare, etc. are compared in one table. Shipments are compared in a different table. The data is segmented several ways including adoption in each of 6 geographic regions, client vs. server useage, etc.
Re:Why the emphasis on revenue? (Score:1)
As others have noted, the free downloads and replicas made of either free downloads or paid copies are not included in the total.
IDC publishes installed base information to its subscribers but does not provide that type of information to the media.
Re:Interesting.. but.. (Score:1)
Revenue data and the shipment data which is modeled based upon revenue data comes from published financial reports for public companies and public statements and private interviews of executives of privately held companies. The methodology used to go from revenue data to shipment data has been discussed earlier in this string.
Re:What??? (Score:1)
The only way to track free shipments and replicas made of either paid or free shipments is to do a survey.
Since there is no way to independently verify the numbers provided by the mirror sites, just calling them up and reporting those numbers isn't enough.
IDC has done enough operating environments surveys in the last 5 years that we finally have enough data to project free shipments and replicas made of either free or paid shipments. This data is reported in the report, "Linux Software Market Forecast and Analysis, 2001-2005".
Boring title to be sure.
Re:Well done -- up to a point (Score:1)
I think that people should be skeptical about market research and find out more. I agree that people would be best served if they understand what's being counted, how it's being counted, what's included and what's being excluded.
That's why I always begin a presentation with a joke about "lies, damn lies and statistics" or "figures lie and liars figure."
In my 7 years with IDC, the subscribers I enjoyed the most were those who asked those types of questions. We then explored together what was behind the methodology, the data colection, the analysis, and how this information could be of use to the company they represented.
I remember fondly my discussions with the CEO of one company. We'd meet just about every year at that CEO's user group meeting. We'd debate for about 45 minutes, smiling the whole time. In the end, we'd shake hands and promise to do it again next year.
I think that I learned more from that CEO that he learned from me.
Re:Heavy questions, Light answers (Score:1)
It's become quite clear to me that being a contortionist is part of the job of an industry analyst. Avoiding the mid-flight, display crunch while typing is another.
I did my best to reply where I thought the reply would add some value to the discussion while taking part in a highly distributed strategic planning meeting. Meeting participants were in many countries.
Re:Investing is different than choosing technology (Score:1)
The broad needs they present is the reason that IDC so many analysts looking at so many things.
Re:Paid Shipments? (Score:2)
Re:Footnote on data gathering (Score:2)
Re:The important bits (Score:2)
Re:The old adage... (Score:2)
"Not everything that counts can be counted and not everything that can be counted counts." Albert Einstein
Re:Why the emphasis on revenue? (Score:2)
As others have noted, only a small percentage of BSD is packaged software.
BSD shows up in IDC's operating environment surveys but, this data is not used to create supply side reports. Only revenue data and the resulting paid shipments are mentioned.
IDC also publishes customer profiles, vendor profiles, surveys and other types of reports. IDC subscribers see these documents.
Linux and BSD use often hidden (Score:2)
When I ask how it is possible for so much Linux (or BSD) to be installed without executives being aware of it, I'm often told something along the lines that within the organization it is much better to get congratulated for doing a great job and solving a problem than to listen to criticism about what tools were chosen to solve the problem.
I believe that a portion of Linux (and BSD) usage is hidden. This means that traditional surveys of IT decision-makers won't find it. Interviews of computer users in business units might turn up clues to its presence.
Re:Reading between the lines (Score:2)
IDC conducts many types of research. A great deal of this research is published use of subscribers of one or more of IDC's 300+ "continous information service" subscribers. It is correct to point out that the subscription fees pay for the research. No subscriber, however, controls the topics researched, the methodology, or the findings beyond the choice to subscribe or not to subscribe.
A small portion of the findings may (or may not) be released in a press release, in a media interview, or in a presentation at an event.
The press releases, interviews, presentations, etc. all serve to publicize IDC's publications and services. In some cases, this encourages individuals or companies to purchase the document or become a subscriber.
Yes, you are right. The resulting revenue allows the company to stay in business and continue conducting research for people.
Re:Heavy questions, Light answers (Score:5)
IDC's software research group has an editorial review board. All forecasts are reviewed before they are published. Since IDC's name and reputation are on the line, every attempt is made to make sure that the assumptions are reasonable and that they have been applied in a supportable way. They still, by the way, may prove to be wrong in interesting and highly public ways.
IDC analysts review previous year's forecasts each year when the report is published and discuss whether or not they were accurate. If they were not, the analyst tries to point out how market trends differed from those projected in the previous report. This is an attempt to help the subscriber gain some value from what the analyst learned from thet time the first forecast was published.
The bottom line, however, is the review provided by subscribers. If they renew their subscription, they found value in the program. If they don't, then they didn't.
Re:Would by zealots, for zealots be better? (Score:1)
Oh and before you flame me, I wouldn't know about correctly configuring an Exchange Server to not be succeptable to these attacks, as I don't have one.
But I do know I didn't have to rebuild my Lotus Notes servers after any of these attacks, and I still can let my users read attachments (I know there's software to scan attachments too, you don't really have to turn it off).
Oh and I don't even like Lotus Notes that much.
Two questions answered here (Score:1)
"Do you think that there exists a possibility...of self-fulfilling analysis...?"
and
"Do those who fund a study influence how the study turns out?"
but not this one
"But how is policing done when the analysts don't analyze very well?" "
Re:Would by zealots, for zealots be better? (Score:1)
Lots of people are able to crash exchange, but because you haven't yet, you'd refute all their claims?
Statistics & Methodology (Score:3)
As a former statistician, let me tell you that performing this type of analysis is not cookie-cutter stuff.
The methodology for gathering information, and the feedback mechanism he describes, seem prudent to me. I see little fault with his analysis, given the constraints on his access to the target data set. I think he did a good job explaining it to a group of non-statisticians. Nice interview.
Re:Interesting.. but.. (Score:1)
Re:What??? (Score:1)
On another note, I would like the option of sending IDC a note that says "another XXXX install". You really won't need more info then that for a raw count.
One last thing, A lot of posts may be 'hostile' towards IDC because we know its easier to get a count on sales, and volume of use on a produt when it only has one overall control point.
However GNU/Open source/GPL/etc... products do not get counted nearly as acuratly because of the distrobution method. I have 4 machine running linux that was copied off one disk that had been passed around our office of 500 people. How do you count that?
Re:Why the emphasis on revenue? (Score:1)
Not necessarily. If you're the CEO of a company that is thinking about selling widgets as a profit making venture, then you're very interested in paid shipments. If you're interested in interoperability or total user base, however, then you'll care about actual deployments. Both types of numbers are useful.
The key thing for the rest of us is to pay close attention so that we don't use the information out of context.
Reading between the lines (Score:1)
Would this be an accurate paraphrase?
Though our studies aren't usually funded by a particular company beforehand, when the result presents a company in a favorable way, then the company will tend to buy the study to use in a Marketing campaign. (And our studies cost more than some of your cars)
Investing is different than choosing technology (Score:2)
IDC is paid to provide research to institutional investors (for the most part). They don't care what the best technology is, only if it will make them a bunch of money if they buy the stock.
I would like to see someone perform research that was of more use to a techie like myself, but I think we get that job ourselves. Since we aren't willing to pay millions for that research, they don't provide it. 8-)
Re:Methodologies and Trust (Score:1)
I disagree. My question (question #9 [slashdot.org]) was also about analyst methodologies in general, from the viewpoint of who holds them accountable when their predictions are proved to be false. Unfortunately, Mr. Kusnetzky glossed over that one...
Re:Heavy questions, Light answers (Score:1)
Re:Self-fullfilling? (Score:2)
Well, I posted the above question you replied to, so I guess I'll take a crack at your reply. First let me say that I think you're taking the story you're quoting (available here [cnet.com]) out of context. The report states that Compaq may soon start shipping more iPaqs than Palm does Palm devices, not taking into account that Palm already has a lot of devices on the shelves, the closer-to-reality average device cost numbers of Palm, return rates of CE devices, etc. And that doesn't mean that Palm's install base of 10,000,000+ devices is going to evaporate if more iPaqs are sold, a la IE vs. Netscape.
Secondly, I don't think that she was right all along, although if Palm doesn't solve some short-term problems they will have made her right through no insight of her own. I do think that her predictions had some influence on the people that had buying power and very little technical knowledge.
When she first started posting pro-CE reports, anyone worth their salt would have laughed out loud. In 98 or 99, CE was absolutely horrible and Palm was nothing short of golden. Palm devices actually synced better with Windows than Pocket PC devices (back then they were called Palm-size PC's) did at the time. So we--some of the old school Palm developers--read her forecasts and had a good chuckle.
So to bring this post back around on-topic, now the situation is one where the IDC analyst may end up becoming right, even though for the wrong reasons. She'll probably proclaim, "I was right all along!" but there is no way she could have known that Palm would make several key missteps, more CE device makers would not jump ship (as they were when she was predicting) and that the corporate world would blindly buy inferior devices on the whim of an uninformed purchaser. Which brings this question [slashdot.org] around full-cirle: there doesn't seem to be a mechanism for analysts to be shown wrong, or say "oops, I screwed up--sorry" or be held accountable for poor predictions. And until there are, it's not prudent to make important decisions based on their reports.
Heavy questions, Light answers (Score:4)
So I was pretty surprised to read his interview. After thinking we were going to get some "real" answers, it was disheartening to read his corporate, fluffy answers to the questions. My question (question #9 [slashdot.org]) was glossed over with a "that's not in my group" answer, so the question of who keep the analysts honest is still a very open one. Someobody other than yourself has to keep you honest. Accountants have audits, programmers have code reviews and people in construction have inspectors. Reviewing your own data from last year isn't good enough without either a) an oversight organization to make sure that your mistakes won't happen again or b) a watchdog agency to make sure that your mistakes won't happen again. Apparently, IDC has neither.
Explanation (Score:3)
Independence? (Score:1)
Over 95% of the research which IDC produces has not been sponsored. Therefore, our research studies are completely independent.
Is it just me, or does this not make any sense at all? 95% == completely? 0.95 == 1?
You see his userid? (Score:1)
Which are both prerequisites for reading Slashdot. Which Dan apparently does. He appears to have a userid in the 78000s. Interesting. Maybe a better question should've been what he thinks about slashdot's stories.
Re:Self-fullfilling? (Score:2)
But I have yet to notice one person using an iPaq where I work (or anywhere), whereas out of 100 people in my office, I'd guess there are at least 15 Palm users.
Re:Would by zealots, for zealots be better? (Score:2)
I agree that polling zealots will get you less useful results - regardless of which product they're zealous for.
My analysis (Score:5)
IDC: By PHBs, For PHBs (Score:5)
IDC conducts research which will help its subscribers make better business decisions. This often means surveying decision-makers who make the purchasing decision rather than the people who actually use the products. This also tends to mean that academically interesting, but not commercially viable, research may not be done at all.
To summarize: they make recommendations to the purchasers based on comments received from the purchasers, without regard to what the users think!
Now I know why I have to use Exchange.
Linux on the Desktop (Score:3)
Re:Would by zealots, for zealots be better? (Score:1)
I have found them extremely reliable myself, so that doesn't prove anything. If you are critisizing Exchange it would also be nice if you could suggest an alternative with similar functionality.
Re:Self-fullfilling? (Score:1)
Self-fullfilling? (Score:3)
It was announced this week that iPAQ will overtake the Palm very shortly (assuming you could Palm, inc seperate from Handspring, etc). Maybe she was right all along? Or did her predictions have some influence? Personally, I think the problem is that palms were great in 1997 but they haven't really gotten significantly better since then. But it is interesting to think about.
Footnote on data gathering (Score:2)
-- .sig are belong to us!
All your
Exactly, revenue or paid OS share, is a bad metric (Score:2)
The problem is this - let's look at Rogers Cable, up in Canada, who are putting out a set top box with a version of Linux. It could be BSD too, for all it matters.
The main problem is they may have only five paid copies and hundreds of thousands of boxes running Linux or BSD that they did not pay for a copy of the OS.
It's free software, open source. So long as the industry uses the wrong metrics, asks the wrong questions, they'll continue to underestimate the number of Open Source OS boxen out there.
Let's look at your average Fortune 500 company using Linux. They probably buy 10 licensed copies and a support agreement, but since they standardize on only three PC versions (laptop, desktop, workstation), that's all they need. And they have tens of thousands of PCs running Linux or BSD. They may have only one-third of their PCs running Windows (any flavor) and two-thirds of their PCs running Linux or BSD, but IDC and other industry groups will report that Linux and BSD have less than one percent of the OS share, because they measure only paid OS copies.
You must measure something correctly to be able to make correct decisions. In modern accounting, pollution has no cost, only trying to stop pollution has a cost. So a CEO will try to increase pollution, not decrease it.
Re:Thanks IDC! (Score:1)
Thanks IDC! (Score:4)
1. The consulting company brought in by our PHSM uses IDC demand-side research (for PURCHASERS by PURCHASERS)?
2. We (the IT group) and our users are getting tired of our Novell file servers and *nix web servers never crashing?
3. Microsoft invested $600 million in our company a year ago when we really, really needed it?
And the bonus question: is it still called demand-side when it's being shoved down our throats?
(BTW, Deloitte Consulting believes in Microsoft software and IT outsourcing.)
Re:Methodologies and Trust (Score:1)
 _
Re:Thanks IDC! (Score:2)
Is that your company running business burying ships deep into the sea [gcn.com], called US NAVY?
 _
Technical Sector (Score:1)
Least these guys tell you how to do it properly.
Interesting... (Score:1)
"Our analysis is that many users of Linux don't want their organizations or their competitors knowing what they're running. "
Why is this? Why don't many users of Linux want their organizations or their competitors to know what they're running?
I honestly don't understand that statement. What does it mean?
Re:Interesting.. but.. (Score:2)
All I wanted to know was if and how we can see what types of questions were asked, and then draw our own conclusions of the analysis. I didn't want to know about software vs. hardware server numbers. Just say: "No, you're not allowed to see the questions we ask, no one is." Then I'll be able to say, fine, then I should be expected to trust your judgements of what the data means. But maybe if he had just said: "Yes, you can see how we 'poll' people, but you have to pay for access to it like our clients do.", then I would likewise understand his answer. Is it so hard for someone to take a stand these days and just give a straight answer? My question was simply meant to find out why analyses of polling data was so often quoted as fact these days, when it's possible that the way the poll was conducted was completely bunk. Yes you can independently verify our analysis, or no, no one is allowed to verify our conclusions, was all I was looking for out of him.
Re:Interesting.. but.. (Score:2)
OK, I understand what you're saying now. I guess my original question was more directed at how do we (or you) verify the data with which you are presented (be it actual sales figures, or subjective questions asked of individuals and/or companies). As you mentioned in response to my post to this discussion, some clients rely upon you to merely collect data for them, then they analyze it as they see fit. So I guess what your saying is that any 'result' that someone publishes should definitely be taken with several grains of salt, and merely used as a guide post, but not the only way to measure a conclusion. This seems fair, and really does answer my question in that, if I pay your company for all the collected data, in essence, I would have access to see all the sales figures with which you used to do the analysis. Thank you!
Re:Thanks IDC! (Score:1)
Re:My analysis (Score:1)
"What are we going to do tonight, Bill?"
Not commercially viable (Score:2)
Ummmm....what? Let's say I'm a purchaser who needs to know what the best file and print server is. If I didn't have IDC, who would I ask? A purchaser at another company or a tech who has previously setup and maintained a file and print server?
In other words, tell me the opinions of people who have valid opinions, not J. Random Loser. If that ain't "commercially viable" I don't know what is.
--