Stories
Slash Boxes
Comments
Search

News for nerds, stuff that matters

Slashdot Log In

Log In

Create Account  |  Retrieve Password

Linux From A CIO's Perspective

Posted by Zonk on Fri Jul 01, 2005 01:48 PM
from the look-at-all-the-penguins-down-there dept.
Linux Business IT
An anonymous reader writes "CIO.com has a story on Linux and OSS in the enterprise from the perspective of the CIO of Cendant Travel Distribution Services, Mickey Lutz. 'In the summer of 2003, Mickey Lutz did something that most CIOs, even today, would consider unthinkable: He moved a critical part of his IT infrastructure from the mainframe and Unix to Linux. For Lutz, the objections to Linux, regarding its technical robustness and lack of vendor support, had melted enough to justify the gamble.' His organization saved 90% in costs in so doing. Read on if you want to see how the top brass views OSS."
+ -
story
This discussion has been archived. No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.

Linux From A CIO's Perspective 50 Comments More /

 Full
 Abbreviated
 Hidden
More
Loading... please wait.
  • This pretty much sums it up:

    Lutz's IT group rewrote a complex, real-time airline pricing application that serves hundreds of thousands of travel agents around the world and that also acts as the system of record for all of United Airlines' ticket reservations. When this application came up on Linux, it proved to be so demanding--it handles up to 700 pricing requests per second--that it completely redefined Cendant's expectations about what it would take to get Linux to work. "We have broken every piece of software we've ever thrown at this platform, including Linux itself," says Lutz.

    With Big Iron you're paying a LOT of money. But you're not paying it for nothing. Big Iron will give you a lot of guarantees for stability, reliability, and thoroughput that don't exist on other systems. The key to this CIO's success is that he was willing to accept the challenges of doing Big Iron work on Little Brass systems. As long as you work all the details out yourself, this *can* work. (As Google has so eloquently proven. [linuxtoday.com]) The issue is that you're working without a safety net. If things go really wrong, there's no backup army of specially trained techs to run in and fix things. (And trust me, if you're paying enough money you'll have your own personal army of techs.)

    The upshot to all of this is that if the gamble pays off, it pays off in a big way. All that money you were spending for a personal army, plus some other company's R&D now goes into your own pockets. You don't get away scott free (someone has to maintain the systems), but you see your rewards. And isn't that what business is about? Taking risks and making profits? If you've got the infrastructure to go for something like this, then go ahead and grab fate by the balls. No one ever got anywhere in life by playing it safe. ;-)

    The "black box" of open source has transformed into something any CIO can appreciate: reliable performance and consistent uptime. The penguin can fly now.
    • The issue is that you're working without a safety net. If things go really wrong, there's no backup army of specially trained techs to run in and fix things.

      Well, there is a backup Army, and it's you.

      Google can have a 4000-node Linux cluster because they have enough staff to maintain and optimize the system (Keep an eye on their job pages to get an idea).

      Google also has some highly specalized needs-- some machines only crunch data for the DB, other machines only serve webpages, etc. It's in their interest to optimize the Kernel, OS, Database & Web applications as much as possible. Take a tweak which gains a 1% performance gain, multiply that against 4000 machines, and it's quite an advantage.

      There isn't a vendor in the world that can totally support their infrastructure, so Google does it themselves.

      • Re:Difficult, but big payoff (Score:5, Funny)

        by Anonymous Coward on Friday July 01 2005, @02:26PM (#12963755)
        Take a tweak which gains a 1% performance gain, multiply that against 4000 machines, and it's quite an advantage.

        Let's see . . . that's . . . [pencil scratching] . . . 1%! Amazing!

      • Well, there is a backup Army, and it's you.

        No, you are the front lines army. The backup army was the reason you were paying the annual fees. Without those annual fees, there is no backup army. i.e. If you can't get it right, there's no one else to come in and fix it for you.

        Take a tweak which gains a 1% performance gain, multiply that against 4000 machines, and it's quite an advantage.

        That's something of a straw man argument. If 3 Sun machines and 10 LinTel boxes have the same Flop capacity, then a 1% increase in either one will add up to the same increase in computing power. The key difference is that there are only three Sun machines to update.

        There isn't a vendor in the world that can totally support their infrastructure, so Google does it themselves.

        That doesn't mean that there couldn't be. Google made their choice to go with a large number of decentralized systems. It works for them and it works well. But they have to do everything internally *because* of that decision. Had they gone the EBay route, they would be able to get that backup army, but then they would pay for the priveledge.
    • Dude I use to work for Cendant, and I can tell you that the "Army of IBM" people you got was mostly a bunch of idiots. They use to be good, but most don't know jack anymore.

      I won't get in to the fact that they (IBM) flat out stole $150 Million from a Cendant affiliate (RCI) by telling them that they could build a distibuted system to replace their mainframe. You could sit in the room with the IBMers and they would say "This thing is NEVER going to work, we can't replicate data to all these AS400's around

  • unthinkable? (Score:4, Insightful)

    by delirium of disorder (701392) on Friday July 01 2005, @01:51PM (#12963291) Homepage Journal
    Moving desktops to linux might be considdered revolutionary, but this isn't. The big iron market of servers and HPC machines has really been dominated by linux for several years now.
    • It is in the Fortune 500. As a previous poster pointed out, big iron means very high reliability. Lutz made a very brave decision and made it work.

      • Re:unthinkable? (Score:5, Insightful)

        by delirium of disorder (701392) on Friday July 01 2005, @02:33PM (#12963830) Homepage Journal
        I donno....google seams pretty reliable. Akamai web caching is designed to ensure online reliability (and is Linux based). The standard for Internet based systems that need long uptime and robustness is GNU/Linux. This is because the web as a popular medium is only about a decade old. The only reason why other organizations (banks and the military for example) that need reliable systems have not migrated to Linux is because they use legacy code that would be expensive to port.

        The particular solution the CIO in the article choose gambled with reliability because they used 144 separate servers in 12 clusters. Well implemented clusters of x86 hardware can run seamlessly, but individual machines are likely to fail. Redundancy should mean that a couple blown power supplies or corrupted disks a year is no big deal, but it's still a slight risk and a pain to fix. There are Linux solutions that run on larger machines. They could have replaced their four IBM mainframes with four Linux mainframes. IBM supports Linux on the zSeries mainframes (formerly called System/390, before that System/370, which was the successor to System/360..its about as traditional a mainframe as you can still buy). The top high end computers are Linux based. The top 3 most powerful computers in the world at this time, two IBM eServers and a Sgi Altix, all run Linux. Linux offers the most flexible, powerful, and reliable solutions out there.

  • Linux beats Unix on cost (Score:4, Funny)

    by bedroll (806612) on Friday July 01 2005, @01:51PM (#12963300) Journal
    The only thing that makes this news is that a CIO actually recognized it.

  • Considered unthinkable (Score:3, Insightful)

    by Anonymous Coward on Friday July 01 2005, @01:52PM (#12963305)
    would consider unthinkable: He moved a critical part of his IT infrastructure from the mainframe and Unix to Linux.

    This is actually quite thinkable. Now if some CIO moved all his desktops to Linux, I would be impressed. Moving the backoffice stuff from expensive licenses of Unix and mainframes to Linux is a no-brainer.

    • Re:Considered unthinkable (Score:5, Insightful)

      by Decaff (42676) on Friday July 01 2005, @02:39PM (#12963901)
      Moving the backoffice stuff from expensive licenses of Unix and mainframes to Linux is a no-brainer.

      No it isn't. There are many very high volume commercial and financial websites that use features of commercial Unixes, such as memory and resource partitioning, self-healing, fault management and very high scalability. Linux will certainly get all these at some point, but until then it is certainly not a 'no-brainer' to move. Even with smaller systems there are many applications that require certain Unix versions.

      • I'm not sure how to respond to the "no-brainer" label. My development team just migrated our company's production software development platform from a network of aging HP-UX machines (which served us quite well in their day, don't get me wrong) to a Linux network. The new development tools run 20 times faster (that's the actual figure, not hyperbole), our experience over the past two years is that the Linux network is much more reliable, and the server hardware is simply an increment onto their existing W

  • I'm reading through this and it's interesting. (Score:5, Interesting)

    by Anonymous Coward on Friday July 01 2005, @01:54PM (#12963332)
    There are two things I find really interesting here:
    1. Vendor support. OK, so if vendor support has gotten better, then which vendors does this CIO recommend? I don't see which ones he used in the article, maybe I just missed something.
    2. This quote
      "Open source is propelling us to adopt Java and a new way of programming," he says.
      Should be a bit of a cluebat for both Sun and the Open Source extremists. Java and Open Source can be extremely good for each other, it's just that both Sun and the Open Source community need to learn to cooperate on practical matters with those whose ideological goals differ... unfortunately it seems that neither Sun nor the Open Source community is extremely interested in realizing how much they could benefit from the other.
  • Clearly (from the get the facts site) it costs even less than Linux (just kidding) -- so I'm guessing Linux won not because of cost but because of technical superiority.

    Any other ideas?

    • The obvious answer is that it is an order of magnitude easier to port legacy, core business applications from Unix to Linux. This wasn't some startup with no existing infrastructure.

      When making a decision to change OS platforms, you must consider the cost in moving legacy applications over.

  • interesting questions raised (Score:3, Interesting)

    by Exter-C (310390) on Friday July 01 2005, @01:59PM (#12963403) Homepage
    An interesting question that this article raises for me. Is what intel arch was being used (itanium/x86). For example could the costs have been reduced just by using linux on say a large scale IBM server similar to their other mainframe?.

    It also goes to show that just because something is old does not mean its slow..

  • hey Mickey... (Score:3, Funny)

    by C0vardeAn0nim0 (232451) <covarde,anonimo&gmail,com> on Friday July 01 2005, @02:03PM (#12963464) Journal
    some guy name "bill" called from redmond. he wants to explain you why linux is more expensive...

  • Small point of correction (Score:3, Informative)

    by Alphabet Pal (895900) on Friday July 01 2005, @02:05PM (#12963496)
    Lutz was in command of the alternative to those bright, shiny websites: an expensive, aging global distribution system (GDS) called Galileo.

    Actually, all of those bright, shiny websites (Expedia, Travelocity, and Orbitz) rely on a GDS (Sabre, Amadeus, Worldspan or Galileo) to provide their content.

  • Cost breakdown (Score:5, Interesting)

    by alvinrod (889928) on Friday July 01 2005, @02:06PM (#12963498)
    Mainframe: $100 million

    Unix: $25 million

    Linux: $2.5 million

    These numbers were taken from a table in the article. Interestingly enough, the cost if something does break favors Linux as well. From the same table we get that the mainframe solution consists of 4 IBM mainframes, whereas Linux and Unix solutions require around 144 servers for Linux and 100 - 120 servers for Unix. If the hardware goes to hell it's so much easier to replace the single bad part than a mainframe.

    Hopefully, more people will begin a transition to open source solutions when they realize it can be successful.

    • Except IBM mainframes tend to call home and a tech is at your site before you typically know there is a problem... drastic failuers not included.
    • If the hardware goes to hell it's so much easier to replace the single bad part than a mainframe.

      Not to detract from your point, but mainframes don't break as a single piece unless the machine blows up or is otherwise completely destroyed. Big Iron systems are designed with redundant *everything* including motherboards, CPU, memory, network cards, power supplies, and disk drives. If any one part fails, the system will route around it. The part can then be powered down and ejected from the machine. To bring it back up to full capacity, you simply plug in the replacement part and walk away.

      In that light, Linux system failures are actually going to be more difficult to repair. However, the cost of repairing a Linux system is far less (disposable box) despite the inherent difficulty. :-)
    • I'm not a Big Iron guy but it's my understanding that main frame hardware does NOT "go to hell" unless someone hits it with a hammer. Repeatedly. And with forethought. They are highly redundant devices, hence the expense.

    • Please note that these are annual costs associated with each system.

      If the hardware goes to hell it's so much easier to replace the single bad part than a mainframe.

      The reason the mainframe costs so much per year is exactly this issue - first, if something breaks it likely cripples but does not disable the machine. Second, IBM fixes it within four hours or less.

      Of course, they are still using IBM hardware, and the reason linux is so expensive (2.5M/Yr) is because they likely have the same type o

    • You forgot a line (Score:3, Informative)

      by jd (1658)
      A CIO who takes a "chance" with Linux: Priceless

      Seriously, the biggest problem with mainframes is that switching them off is a big problem. These are not boxes you can easily - or safely - reboot, if there is a problem. There usually isn't, because the hardware is usually of very high calibre and massively redundant, but scheduled maintenance of, say, an Amdahl or a (when they existed) a Prime was not a trivial affair.

      "Routine" maintenance wasn't much better - DEC would charge the Earth (and Mars) to s

    • Well you can turn that logic around too. When you have 144 boxen, you much more likely to have a failure then when you have 4. But neither that nor your arguement makes any sense. Mainframes as a whole do not go down. Period. A CPU (or 3) can get completely fried and the machine won't miss a beat. Really. And you'll probably have an IBM tech there to fix it before you even know it's happened, since the machine phoned the problem in as soon as it happened. Big iron is expensive no doubt, but if there

  • Spread the word! (Score:4, Interesting)

    by bogaboga (793279) on Friday July 01 2005, @02:11PM (#12963562)
    [...] Mickey Lutz did something that most CIOs, even today, would consider unthinkable: He moved a critical part of his IT infrastructure from the mainframe and Unix to Linux. For Lutz, the objections to Linux, regarding its technical robustness and lack of vendor support, had melted enough to justify the gamble.'[...] His organization saved 90% in costs in so doing.

    Now, let's get prepared to rebut any Microsoft officials whenever they talk about the common "Total Cost of Ownership" as far as Linux is concerned.

    • Sure! :-) Windows 2003 Server license, that's about 300$, multiplied by 144, 43.200$, right?

      And who says windows needs TCO? Any guy in the office can admin a windows box...

  • The risks and the rewards (Score:5, Insightful)

    by AB3A (192265) on Friday July 01 2005, @02:18PM (#12963652) Homepage Journal
    Like most critics, I'm not good at leading large companies. But I know good leadership when I see it. This guy Lutz has his head bolted on right.

    The first thing most CIOs usually throw at their workforce is not to re-invent anything. If a product exists off the shelf at a reasonable cost, there are lots of disadvantages for taking the risk of inventing another one and few advantages if you succeed.

    However, most of us workers have known that the "big iron" mainframe technologies of yesteryear are starting to "rust." It's getting difficult to find technical help who understand this stuff reasonably well. That brings me to the second point: Follow the technology market. The people will be there.

    I suspect that in the not too distant future, many big-iron mainframers are going to be asking theselves whether the many millions they're spending are a good ROI. Open source databases and distributed computing are starting to look awfully attractive.

    It's scary from a CIO's position because the old systems are working, even if they're not well understood any more. They're leaping from the systems they know, toward a high cost potential boondoggle. This guy apparently knew how to hire and retain good technical help, he knew how to organize that help, and he knew how to keep them focused on the goal.

    Most leaders aren't that good. All too many businesses operate by habit. Only the red tape holds them together. Those organizations won't be making this leap until a certain critical mass has been reached to convince them one by one to make the effort.

    We should be doing everything we can to encourage others like Lutz to push these efforts. This is how you really evangelize Linux. And when all this is over, the desktop will be an afterthought.
    • Like most critics, I'm not good at leading large companies. But I know good leadership when I see it. This guy Lutz has his head bolted on right.
      I'll have to disagree with that. He made a good choice in going to Linux from Unix, but he did so is such a fucked up way that it was only Linux's technological goodness that saved him from being a poster boy for Microsoft's "Linux sucks" campaign.

      Here, from TFA:
      The decision not to focus more on testing came back to haunt them.
      The CIO decided not to TEST the system correctly?
      Frantic calls began coming in from some of the 44,000 travel agency locations in 116 countries that were unable to access Fares.
      Their customers cannot access their new Linux system!
      Lutz would not comment on the financial losses incurred by United or Galileo during the downtimes.
      They were LOSING money with their new Linux system.
      "In hindsight," says Lutz, "we shouldn't have tried to cut over to a new infrastructure at the same time we were deploying a new software application. It was too much at once."
      This guy made novice-level mistakes and it was only because Linux is so good that this became a huge success rather than a terrible failure.
      Rather than falling back to the old platform at the first signs of trouble and reworking the new one, the engineers always thought the answer was around the corner.
      You always have a back-out plan. Always.

      This guy took a huge risk ... screwed it up royally ... and was saved by IBM, Red Hat and Linux.

      And the Linux system STILL saves him $$$MILLIONS$$$ every year and OUTPERFORMS his old system.

      It's one thing when you're a genius CIO who plans and test for every contingency and deploys a working Linux system.

      It's a completely different thing when you don't BUT YOU STILL SUCCEED BECAUSE OF LINUX.

      This story is important because it shows the average CIO that, even if you aren't a genius and you DO make mistakes, Linux can STILL save you barrels of money and make you LOOK like a genius.

      • Look, you have three ways to trasition. -
        • Pure cut - which they did
        • metered out - ie 10% then 20% ...
        • parallel systems

        This gut chose the first one - linux had nothing to do with it. If he had gone to a new propriatry system - the SAME thing would have happened. Linux is only a bit player here.

        Sera

  • A Bold Move (Score:3, Insightful)

    by endeavour31 (640795) on Friday July 01 2005, @02:20PM (#12963686)
    An interesting read. But very interesting for what was left unsaid as well. There was a fair amount of pain associated with the switch - aggravated by rolling out a new application simultaneously. The slowdowns and the associated costs are glossed over but I wonder how the business side feels about this change to only 25% of the entire infrastructure.

    The time window seems fairly broad as well. No one disputes that lots of cheaper intel servers can do the same job as big iron. THe question is how many does it take and what happens with the applications involved.

    Quite telling is the comment that they needed every bit of support possible. Although it is great that one CIO bit the bullet here....there is an ominous side to this story which means that few others will follow suit.

  • aww (Score:2, Funny)

    by Pyrrus (97830)
    I was hoping to see a "Windows has lower TCO than Linux" ad that slashdot runs for Microsoft when I clicked the article.
  • His organization saved 90% in costs in so doing.

    But did he get a raise? Say about half of what he saved them.

  • "In hindsight," says Lutz, "we shouldn't have tried to cut over to a new infrastructure at the same time we were deploying a new software application. It was too much at once."

    They found that their Linux servers couldn't support the new application they had deployed at the same time. That doesn't mean it's less capable than the mainframes they replaced: they didn't even try running the higher-load application against the mainframes.

    They should have first ported their servers to Linux on the mainframes, then switched them to Linux on clusters, then sent out new software that they could force back to the old behavior, then supported the new software in general.

    That way, they'd have been able to isolate the problems more easily (which really turned out to be that the new application generated extreme peak loads, and nothing to do with Linux per se, aside from that they managed to improve the Linux performance to deal with it) and keep things stable while they fixed the issues.

    • Re:Well (Score:5, Insightful)

      by Halvard (102061) on Friday July 01 2005, @02:01PM (#12963428)
      Most don't come out of geekdom, rather from business school and worked previously marketing, sales, or some other management area. The don't have the knowledge or skill to be geeks and must rely on them. To make a move like this one, you must have good ones that you trust not just with the business but **with your career**. That's ultimately more important since your family depends on it.
    • This news is all well and great, but it's been known for a while that moving from UNIX->Linux was cheaper.

      That's not entirely true. If you look at the TCO, Linux is only cheaper if you're willing to cut out the safety nets that are so expensive. i.e. If you get an annual mantenence contract with RedHat and Dell, then how much are you actually saving over a Sun machine with a contract for both?

      Corporate purchasing decisions are never as simple as the upfront cost. The key is that if you're willing to t
    • This may just be my ignorance speaking, but last time I checked Windows had horrific High Performance Clustering capabilities, so there is no comparison to make. In addition, the licensing issues to go along with Windows 2003 advanced server or whatever you need to get HPC is ridiculous. Meanwhile, Red Hat has great clustering capabilitiesn enterprise support for it, and the clusters work well and are integrated giving high ease of use and great performance. Thats why this CIO went with them and if you read
    • Of course there are people in this audience who have experience switching corporate computing platforms to linux - I've been working with a number of companies who have moved, or are in the process of moving services to linux, not only from old school risc unix systems, but also from high-maintenance microsoft windows platforms.

      I'm sure my experiences are just like those of a lot of other sys admins here - nothing surprising, just a quiet evolution that is working quite well.

      On the subject of linux to oth
    • Another flamebait from the idiot stephanie. This exact article was posted the other day. Please mod it down. Bill. Go Away!
    • I installed WinXP Pro SP2 on a machine at my house the other day. Not difficult, and it took a while to get all of the little extra we use: Java, Flash, Adobe Reader, PDF Creator, Firefox + plugins, nVidia drivers, wireless card drivers, etc.

      However, the next step was to go to Windows Update and apply all critical & security patches. It did and wanted to reboot.

      Then refused to reboot, even into Safe Mode. WinUpdate had hosed the system but good.

      After searching around I found that one of the update
      • What you need to do is get another machine (it doesn't have to be top-of-the-line) solely to experiment on. This machine is called in CompSci circles a "testbed". When testing is done and you are sure everything works (and are confident that it will stay that way) then, and only then, install that program on the computers in use.

        First, you're feeding a troll.

        Second, that's not exactly the most convincing bit of Linux advocacy I've ever heard...

All trademarks and copyrights on this page are owned by their respective owners. Comments are owned by the Poster. The Rest © 1997-2009 Geeknet, Inc.