Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 



Forgot your password?
typodupeerror
×
Linux Business

Unreliable Linux Dumped from Crest Electronics 960

nri writes "The Age writes, Linux misses Windows of opportunity. Crest Electronics chose a Linux operating system, then seven months on, the company chose to abandon it for Windows. Mr Horton says. ".. the machine would basically, putting it in Windows terms, core dump or blue screen at random. It would run for weeks or so and then just bang, it would stop....I fully support Linux but if I had to make the decision again I'd pick Windows. A big reason is the fact Windows was up and running in two hours at all the right patch levels. The installation of SAP took two days on Windows, the installation on Linux Red Hat took two weeks. The total cost of ownership is actually lower in this case than with Linux because of the hidden costs of the support.""
This discussion has been archived. No new comments can be posted.

Unreliable Linux Dumped from Crest Electronics

Comments Filter:
  • by LittLe3Lue ( 819978 ) on Wednesday September 28, 2005 @10:59PM (#13672827)
    ...we will see what you have to say about hidden costs and core dumps.
  • Windows vs Linux (Score:4, Insightful)

    by mboverload ( 657893 ) on Wednesday September 28, 2005 @11:00PM (#13672833) Journal
    Anyone that says that Linux will beat out Windows in every situation is a fool.

    Choose the product that best suits your needs. If Linux doesn't cut it, get Windows. If Windows doesn't cut it, get Linux.
  • There's no debate. (Score:5, Insightful)

    by amarodeeps ( 541829 ) <dave@dubitab[ ]com ['le.' in gap]> on Wednesday September 28, 2005 @11:00PM (#13672836) Homepage
    It costs money to hire qualified admins, Windows or Linux.
  • by MrMista_B ( 891430 ) on Wednesday September 28, 2005 @11:02PM (#13672844)
    >If Linux doesn't cut it, get Windows. If Windows doesn't cut it, get Linux.

    Or, y'know, a Mac.

    OS X and all that. Hell, Intel stuffs even, in a couple months.
  • by little alfalfa ( 21334 ) <cohen.joel@NOspAm.gmail.com> on Wednesday September 28, 2005 @11:03PM (#13672850)
    Obviously, your admins were not qualified to administer a Linux server like this. If it took them two weeks to get software installed and running like that, I'd fire them right away. Even if it is SAP, a complex piece of software. Just because you got it up and running in 2 days on Windows doesn't mean it was done right, or done securely.
  • by detritus` ( 32392 ) <awitzke AT wesayso DOT org> on Wednesday September 28, 2005 @11:03PM (#13672851) Homepage Journal
    Yes, but windows admins come a lot cheaper... at least up here where everyone and their dog has an MCSE
  • by CyricZ ( 887944 ) on Wednesday September 28, 2005 @11:04PM (#13672860)
    I wish he would have given us more information regarding the problems he ran into. I'm talking about system specs, the name and version of the Linux distro used, and more information regarding the software they apparently had so much trouble installing.

    When problems do happen, the open source community is notorious for getting them fixed very quickly. If he were to provide us, the community, with more details about the problems he encountered, I just know they could be solved for him and potentially for many other users in a similar boat.

  • Smells fishy. (Score:5, Insightful)

    by SynapseLapse ( 644398 ) on Wednesday September 28, 2005 @11:07PM (#13672881)
    This whole article is useless without really saying what the crash was. You could have the most rock solid stable server in the world, and it won't mean much if the applications you're hosting are buggy and badly implemented. It would be nice to know to EXACTLY what crashes he was getting and why. Not just "Uhh, there were core dumps and blue screens, but with a linux blue instead of microsoft blue." I think this would be a great opportunity for an Ask Slashdot poll. Maybe he'd even post some of the core dumps.
  • by Anonymous Coward on Wednesday September 28, 2005 @11:09PM (#13672891)
    It may be true in some cases that Windows runs more stable than linux. I have seen some flakeyness on more bleeding edge distros, X11 crashing, apps crashing. One of my boxes, I have troubled hardware support for my Promise SATA controller and large data transfers would cause system lockups all the time. Supposedly this is fixed in kernel 2.6.12. But I'm running Windows XP on that machine so I don't really know. But really, XP stablity isn't all it's cracked up to be. I have to reboot often (~once a week). Things just slow down and get really sluggish after ~ 2 weeks, or less.

    But hardware/driver issues aside, I don't believe Windows can be more stable than linux. If you don't have to run Windows for some specific compatibility/software requirement. Linux can be a far superior experience.
  • by bblazer ( 757395 ) * on Wednesday September 28, 2005 @11:09PM (#13672892) Homepage Journal
    I can understand the long install time. This is proof on one of the major flaws with linux. Poor documentation, poor standards across distros, and obscure undocumented dependencies. Don't get me wrong; I have been using linux since 1999 and have come to appreciate a lot of it. But still to date I want to bang my head on my keyboard when I install some new software and I am told that I need such-and-such lib or a different version of something. Then a good part of a day is shot trying to track all of this stuff down and get it installed. What I have just said should be tempered by the fact that I do not believe that windows is a good choice. 99 times out of 100 you have an unstable machine that costs you huge $ in downtime. This is where linux (and Mac) is good. Once you get it set up it is rock solid. I guess that you have a choice, long set up with linux then less maintenance or short set up with Windows, and a lot of further maintenance.
  • by The Famous Druid ( 89404 ) on Wednesday September 28, 2005 @11:11PM (#13672915)
    That the decision to go Linux was made by his predecessor.

    Looks like 'new manager' syndrome to me...

  • Comment removed (Score:5, Insightful)

    by account_deleted ( 4530225 ) on Wednesday September 28, 2005 @11:12PM (#13672929)
    Comment removed based on user account deletion
  • by Anthony Liguori ( 820979 ) on Wednesday September 28, 2005 @11:12PM (#13672931) Homepage
    When a program dumps core, it means that the program did something that it wasn't supposed to do (like try to read memory that isn't valid) and the operating system has (correctly), stop the program's execution, and to make life easier on developers, copied the program state into a handy file so that the problem can be debugged. No other programs on your system will be harmed by this one malfunctioning program.

    When Windows blue screens, it means *the operating system* has done something it wasn't supposed to do (like try to read memory that isn't valid) and the operating system bails. Often, it will return execution to the next instruction and hope things will be okay. It almost certainly isn't. You're basically screwed.

    The equivalent in Linux is an Oops. They don't happen that often on production systems. A crappy properitary program doing things it's not supposed to is *not* a Linux problem nor an Open Source problem. It's SAP's problem.

    This is a testimonal about the crappiness of SAP and nothing more. They obviously didn't do enough testing on Linux.
  • by rleesBSD ( 909405 ) on Wednesday September 28, 2005 @11:15PM (#13672948) Journal
    If neither one cuts it, get FreeBSD. (Hey, don't forget about us!)

  • Re:blue screens? (Score:2, Insightful)

    by Anonymous Coward on Wednesday September 28, 2005 @11:15PM (#13672952)
    You haven't used X11 I take it. It makes Windows 95 look like a quality, well-built OS.

    Seriously, if X doesn't get fixed sometime soon, I'm dumping Linux. I like everything about Linux but X, but that's a big but. It's slow, it's unreliable, it's ugly, it's impossible to configure and maintain, it's prone to crashing and leaving your machine effectively unusable (because of all the stupid hacks in the display management system of Linux and X11), etc. etc. etc. And the fonts look terrible and I have spent countless hours recompiling freetype, X, fontconfig, with different options, different flags in the config files and, if anything changes at all, it is usually between varying degrees of ugly.

    It's really unfortunate. Most problems with Linux on the desktop are problems with X.

    PS: the confirmation text is "inaction", which is exactly the problem with X development. Nobody, except a guy here or there like Jon Smirl, is really stepping up and saying we need to *fix* X. It's just band-aid solution after band-aid solution. Have they learned nothing from Microsoft?
  • *nix incompetence (Score:3, Insightful)

    by bombadillo ( 706765 ) on Wednesday September 28, 2005 @11:15PM (#13672958)
    From TFA , a quote from RedHat support regarding Crest...

    "We asked the customer to do a diagnostic test and the customer never responded, so it was impossible for us to address the issue," Mr McLaren says.

    These Crest guy's didn't even have the ability to use support properly.

    and

    "We run Linux on our web server"

    The entire company has 1 webserver? Unless he was missquoted this guy doesn't have a clue what his IT department should be doing.

    Nuff said.
  • by ScentCone ( 795499 ) on Wednesday September 28, 2005 @11:19PM (#13672993)
    whereas you can expect windows to core dump periodically and predictably

    You know, I've had that happen enough to care about - years ago, with older copies of NT, running on flaky/overheated/bad-sectored hardware. But I run things like SQL, or file services, or IIS under 2000/2003... and have machines that cook along without me doing anything month after month after month. No BSDs, etc. Yes, patch = boot, and that's a few moments of taking a machine out of a cluster for a minute... but not because the machine hangs while doing anything routine. For that matter, not even when I'm doing something non-routine.

    This whole "Windows just crashes all the time" stuff, especially on the server side, is pretty much FUD. Bad RAM and drives can piss off Linux, too. Flaky commercial third-party apps can gum up any OS. But I sure don't have anything like the problems that so many people love to rant about - and even though I only have a running sample of a few dozen specific machines that I actually consistently lay hands on every week, you'd think that the mythical "predictably, always crashing" Windows server would rear its ugly head at some point. But it doesn't. The FUD's an anachronism.
  • by clarkie.mg ( 216696 ) <mgofwd+Slashdot&gmail,com> on Wednesday September 28, 2005 @11:27PM (#13673069) Homepage Journal
    I hate those *false* links that redirect to a registration page. Even if it's free, do they imagine anyone is going to fill those long forms for every page they visit.

    Fortunately, the bugmenot bookmarklet did the trick.

    About the story : so we have *one* situation where a problem happenned between SAP and linux. That kind of conflicts happens all the time in IT. Either you solve it or you change one component.

    In both cases, drawing general conclusions on the abandonned product is common but unfair and a sign of lower qualifiquations.
  • by dotgain ( 630123 ) on Wednesday September 28, 2005 @11:28PM (#13673076) Homepage Journal
    Exactly. Great write-up, taco! Do you expect anything other than a flamewar between people boasting more than 400 days of uptime on opposing OS's?

    You're the troll, not the trolls.

  • by Overly Critical Guy ( 663429 ) on Wednesday September 28, 2005 @11:31PM (#13673097)
    Better tell the army [netcraft.com].
  • by bfizzle ( 836992 ) on Wednesday September 28, 2005 @11:33PM (#13673110)
    I've seen systems get slow on Windows machines over and over because of memory leaks. Of course Linux will do the same thing, but at least you are free or have more say to your vendor to fix things. Try telling Micrsoft to fix the memory leak in IIS... they just laugh in your face.

    Neither are perfect nor will they ever be, but getting good support for Linux just seems easier.
  • by Q2Serpent ( 216415 ) on Wednesday September 28, 2005 @11:34PM (#13673116)
    I'm a long term linux user.

    I've never encountered it.
  • by dbIII ( 701233 ) on Wednesday September 28, 2005 @11:35PM (#13673120)
    That the decision to go Linux was made by his predecessor.
    Good point - the first thing a lot of new managers do is tell you that everything that was done before they got there was done the wrong way. Also two weeks does make sense - if you have to learn on the job.

    Even when you aren't learning on the job it makes sense to kick a test system around for a few weeks if it is a major change of complicated production software with some consequences if it's down for a while or runs incorrectly. After that you roll out the installs to the other machines in a few hours.

  • by jrockway ( 229604 ) * <jon-nospam@jrock.us> on Wednesday September 28, 2005 @11:38PM (#13673139) Homepage Journal
    Tell that to our 3TB backup server or the University of Illinois' 500-node OSX cluster.
  • by at_slashdot ( 674436 ) on Wednesday September 28, 2005 @11:39PM (#13673143)
    Do you observe that lately if someone puts Windows instead of Linux is news.

    Just like: a dog bites a man is not news, but a man bites a dog is. That's telling.
  • by aussie_a ( 778472 ) on Wednesday September 28, 2005 @11:41PM (#13673159) Journal
    Any decent sysadmin knows concepts, not platforms, and can work with whatever you hand them

    Boss: I want result X. Sys-admin: Oh that's easy. In theory you do acts A, B and C. In theory it takes about Y weeks to do it. Boss: Great, I'll let you get to it. Y weeks later. Boss: So how is it going? Do you think you'll be able to finish it in a timely manner? Sys-admin: I've barely started, I don't know your platform, I only know the concepts.

    To say a great sys-admin shouldn't know a platform, only concepts AND be able to work with whatever you hand them is ridiculous. A good foundation of concepts and theory is essential to a good sys-admin. But experience and knowledge of a particular platform that they're expected to use is also important. It's like knowing the concepts in OO programming is much more important then knowing a particular OO language. However you can't hand someone a task to do in a specific OO language without some training prior in that particular language (whether it be self-taught for a few days or a course for a few weeks).
  • by soconnor99 ( 83952 ) on Wednesday September 28, 2005 @11:44PM (#13673166)
    It's that kind of clear, level-headed thinking that we just don't care for around here.
  • by Nefarious Wheel ( 628136 ) on Wednesday September 28, 2005 @11:47PM (#13673181) Journal
    We asked the customer to do a diagnostic test and the customer never responded..

    "So what'd they say? They have any bodies they can throw at us?"

    "Nah, they just told me to run a fricking diagnostic. They're not interested."

  • by menkhaura ( 103150 ) <espinafre@gmail.com> on Wednesday September 28, 2005 @11:50PM (#13673197) Homepage Journal
    Yeah, please don't hurt the feelings of both of us!

    But, seriously, BSD > any Linux flavor > Microsoft's sorry-excuse-for-an-OS.

    The BSDs don't have the fragmentation that Linux has. If anyone asks me what is my OS, I say simply "FreeBSD". By that I qualify my package management, my system boot scripts, where my conf files are, how the system works. "Linux", on the other hand, can mean a bunch of things: maybe the kernel, maybe one of those hundreds of distros, each with its own idea of package management, file placement, system configuration, or boot method. Of course, they are all Linux, they all run roughly the same software (Apache is Apache no matter in which Linux distro you run it), but the details, the little differences, do hurt Linux (okay, Stallman, GNU/Linux, as you wish) by making it into a moving target for support and maintenance.

    Back on topic, that Linux machine must have had some hardware flaw. Bad memory comes to mind...
  • by utnow ( 808790 ) <utnow@yahoo.com> on Wednesday September 28, 2005 @11:52PM (#13673206) Homepage
    Only because of a smaller user-base. It's easier to garner to your customers when you don't have many, not to mention you're more inclined to do so. In this case, good support in Linux is only good as long as not everyone is using it. So shhhhhh.... stop telling people to switch.
  • by mcrbids ( 148650 ) on Wednesday September 28, 2005 @11:56PM (#13673219) Journal
    Hmm. How about 1 day to install Linux, and the rest is setting up SAP and testing?

    1 day. WTF?!?!? I routinely sell embedded server systems (using Whitebox Linux) that update themselves (a la yum) and have it all set up in under 15 minutes.

    Maybe those SAPs really outta learn what an installer script is - I can (no kidding!)

    1) load an installer CD (maybe 10 minutes for a "minimal" install)

    2) stick in an installation CD, and run the installer

    3) Have a functioning, self-tested software install in a total time (including unpacking the box) of less than 20 minutes per machine. The installer uses yum to resolve any missing package dependencies, and downloading all current OS updates is inherent into the process.

    So, I point the finger at SAP. Where do they get off not having a decent installer? 2 FRICKKEN WEEKS to come up with a working system?

    "Software updates had to be manually installed to ensure SAP certification."

    Screw using RPM for individual updates - that's rediculous when you manage a large number of machines. Keeping track of packages installed an dependencies will drive anyone batty - which is why the august powers that be gave us yum (or apt, the Debian equivalent) Why isn't SAP running their own YUM server? That way, "approved" patches can be run with a simple "yum -y update" !??!

    "We asked the customer to do a diagnostic test and the customer never responded, so it was impossible for us to address the issue,"

    Hmmm... now the story begins to make some kinda sense. Something is very Very VERY wrong here..
  • by al_broccoli ( 909467 ) on Wednesday September 28, 2005 @11:56PM (#13673221)
    SAP products require patch after patch, and take MONTHS to really install. We had a team of engineers working around the clock (literally) for 5 months to get our base systems set up to SAP specs. Even then we would receive "mystery" patches, frequently resulting in system crashes as they weren't designed to work with other patches. Bottom line - SAP is the problem. They churn out highly unstable software and have armies of consultants who will sweep problems like this under the carpet or find something else to blame.
    This is a load of crap. Everyone hires consultants that are idiots, but the interactions you describe with SAP just don't happen. I've been administering SAP systems for 10 years now, and I've never had anything like what you describe.
    They claim support for linux and other non-MS platforms, but that's only for their core products. Everything outside of CRM and R3 is riddled with technotes and disclaimers about needing MSSQL and WINDOWS.
    What a joke. MS SQL/Windows were among the last platforms supported by SAP. In all my years of supporting SAP systems, I have NEVER run across a note saying that something was only supported on SQL Server/Windows.
    SAP has a HORRIBLE track record on linux.
    Bottom line, I've been running SAP products on Linux for over 3 years now, with not a single complaint. You obviously don't know jack about what you're talking about.
  • by croddy ( 659025 ) on Wednesday September 28, 2005 @11:56PM (#13673222)
    the problem today with gazillions of copies of the same library isn't that they waste disk space -- it's that they each present an independent pathway for security failure.

    for example, if you only have one copy of zlib on your system, and it's managed by the OS vendor (up2date, apt, or similar), then you only have one copy of zlib that can be exploited, and you only have to worry about applying your vendor's updates to keep all of your zlib activity patched.

    if you have 80 copies of zlib, each one shipped by a different application that uses the library, you've got a frigging mess on your hands, and you've probably got no hope of patching them all if there's a security bug.

    what we need is more centralization of libraries, not the wild-west free-for-all that would result from what you're advocating.

  • by NutscrapeSucks ( 446616 ) on Thursday September 29, 2005 @12:03AM (#13673254)
    Whatever. FreeBSD has absolutely zero support from enterprise vendors like SAP. The GP was "Funny", you are just "Offtopic".
  • by ThaFooz ( 900535 ) on Thursday September 29, 2005 @12:04AM (#13673262)
    To say a great sys-admin shouldn't know a platform, only concepts AND be able to work with whatever you hand them is ridiculous...It's like knowing the concepts in OO programming is much more important then knowing a particular OO language. However you can't hand someone a task to do in a specific OO language without some training prior in that particular language (whether it be self-taught for a few days or a course for a few weeks).

    Well of course they might need a couple days or even a week or two of studying/training. But when you're talking about a new hire, or rolling out a new system/archtecture/whatever, isn't that always involved? The training period is very likely insignificant compared to the life of the project... if my next project required Ruby on Rails instead of my more familiar PHP and JSP, you could bet that the bulk of my time wouldn't be spent in the Ruby manual.
  • by JPriest ( 547211 ) on Thursday September 29, 2005 @12:05AM (#13673267) Homepage
    If Red Hat does not work try SUSE, if SUSE does not work, try Slackware, if Slackware does not work, try Debian, if Debian does not work try Red Hat.

    Also, if you notice that everyone is like "well they are using SAP, so they should be using Windows duh!" But if someone posted an "Ask Slashdot" and said "Were is installing a SAP solution, do we pick Linux or Windows". They would probably laughed at for even considering Windows.

  • by bedroll ( 806612 ) on Thursday September 29, 2005 @12:06AM (#13673274) Journal
    Perhaps you're one of the few decent administrators that runs an all MS network. The FUD is slung both ways, in large part because no one wants to blame the administration. Everyone wants to think that the OS is the end of line when it comes to reliability and productivity. Obviously you have to figure in hardware, third-party software, and, most importantly, administration.

    *nix usually gets a better reputation because corporations haven't had much opportunity to hire the off-the-street administrator with a degree in law and a certificate saying they can setup a server. That's changing and, as such, you'll start to hear more and more stories about *nix migrations gone bad and the like.

    Of course, the major difference is that MS is just now learning to try and lock down their machines by default and force the user to unlock what they want to use. This makes the bad Windows admin have a higher likelihood of failure because they start with a bad setup and have try to fix things, instead of starting with good setup and trying to make things work with it.

  • by uberdave ( 526529 ) on Thursday September 29, 2005 @12:23AM (#13673351) Homepage
    Ahorton: "Bill, They're asking us to run this diagnostic test."

    BillG: "For heaven's sake don't run it. We don't want them to know we've deleted /usr folder in order to force it to fail. Remember this Tony, if Windows doesn't come up smelling like roses, I'll have to send some of my boys over to 'buy you out'."
  • by YrWrstNtmr ( 564987 ) on Thursday September 29, 2005 @12:26AM (#13673361)
    Do you observe that lately if someone puts Windows instead of Linux is news.

    It's 'news' only in the navel gazing world of /. Meanwhile, the rest of the world just keeps on doing business.

  • by ekimminau ( 775300 ) <eak@kimminau.org> on Thursday September 29, 2005 @12:34AM (#13673388) Homepage Journal
    And... you get what you pay for. Just because a Windows admin is cheaper doesnt mean he is better or worse than the higher paid Linux admin however I would put the network troubleshooting skills of an "average" Linux admin vs. and "average" windows admin on any day and Im willing to bet the Linux admin would run circles around the Windows guy. In my 15 years of experience guys who only know windows normally are a waste of breath and most *nix admins exceeded the expectations of the windows support community and moved into something better. MY .02.
  • X11 (Score:3, Insightful)

    by Craig Ringer ( 302899 ) on Thursday September 29, 2005 @12:43AM (#13673419) Homepage Journal
    "Most problems with Linux on the desktop are problems with X."

    I couldn't disagree more. There are usability issues, documentation problems, missing features, etc. None of this is caused by X. I have seen _zero_ evidence that X11 is in any way a problem. The protocol is great, and I think we'd be nuts to ditch such a powerful, network transparent facility. As a developer, I'm not fond of the Xlib APIs, but there's work to replace Xlib now. The XFree86/XOrg implementation of the server could be better built so that it was in many small parts - but that's only a problem for people doing lots of low-level distro hacking, and for distributors. Again, there's progress to modularize it anyway.

    X11 is not slow. Some X11 drivers are slow, but that's a driver issue and changing the window system will still leave you with crap drivers. For that, you need people who really understand the guts of the hardware, and you need good documentation. I should note that my system is *extremely* snappy under X11. In general, I find decent ATi and NVidia cards get very good results. If you're talking about 3D, that's in my view quite separate - but again, comes down to driver support and no documentation from vendors.

    Nothing in X11 makes apps that use X11 ugly. Seriously. It's *WAY* too low level. Your complaint is most likely with the toolkits, themes, etc. If not, I'd be interested to know what in X11 you think causes the problem.

    I'll certainly give you the points on X11 configuration and maintainance. I personally find it pretty painless, but then I have good hardware. I also find X11 to be very stable, though there have been times in the past I've sworn rather loudly about it (usually due to bad drivers or hardware).

    The VT system could work a lot better, and I'm looking forward with enthusiasm to the move of much of the frame-buffer programming back to the kernel where it belongs. That should help solve a number of irritations.

    I suspect you may have hit the reliability nail on the head if you're talking about rebuilding Xorg/XFree86, fontconfig, etc. If not done very carefully and with a good knowledge of the system, you'll quite possibly break things here. In particular, you need to be 100% sure that your new versions are ABI-compatible, unless you isolate them and only use apps you built against them with them. Your comment suggests that you do not, since Fontconfig has nothing to do with font rendering, and if there's anything you should be rebuilding (but you don't actually generally need to) it's freetype.

    Of course, I find I get extremely good quality fonts anyway, so I can't say I've ever felt the need. Fonts under Linux used to be horrific - eye searing examples of pure horror. This has, in my view, been entirely resolved by recent freetype libraries and the ditching of X core fonts in favour of client-side rendering.

    I personally find X11 one of the most attractive things about Linux. There are some issues with the implementation, but the power and flexibility of the protocol is not something I'd want to give up. I do agree that it could use some more work, but I'm unwilling to whine about it when I lack the time, skills, and motivation to do it myself. I personally think the current X work is important, and it looks like it'll lead toward more radical enhancements once the more basic issues with the codebase are addressed.
  • by screwthemoderators ( 590476 ) on Thursday September 29, 2005 @12:50AM (#13673452) Journal
    To say that because you've never had problems, the whole "Windows crashes is FUD," is really quite arrogant. Windows certainly crashes a lot, although more often on the client side, where the user can do more damage. There is also the problem of diagnosing crashes once they happen. I've found it much easier to diagnose *nix crashes than the infamous BSofD. There's also a question of motive. M$ has millions, perhaps billions to lose if they don't sell. There's little profit to be made spreading FUD about Windows, while proprietary software companies do have a lot of money to lose to *nix users. Even companies that make money from linux are always vulnerable to a customer switching to another distribution. If you follow the money, it doesn't lead back to linux. FUD is pretty specifically a corporate strategy.
  • by Anonymous Writer ( 746272 ) on Thursday September 29, 2005 @12:51AM (#13673457)

    To me, the interesting thing about this is that it's newsworthy enough to publish on TheAge.com

    It isn't exactly a publication with IT personnel as a target demographic either, but rather laymen in the general population. Perhaps the $100 Million marketing campaign [slashdot.org] has already begun.

  • by benjamindees ( 441808 ) on Thursday September 29, 2005 @12:59AM (#13673501) Homepage
    I *know* this guy is not an admin. He is MIS, at best. *Huge* difference. This guy gets paid to write reports and macros for applications for whatever software this businesses uses, clearly SAP, not to install or administer servers.

    I mean, just listen to him. He outsources everything. He seriously believes all operating systems are the same. He complains about having to spend two days a month updating and testing. Then he goes on to include this work in an increased "total cost of ownership" for Linux, completely ignoring the fact that it's his job and he's being paid whether he does it or not. He doesn't know the difference between an application failure (core dump) and an OS failure (panic/oops). And, to top it all off, he thinks autopatching is a great feature.

    Lots of "small" (multi-million dollar) businesses make the mistake this one has: they think they can get away with having just one "admin" who is really MIS, who spends all of his time dealing with the business side of things rather than the computing side. To maintain the illusion that this is a workable combination, they switch everything to Windows and spend almost as much on licensing and consultants as they would on a competent admin. Then they wonder why their customers' credit card numbers mysteriously show up on the 'net.

    News flash to all the "small" businesses out there: well-maintained computer infrastructure can replace 50% of your employees. Skimping on IT personnel is a stupid, stupid mistake. You can afford to have *both* a proper IT guy and a report-writing business grad. Despite their misleading marketing, Microsoft software is not a substitute for a qualified admin.
  • by Anonymous Coward on Thursday September 29, 2005 @01:16AM (#13673566)
    It sounds like you think Google buys a few hundred thousand computers, loads up the latest kernel, and just starts running! If they did that, they would probably stay up for less than a minute. Google is not running IBM hardware or RH Enterprise (and obviously not SAP).

    No, Google uses custom hardware, custom kernels, custom drivers, and custom everything else (web servers, filesystems, etc). They have gone through immense pain fixing unreliability at heavy loads for all of those things. You might think that ext2 or a driver for your favorite hardware is rock solid, but that's only because you haven't run Google on it.

    The only way Google gets scalability from Linux is to use lots of cheap hardware; each computer has a single CPU, cheap RAM and a couple hard drives. Each machine does the same job as dozens of other machines, so that when one falls over it is hardly missed. Google can afford to do all of this because that is their job. Crest's job is selling electronics, not supporting their ERP system.

    Of course all of this customization is possible because they have the source to all of the software they run. On the other hand, I suspect that MS would give them a source license if they had 100,000 Windows servers. FYI, I believe that MSN has on the order of 30,000 Windows servers.

    dom
  • by TheLink ( 130905 ) on Thursday September 29, 2005 @01:23AM (#13673596) Journal
    Nah. Doesn't have to be a hardware problem.

    More than a few Linux kernels have had some memory management issues. If he was using RedHat 9 he'd be having the same problems we had - had to reboot every few days.

    Just do a google search on kswapd and cpu for some examples. If you bother to look around I'm sure you can find other stability problems with Linux.

    I use FreeBSD, SuSE Linux and Windows 2000 at home. They all have their uses. They have their strengths and weaknesses.

    Unlike what the fanatics believe, Linux isn't that much better than Windows. Even in terms of security and stability.

    That said, I'd still prefer to use FreeBSD/Linux for most server stuff.
  • by techno-vampire ( 666512 ) on Thursday September 29, 2005 @01:36AM (#13673663) Homepage
    So, what you're saying is that the people at your customer's site would rather work 12-16 hour days for an indefinite length of time than spend 10 minutes getting you the information you need to get the problem solved right away? No wonder they're getting yelled at by their managers.
  • by harves ( 122617 ) on Thursday September 29, 2005 @01:36AM (#13673664)
    Sorry but that's one of the dumbest things I've read in a while.

    You're noting that the name "Linux" covers a broad range of things, and comparing it to the name "FreeBSD" which refers to one thing. You're then trying to say that "the BSDs don't have the fragmentation that Linux has". I call bullshit. Your example proves nothing remotely near that. It proves that FreeBSD isn't fragment, but then neither is the Debian project's distribution.

    If I say "I run BSD" then there at least 3 different systems I could be running. Would you then say that "the BSDs have fragmentation just like Linux does"?

    Inversely, if I say "I run Debian" then "I qualify my package management, my system boot scripts, where my conf files are, how the system works".

    Sorry, I'm not normally this harsh, but what was your point again? If you try to compare Linux to FreeBSD then yes Linux will appear more fragmented. But how about we compare FreeBSD to Debian shall we? Apples to apples? Does your argument that it "damages" Debian still hold?
  • Re:Two Weeks! (Score:5, Insightful)

    by Malor ( 3658 ) on Thursday September 29, 2005 @01:36AM (#13673665) Journal
    At one job, a few years ago, I installed a small, simple SAP program, SAPRouter. It was basically a program that would route net connections over a modem into a foreign network. I don't remember the details very well, because it has been six or seven years, but some of the stuff I definitely do recall. My memory of cursing, intensely, for DAYS is clear and bright. SAPRouter was among the stupidest pieces of software I've ever been forced to work with.

    It was just bizarre. Out in left field.... way, way out. They implemented an entire routing protocol, kind of like IP, but very poorly. It was completely unrelated to any other form of routing I've seen.

    From what I remember, you had to install the router software on a PC that had a modem. That was going to do the call out. (VPN wasn't common at the time, you had to use a modem for a network backdoor.) But then you had to configure the client to talk to that PC over the network... and you also, if I remember correctly, had to tell it about every hop it had to take in the foreign system.

    In other words, it would be like having to manually configure your PC with every hop between you and Slashdot before you could read web pages. And if one of the hops changed, well, too bad. No Web for you.

    There was more, too, lots more, but I have lost the details. All I remember is that it was problem after problem after problem for DAYS. And this is relatively simple software.

    The documentation was horrible too. It made no sense at all. (which shouldn't be that surprising, really, since the program made no sense either.) SAP was kind of bleeding edge in one regard, and provided fairly complete Web documentation. Sadly, instant access helped clarity not a whit. I ended up taking three or four days and making repeated calls to SAP to get the stupid thing working. It felt like I was trying to push my head through a cheese grater. I'm not an idiot... I was learning IP routing at the time, and I can assure you, it was _trivial_ in comparison.

    In some ways(the bad ones), SAPRouter reminded me of learning Netware for the first time. Netware was full of weirdnesses that didn't make sense at first. But after you'd been working with a given feature for awhile, nearly always there was an 'aha!' Netware had a payoff for the struggle... you'd finally see why they had modeled a given problem the way they had, and it was inevitably elegant, powerful, and aesthetic all at once. It was hard to figure out their context, but once you did, their solutions made beautiful sense. They thought out problems incredibly thoroughly, and solved them completely.

    SAPRouter wasn't like that. It felt like, well... like a bureaucracy that's very sure of its own brilliance. They reimplemented, badly, what IP was already doing. It was grossly inferior, complex when it didn't need to be. Once I understood their context, and why they solved the problem how they did, my conclusion was that they were idiots. It felt like something designed by people who had *no idea* what routing is or how it should work.

    To be fair, it was nicely stable once it was up. I didn't have to fool with it anymore after it was (finally) running.

    Basically.... don't be so serenely certain these admins are idiots. The reason you're good at figuring this stuff out is because smarter people than you (or me) took the time to make it (relatively) easy. They chose good models and clean implementations, so the programs are fairly easy to configure and use. You being good at building solutions from open source stuff is partially your brainpower, but the lion's share of the credit goes to the original designers. You had an easy time of it because, for the most part, the software is fairly easy. It could have been far, far worse.

    It could have been SAP.
  • by webengr ( 301595 ) on Thursday September 29, 2005 @01:56AM (#13673740)
    Okay, so I see that I missed that it took two days to have "SAP" up on Windows, not a couple of hours. My bad.

    I stick by my statements about the lack of specific information in this "story." What exactly was Crest trying to get running on a single host (reference to "the machine")? An SAP implementation landscape typically spans many hosts, and it can be a heterogenious environment; in fact, up until recent versions there were a few Win32-only components, such as the IGS server and the ITS web-enablement middleware, so heterogenious SAP environments are quite common.
  • by syousef ( 465911 ) on Thursday September 29, 2005 @02:00AM (#13673760) Journal
    This argument is brilliant.

    User A: I used SAP and had lots of problems and it didn't work and the consultants took lots of money and re-engineered everything around their system. SAP is always crap.
    User B: I've used SAP for years and had no problems. You must be the problem. Never mind that I know nothing about your situation or your dealings with SAP I'm going to call you a liar and say SAP is wonderful.

    Neither of you are being reasonable, but man, pass the popcorn! This is entertaining! Just like Jerry Springer.
  • by JourneyExpertApe ( 906162 ) on Thursday September 29, 2005 @02:11AM (#13673788)
    "This whole "Windows just crashes all the time" stuff, especially on the server side, is pretty much FUD."

    I wouldn't call it FUD; I'd call it outdated information. Remember Windows 95/98/Me? When they created Windows Me, they had hundreds of billions of dollars and thousands of employees, and they still got it wrong. Very wrong. I've switched my home PC from Me to XP recently, and the contrast is striking. It'd say it's understandable if many Linux users who don't have experience with the latest Microsoft OS are still wary of using microsoft products.

    "Flaky commercial third-party apps can gum up any OS."

    So you think it's OK for bad software to screw up an operating system?
  • by B747SP ( 179471 ) <slashdot@selfabusedelephant.com> on Thursday September 29, 2005 @02:20AM (#13673816)
    Why should a single buggy application bring down the whole server?

    Actually, my impression is that it's not SAP bringing down the server, its red hat enterprise. I had this conversation with a couple of friends yesterday (right after I submitted the same story to slashdot, bah!).

    One of them runs a massive application - gigabytes of traffic served per hour at certain times of the year. She commented thus: "xxx and I NEVER had any stability issues with any of the xxx of xxxxx servers until we moved them to Red hat Enterprise. Since then we've had two kernel panics......."

    'course, what is arguably more interesting about this Crest Electronics [crest.com.au] situation is the reasons that the IT Manager cites for changing. They just seem full of holes to me. Reading between the lines, I reckon this guy came in, didn't like the Linux install, and wanted an excuse to move back to his beloved Microsoft. And who in their right minds lets any mission critical server auto-patch itself, regardless of operating system. That's just utter madness!

    My other friend (yes, I have two!) put it best I think, when he said "I hope the guy got a major payout from Microsoft, because such a public display of incompetence makes him unemployable.".

  • by drsmithy ( 35869 ) <drsmithy@nOSPAm.gmail.com> on Thursday September 29, 2005 @02:47AM (#13673896)
    Reading between the lines, I reckon this guy came in, didn't like the Linux install, and wanted an excuse to move back to his beloved Microsoft.
    "Having previously run SAP on AIX - IBM's version of Unix - Horton was comfortable with deploying such a mission-critical application on Linux."

    Yep, he sure sounds like someone who would go running back to "beloved Microsoft".

    And who in their right minds lets any mission critical server auto-patch itself, regardless of operating system. That's just utter madness!

    No, it's efficiency and good systems management.

    Of course, what they mean here by "automatic updates" are updates distributed from an internal updates service (WSUS) after being approved, not "automatic updates" from windowsupdate.com.

    My other friend (yes, I have two!) put it best I think, when he said "I hope the guy got a major payout from Microsoft, because such a public display of incompetence makes him unemployable.".

    The numbers say he's saved his company money and made their systems more reliable. That usually makes you *more* employable, not less - at least with the people who actually do the hiring that don't care about Operating System holy wars, at any rate.

  • by Anonymous Coward on Thursday September 29, 2005 @02:47AM (#13673898)
    Case in point: I had a P3-800 with 3 256MB DIMMs. One was bad. I had Windows98 on it for years and it ran as well as expected.

    That might have something to do with Windows 98 only supporting 512 MB RAM, and the bad one being the one that it didn't use...
  • by shortscruffydave ( 638529 ) on Thursday September 29, 2005 @02:53AM (#13673908)

    I've seen systems get slow on Windows machines over and over because of memory leaks

    Hmmmm....so where does the problem actually lie - the operating system or the apps? I was having a conversation with a non-very-technically-minded friend a while ago who was saying how often "Windows crashed" when what he actually meant was that he was running a piece of badly written shareware which was throwing an exception which was being caught and reported by Windows.

    Try telling Micrsoft to fix the memory leak in IIS

    Valid point - there may be a memory leak in IIS, but that's not a "Windows" problem, although it does come from the same vendor. If you replaced IIS with Apache would that make Windows itself more stable.

  • by PitaBred ( 632671 ) <slashdot&pitabred,dyndns,org> on Thursday September 29, 2005 @03:10AM (#13673958) Homepage
    Yes, throw random bodies, rather than someone who has specific experience. I know many things my co-workers don't, and they likewise. When we get a tech support question, it goes to the main list and then whoever is most qualified answers it. Or digs for more info.
    I wouldn't put it past RedHat to have some kind of filtering like that for their muckity-muck engineers so that they make sure they send the right guy for the job.
    But if you ask for help, someone asks for clarification or a bit more info THAT THEY DON'T HAVE TO BE ON SITE FOR, it's your fault if you refuse their assistance.
  • by Greyfox ( 87712 ) on Thursday September 29, 2005 @03:18AM (#13673971) Homepage Journal
    Admins are cheap. Admins who can get a dead system up and running with an angry customer, manager or both breathing down their necks are a lot more expensive. You can take any will work for food guy off the street, give him a cheeseburger and show him how to install your operating system. If that's the kind of guy you want to trust your company to, more power to you. Chance are he wrote his will work for food sign on the back of his MCSE certificate.

    Problem of course is that most hiring managers can't tell the difference between the will work for food guy and the guy who can actually save your company when its systems are down and millions of dollars are on the line.

  • by OwlWhacker ( 758974 ) on Thursday September 29, 2005 @03:32AM (#13674015) Journal
    If you have an MCSE certification it doesn't automatically make you competent to administer a Windows network.

    I've had MCSEs call on me for help with simple networking problems.

    I find that many qualified people just forget what they've learned. I even have the same people calling me up every once in a while, with the same questions, purely because they keep making the same mistakes.

    It may just be coincidence, but, I find that the most incompetent MCSEs are those who go out of their way to tell you they're an MCSE. They seem to use it as an excuse for their incompetence - like saying "Well, I was smart once!"

    : )
  • Re:Two Weeks! (Score:1, Insightful)

    by Anonymous Coward on Thursday September 29, 2005 @03:33AM (#13674017)
    You missed the point, the SAP installation on Red Hat took two weeks, not the installation of Red Hat
  • by Jon Peterson ( 1443 ) <jon@@@snowdrift...org> on Thursday September 29, 2005 @04:04AM (#13674098) Homepage
    "Hello, Vendor. We've spent several man months trying to get your software to work, but we still get random lock-ups. What do you suggest?"

    "Hi, customer. Could you please spend several man-weeks running these complex diagnostic tests on our software, so we can try to fix it. Yes - very similar to the last set of diagnostic tests - yes, the ones that didn't help us diagnose the problem - yup, like the ones before that too, but this time with a few different settings - yes, please, if you could send us all that data then we'll have a bit more time to think up some excuses. Thanks!"

    Now, I've no idea what the particulars here are, but I've been in plenty of situations where it's a waste of time sending vendor support yet more giant error logs, and running yet more diagnostic tests. I used to regularly 'help' Vignette try to fix our problems, but they never did. I could have spend the time better doing our own workarounds - or switching CMS vendors, which is what we ended up doing.
  • by Heembo ( 916647 ) on Thursday September 29, 2005 @04:51AM (#13674207) Journal
    Why does your linux server ever need to be under "heavy load"? Aren't Linux boxes supposed to be cheap to set up so you can have yourself a nice Linux cluster and balance the load so you are never running above 50% or something like that? If you buy a cheap server - and run it near 100% all the time - then you deserve to have problems! Dude. Upgrade your hardware.
  • by NickFortune ( 613926 ) on Thursday September 29, 2005 @04:59AM (#13674228) Homepage Journal
    The big difference being that you are not tied to your distro for support. If RedHat grow to the point where they become unresponsive, you can always ditch them in favour of a smaller third party support outfit.

    Hell, if you have the expertese, you can even fix it in-house.

    Try doing either of those with Windows :)

  • by rikkards ( 98006 ) on Thursday September 29, 2005 @05:24AM (#13674303) Journal
    If you replaced IIS with Apache would that make Windows itself more stable.

    Apache on windows stable? you are funny! Anyways I digress. You make a good point where the issue may not necessarily be the OS but applications on it. However since MS has interwoven IE so deep into the OS that it is not easy to remove without third party tools that it is difficult to distinguish one from the other
  • by Darkling-MHCN ( 222524 ) on Thursday September 29, 2005 @05:36AM (#13674336)
    I'm definitely a windows boffin but have tinkered with Linux. My experience with both are the same, the kernel's are rock solid in both products these days and with the RIGHT device drivers. The only time you see kernel level crashes is when there are hardware issues usually as a result of buggy device drivers, or faulty hardware.

    The thing I find with linux is that you invariably find hardware vendors drag their feet on the linux drivers as it's far more important to get the windows drivers to market (due to the market's size). I'm no expert but I have found unless your machine's config is pretty vanilla Linux can be really hard to work with. Rate me a troll, maybe I don't know what I'm talking about, but I just find windows with it's hardware auto-detection and out of the box support really kicks ass over linux.

    Of course these problems aren't an issue with Apple and OS X as things are shipped as one complete package ready to work. If they wanted unix, maybe they should have gone apple....
  • It's a problem (Score:3, Insightful)

    by Apreche ( 239272 ) on Thursday September 29, 2005 @05:59AM (#13674388) Homepage Journal
    This is a problem I have noticed with Linux many times. On the whole, Linux is incredibly rock solid. But there are rare instances where specific combinations of Linux software and hardware will cause crazy problems. For example, at one time there was a problem with APIC in the kernel. If you had an nforce2 motherboard and a kernel with APIC enabled it would freeze up semi-randomly. 99% of people did not have this combination, so it wasn't a problem for them. But for the 1% of people who did, how were they supposed to figure it out? Only if they are very involved in a Linux community would they discover this.

    Another problem I had was with the combination of Ubuntu, Nforce2 IGP, the NVidia driver and not having DDR Dual-Channel enabled. This combination brought about frequent freezing. But who could know without good googling skills that this combination was the cause of the freezing?

    I'm willing to bet that this guy had one of these weird combinatory problems. It just goes to show that the Linux testing procedure is not 100%. But switching to Windows when this happens is basically just claiming ignorance instead of figuring out why it's crashing and fixing it.
  • by Anonymous Coward on Thursday September 29, 2005 @07:19AM (#13674591)
    But really, XP stablity isn't all it's cracked up to be. I have to reboot often (~once a week). Things just slow down and get really sluggish after ~ 2 weeks, or less.

    I don't know what you are running on your PC, but if you were running a server I hope you wouldnt have all that junk installed on it. I've had Windows NT servers which could have had over a year of uptime if it werent for doing patch maintenence. And it's only gotten better since Win 2000 Server. With Win2k3, I don't even need to reboot for patching except for a very few cases.

    Support is the hidden cost- that's exactly what people in the know have been saying about Linux for years already.

  • by NineNine ( 235196 ) on Thursday September 29, 2005 @07:26AM (#13674615)
    So then, because this company's IT philosophy differs from your your personal, religious philosophy about operating systems on personal computers, they must be either lying or incompetent? Boy, that's open minded.
  • Why MS won out.. (Score:2, Insightful)

    by Bigmilt8 ( 843256 ) on Thursday September 29, 2005 @07:33AM (#13674635)
    I understand the love for Linux (or rather hate of MS). To each his own. However, in this situation it is a common problem that companies are having with Linux. 2 weeks to configure and deploy. Come on, that's ridiculous. This guy had a business need and needed to get the job done. If you can't understand that, then you'll be unemployed 6 months after convincing management to make the switch to linux and you still haven't gotten the entire enterprise up and running. Also, and read this slowly so that you don't pass out: LINUX CRASHED.. Now, the hardware was blamed and then the administrator. What's next, code fairies???
  • by NineNine ( 235196 ) on Thursday September 29, 2005 @07:34AM (#13674643)
    The customer wasn't interested in helping the support engineers do their own jobs. I've seen this situation many times before. I'm sure that they had been working with these support engineers for a while, were getting nowhere, and this situation was the last of many where the engineers would request their internal IT department to spend a lot of time getting the support guys data, and the support guys would still shrug their shoulders. Makes total sense to me. After a point, I would've done the same. At some point, you have to move on with your business, and stop troubleshooting something that obviously isn't working.
  • by Dan_Bercell ( 826965 ) on Thursday September 29, 2005 @08:16AM (#13674837)
    Reboot once a week??? Oh no, an entire 1 min wasted! Linux is similar to Windows, both are stable when first installed, but after installing additional apps, most are free and downloaded from the Internet, either machines will slow down, often will crash. Windows has to put up with Spyware/Adaware, which in my experience can totally f**k a machine up, while Linux deals with alot of open source softwars, where the 'frequently' released updates of its packages can result in odd behaviours.. basically your a beta tester the majority of the time. I know people who have Windows XP installed and it runs perfect since the day they got it, mind you they have clean systems and just use it to type/chat/print pictures. Most IT people will mess up any system within a matter of months simply because we have the need to install everything that looks cool. Norton Ghost is your friend ;)
  • by NickFortune ( 613926 ) on Thursday September 29, 2005 @08:40AM (#13674940) Homepage Journal
    what's the point of replying to this poor guys story by saying that, given exper tise, he cld fix memorry leaks and instabilities in house

    ... or pay a third party support firm to do it for him. That's what I actually said.

    The point being that these are options that you just do not have under Windows. Run Windows and you're tied to whatever support MS deign to give you. Typically, that's not a lot. With Linux you can hire anyone to fix the problem. Or take out a support contract; or put up a bounty.

    The important point is this: you have more support options under Linux than you do under Windows.

    And yes, these guys clearly don't want to do that. Not in house and not third party. That's cool. It's not a strategy I would recommend myself; I've had windows boxes that couldn't even apsire to that sort of uptime. But the people have to make the choice they think best.

    Regardless of which, you still have more support options with a free software installation than you do with a proprietory one. That's always going to be true.

  • by Anonymous Coward on Thursday September 29, 2005 @09:24AM (#13675228)
    > Flaky commercial third-party apps can gum up any OS.

    Only if you are dumb enough to run it as root/administrator.

    If it's run as a normal user, and your OS still gets gummed up, your OS sucks.
  • by Anonymous Coward on Thursday September 29, 2005 @11:14AM (#13676247)

    "I hope the guy got a major payout from Microsoft, because such a public display of incompetence makes him unemployable."


    Your 'other' friend is right. If I had one of my IT staff that was having problems with a server do this he'd be out on his incompetent a$$ in about 10 seconds.

    There's clearly no attempt to address the real issue here, no attempt to properly debug the issue, and no talk about what the problem really turned out to be, how SAP or RedHat failed to deliver a solution to the problem thereby justifying their fallback to Widows. Just the classic 'I can't figure it out so I'm going back into my comfort zone with Windows' excuse.

    A more public display of incompetence is difficult to imagine. Luckily most HR managers don't read Slashdot or this guy/gal would be on foodstamps.

    Rule #1 know and undrstand the 'real' problem. Not the perceived problem.. you must identify and understand the REAL, ACTUAL cause of the instability. Only then can you address it.

    ElR
  • by incom ( 570967 ) on Thursday September 29, 2005 @11:28AM (#13676414)
    Yes, I'd like to know how this became a news story too, because I don't see any natural reason for it to be one, somebody had to have been proactive on this one, but for what reasons?
  • by Grab ( 126025 ) on Thursday September 29, 2005 @12:35PM (#13677142) Homepage
    Oh no, an entire 1 min wasted

    The cry of the man who only uses Linux on a desktop and has never heard the word "server".

    Talk to your IT person. Suggest that a server crashing once a week (at random times) and losing people's work when it happens is merely a case of "1 min wasted". Then watch them laugh in your face. Alternatively, wise up...

    Grab.
  • Comment removed (Score:3, Insightful)

    by account_deleted ( 4530225 ) on Thursday September 29, 2005 @12:48PM (#13677277)
    Comment removed based on user account deletion

HELP!!!! I'm being held prisoner in /usr/games/lib!

Working...