Managing Linux and Virtual Machines? 239
deijmaster asks: "For a couple of months we have been hearing (as a major consulting firm) IBM people pushing the possibility of installing a Z/Linux VM setup at one of our biggest clients (financial). To a Linux user such as myself this sounds great, at first. Now, I am a bit reluctant when it comes to managing this kind of infrastructure, with little or no local expertise at IBM. Has anyone gone through a Z/Linux VM corporate installation and lived through the management of such a solution?"
Doesn't seem like such a big deal (Score:-1, Informative)
The emulation steals the speed increase and abstracts the Penguin Power from the user's environment.
IANAIBMSA(I A Not An International Business Machines System Administrator), but it seems like the only advantage to this is that you can get one hell of a UT/Q3 framerate while the virtual machine is shut down because "somebody was calculating pi on the server" or "Somebody ran vmunix through a spelling checker."
BOFH excuses 16 and 452, respectively.
Re:Doesn't seem like such a big deal (Score:5, Informative)
Linux/390 resources (Score:5, Informative)
Oh, and the Marist linux-390 listserver [marist.edu] is well worth subscribing to.
I would advise against it (Score:4, Informative)
1. What exactly demands this solution?
2. Can a pure Linux box, with mild tweaking, still not be more useful and create less overhead than this?
Someone in this thread mentioned IBM implementing wildly complex systems in order to push consultation, and on some levels it's true. PeopleSoft does it also. In some cases, Oracle will have a go at this tactic. My advice is to do some searching first, without the input of IBM, and see if you can't find a better solution to whatever problem you're trying to remedy.
not cost efffective (Score:5, Informative)
1. for the equivalent # of VM's it was more cost effective to buy new Intel hardware. The annual maintaince cost for the IBM more than paid for all new hardware.
2. Software availability. The only thing you could run it would be home grown apps or existing opensource apps. No commercial software was available. This company was an all Oracle shop, no DB2. They're primary opensystems backup solution was Netbackup. Which at the time had no client for linux on Z. (a year ago).
3. In house expertise. They had no linux expertise and very little Unix (solaris & HP) (jr admins at best) expertise. Let alone running linux on a Z.
So to sum it up. It's a very expensive, somewhat propritary and inflexiable environment. If you have a specialized use for it and can justify the cost go for it. Otherwise stick with commodity Intel/AMD hardware. It'll be cheaper and easier in the long run.
Z/Linux at a big financial firm (Score:5, Informative)
With careful planning, and the expectation that it will be a bumpy start, you'' find that it's a very rewarding experience, both personally and professionally.
Re:I would advise against it (Score:3, Informative)
No fears, IBM will take care of you (Score:5, Informative)
If you have problems, contact IBM and they will get their best people on it. IBM is all about customer service. You never get fired for buying IBM. From an engineers perspective, it's a pita. The best people in a department end up spending most of their time working on customer problems.
Hell, IBM still supports OS/2. If a Z-Series seems to solve your problem, go for it. IBM will take care of you.
Linuxcare? (Score:5, Informative)
http://www.linuxcare.com/ [linuxcare.com]
Chrisd
Re:You WILL need help (Score:5, Informative)
Actually VM itself is fairly straight forward to administer
The biggest hurdle will be:
This is also where security comes in. Each SVM is really isolated from each other.
One thing to remember too is that VM was (and still is) used by many Universities and colleges -- not as much as it was back in the 70's and 80's, but it still has a presence.
Anyway... just some comments from an old timer VM sysprog
Re:not cost efffective (Score:5, Informative)
BTW - I've ported a number of programs to Linux/390 (an IBM G6 mainframe) and compared them to Linux on my 1 GHz Athlon cobbled together from left over parts and a motherboard from Fry's. The net result is that the Athlon is about twice as fast as the G6 mainframe.
The latest and greatest mainframes are about twice as fast as a G6, but PCs have come a long way since 1 GHz. Currently, 1 CPU on a mainframe running Linux costs about $100K, you can buy a pretty impressive Intel server for that price.
So, Linux on S/390 is only effective when you have a bunch of machines with utilization close to zero - let's call it "epsilon", which is what we mathematicians say when we really want to say zero but still need to divide by it. You buy the box for VM, which can run hundreds or even thousands of instances, securely and stably, so long as most of them are doing nothing.
Linux/390 is great for experimental servers, test systems, etc. OTOH - if you have any significant workload, buy a rack-mount PC.
Alan.
Re:not cost efffective (Score:5, Informative)
zLinux admins need VM skills (Score:5, Informative)
Familiarity with Linux will not help you setting up the zLinux environment. It works like this: You dedicate a few processors of your mainframe to Linux. These processors will run VM, which has:
The users are defined in a "user directory". There, you can specify how much memory, disk and CPU share you want to give to each user. These users, remember, are in fact virtual machines that will boot an image of Linux compiled for the zSeries processor architecture.
If you want to create and take down Linux images frequenlty, you'll have to install and customize some VM scripts that will do the job for you. When the scripts are installed, you can setup a new Linux image (complete with its own disks, IP address, etc.) with a single operator command.
Most sysadmins of a zLinux machine spend a lot of time in VM. So learning VM is essential if you are going to do this job. VM was created 30 years ago and is somewhat primitive in places, but the resource virtualization mechanism is incredibly powerful and makes up for it.
Finally, make sure that people understand that there might be dozens of virtual CPUs defined under VM but only a few real CPUs. If you have 4 CPUs, a Linux user with an absolute CPU share of 25% will have the equivalent of one CPU. If the Linux image is used for recompiling its kernel, it might be a tad slow. The mainframe has great I/O performance but only run-of-the-mill raw CPU speed.
Good luck.
Re:No. (Score:3, Informative)
Re:not cost efffective (Score:3, Informative)
It is always a better choice to use clusters of hardware, than a single box.
You have a variety of tools available on the open source market now to monitor, and automagically maintain your cluster, depending on what you choose...the most popular is PVM, and it comes with a ton of very nice management utils you can get off the net, too manage hundreds of machines in a blink of an eye. This is a very configurable cluster architecture.
There is also MOSIX, or open Mosix. A very nice computing facility as well, allowing you to use a single image machine from a large collection of machines tied together through a network.
Do a google on PVM or MOSIX.
-Hack
*warning* I work for IBM (Score:5, Informative)
I second the idea that it is very important to have VM skills on site for a customer looking at this. Presumably the customer is already a z/Series account, so they probably already know a thing or two, but they may have bought into the "VM is going away" speech and gotten rid of their VM stuff years ago and gone to z/OS.
Even if they have VM skills from 5-7 years ago - that will still do. VM hasn't changed all that much, it just has some more bells and whistles. So one or two refresher courses for whoever is still around in their shop will get them up to speed on z/VM 4.4 if they knew it 'Back in the day'.
And yes - Linux on VM is still young. Most shops appear to do a lot of 'roll your own' solutions to the administrative problems. Get hooked into the Marist linux390 mailing list, there are a lot of smart folks there who have at least thought about any problem youre likely to have.
I've run/tested every one of SLES 7, 8, RedHat 7.1, 7.2, RHEL3 beta, TurboLinux (old and crusty now) and Debian with pretty much any IBM middleware you could think of. From the linux side - it doesn't know anything about VM, or care. So you as the administrator must make sure it plays politely with the others it lives with. You probably should not just throw 2 Gigabytes of storage at it just because Websphere says it needs it. Running Linux with VM does require some understanding of how to make the most of shared resources. Check out this redbook:
http://publib-b.boulder.ibm.com/Redbooks.nsf/Re
It makes a lot of these points better than I can.
--Anonymous Coward cause I forgot my password
Re:not cost efffective (Score:3, Informative)
It all comes down to crunching the numbers. I think IBM is actually pretty honest about z/Linux, they're not trying to sell it as a supercomputer but rather as a consolidation solutions.
FWIW - I work for a very large company with thousands of servers. We have dozens of them with utilization of approximately zero, including some very large machines (such as an E10K doing the work of a PC). This is not unusual...
Experience with z/Linux and VM (Score:3, Informative)
As with anything, "it depends." In my experience, L/390 under z/VM works best in I/O-intensive heavy-throughput roles. Do not throw CPU-intensive work at it. If you need CPU, either build an Intel farm, or use an architecture that's designed for serious computing, like a pSeries.
From a manageability standpoint, you will be flabbergasted how much easier it is to manage a z/VM box with 100 Linux instances on it than it is to maintain 100 rackmounted x86 boxes. And once you get your legs under you with VM, it's amazing how tunable the system parameters are. FCON/ESA (now Performance Toolkit, in z/VM 4.4) is really, really your friend in terms of determining where the system hotspots are. And once you've tasted how to deploy additional servers in two minutes without leaving your chair, it's really hard to go back to old-skool provisioning.
Adam
Another User-Mode Linux hosting service (Score:2, Informative)
Bytemark Hosting [bytemark-hosting.co.uk] offers Linux virtual machines via User-mode Linux.
Bytemark supports Open Source with contributions to Debian and discounts for Open Source developers.
Debian is one of the distro options. Primary DNS on Bytemark's DNS servers is included (running djbdns [tinydns.org], win win).
Re:Doesn't seem like such a big deal (Score:4, Informative)
Not quite, though the effect is similar. Both VM and Linux are supported by a combination of software and microcode. However, IBM has been very successful in putting almost all performance critical code sections in microcode. Thus, VM and Linux are 'emulated', but the overall performance usually ends up being quite close to what could be achieved if the support was indeed natively supported.
Re:zLinux admins need VM skills (Score:1, Informative)
not completely true (Score:1, Informative)
Re:zLinux admins need VM skills (Score:3, Informative)
Nobody said the contrary.
Don't do number crunching on zSeries, you fool! (Score:4, Informative)
Covener, you're right. zSeries suck as number crunchers. They are great at intensive I/O jobs. They are great at consolidating servers that aren't all busy at the same time. But "brute-force an RSA key" is exactly what you don't want to spend your expensive MIPS for.
BTW, I found that on a web server mettle test, large file transfer performance was better on zSeries than on RISC boxes. The larger the files, the more advantage to the mainframe. This is an interesting side-effect of having processors dedicated to I/O and freaking huge I/O bandwidth.
Re:You WILL need help (Score:5, Informative)
Actually not quite true (that the VM person won't understand or like linux).
Linux has actually given VM a shot in the arm. VMers are well aware of that fact. I support a couple of VM systems during the day and play around with Linux at home at night.
Also in terms of culture, I think you would be very surprised with the VM culture. VM spent many years as the unwanted child, VMers had to rely on each other in order to be heard above the MVS roar.
If you take a look at some of the history of the internet you will find VM sitting there (BITNET was basically a collection of VM systems). The listserv concept was originally from VM (CERN was -- might still be -- a big VM site).
If you want to see some of the history of VM you can start here: http://pucc.princeton.edu/~melinda [princeton.edu]
Re:I would advise against it (Score:3, Informative)
you have to use their hardware despite the fact you can use hardware from Comp USA. The VPN prevents this.
the buss sits idle a large percentage of the time, even when under heavy load. This can be upto 90% idle. This is due to their token ring-like layout of the procs and buss. If you blow a proc which happen quite often, you are SOL until they can ship one in from Austin, TX or Hitachi, japan. When I was there the 9-11 disaster literally shut-dwon the project because of the mail/shipping stoppage.
The RIO buss isn't as bullet proof as they would make you believe. "SLIC" has lots of microcode problems. (SLIC is between the csp and the OS). More then 50% of their cables are defective from the manufacturer, and they are expensive.
You can't crack the case to fix something. You have to let an IBM field engineer do it. This means more down time. And the whole box is down. Not just one little machine in the cluster.
The failure rate of the backplanes in the CEC is over 90%. There is only one place the manufactures them and they are backlogged for months.
DASDs (harddrives) are small in storage(usually 9-18GB) but very pricey and you need 79 of them to fill one 5077 storage tower(required to even boot).
manistore dump has lots of issues.
The power requirements for an H class Z series machine are two, count them two 440V 60 amp services. Plus you have to have the floor strengthened for the extra weight. This doesn't even include the space/power/weight considerations for the storage towers or I/O towers.
I could go on and on. I am not saying they are bad machines. But you better be willing to make a 30 year commitment to the tech to make it pay.
I don't think your finace people are willing to accept downtime. That means lost money. So I would suggest the clusters. The ability to fix stuff inhouse and replace or remove each node makes it very robust.
Re:not cost efffective (Score:4, Informative)
A mainframe is not a supercomputer - the former is build for I/O, the latter for processing speed. There's a little overlap, but they address different markets - the workload for commercial data processing (for example, updating the accounts of your 10M customers with 100M transactions a day) and scientific number crunching (say, modelling a weather front). An IBM mainframe is designed from the ground up for I/O speed, and has dedicated processors that do nothing but shuffle data from storage to memory and back again. You cannot buy an Intel based system that even comes close to a Z series for I/O at any price. Even clustering Intel boxes won't help, because at some point, you have to have something to manage and consolidate all that data so you can do useful things with it. If you need it, there simply is no substitute for a mainframe. But if you don't, I agree that a mainframe is overpriced, just like a tank is overpriced if you just want to drive to the supermarket.
Re:lead balloon filled with hot air (Score:1, Informative)
Actually, I think there was a recent price reduction. $125K. See IBM pricing updates [ibm.com].
The mainframe has a problem. The people who price the add ons. $10,000/GB. Holy Markup Batman!!!!!!! Triple redundacy but you can divide that number by 10 and still have a large number.
It is kind of hard to put larger workloads onto Linux (and hence need to buy more MIPS... and sell more hardware) when it takes over a $100K just to get there. Alas it is priced for folks already waist high in huge legacy software costs. Probably a Wal-Mart special compared to the z/0S fees. Not sure how they can sell alot more "iron" with these prices though.
However, as outlined in other threads.... Linux/390 is aim at niches where it makes sense.
Re:It works well (Score:2, Informative)
"Keep the customer satisfied" works for me. They know you can go elsewhere so they try very hard to get you not to need to, nor want to.
My favourite analogy is that of European and American Roulette wheels, US wheels have a double zero and European ones don't. They'll likely both get the same money but the US ones want it quicker. (Of course all analogies are flawed, and this one ignores effectively free food and drink at US casinos...)
Re:Don't forget..... (Score:3, Informative)
Good luck at getting one in your door for under $160 an hour - if you sign a 1 year gig maybe but not for a few weeks.
Re:Sticker Shock (Score:1, Informative)
Archaic is relative. When was the last time you saw a VT100? yet there are the escape codes stilled being used to move the cursor around, to set colors, etc.
Yes virtual machines have card readers and card punches much the same way that linux has psuedo ttys.
What is going on under the covers is fairly simple. VM sets up a virtual s/390 (z/Series..) environment for each user. It has z/Series architectured type devices (some may be real, others are virtual). A typical VM userid will have:
Now the virtual reader/punch/printer are simply an interface to the VM spooling system. But that interface has to follow the z/Series architecture rules for access a device -- So.. VM uses the card punch and card reader channel commands as the interface. That means that the rules of the device have to be respected.
Now when one IPL's from the spool, the IPL sequences is going to use card readerchannel commands -- so the kernel has to be blocked as 80 byte records.
To put this in perspective, when you boot Linux from a floppy drive, the kernel has to be blocked in to 512 byte records (which is the underlying block size of floppy disk drives if I remember correctly).