Red Hat Wants Xen In Linux Kernel 278
DIY News writes "Red Hat is aggressively pushing to get Xen virtualization technology included in the Linux kernel as quickly as possible. This move comes as Microsoft is pushing its own virtualization products and recently relaxed some of its licensing requirements around Windows Server 2003 to facilitate more pervasive adoption and use of those technologies."
Xen into kernel (Score:5, Interesting)
That you can run virtual machines with that kernel? that that kernel can be hosted into a virtual machine?
Or that you can install parallel kernels and run part of the ELF binaries on the other machine?..
Re:Xen into kernel (Score:3, Informative)
"Xen is a virtualization technology available for the Linux kernel that lets you enclose and test new upgrades as if running them in the existing environment but without the worries of disturbing the original system"
Re:Xen into kernel (Score:5, Informative)
Re:Xen into kernel (Score:3, Interesting)
Another responder mentioned live migration being much harder - he's right. He's also right when he says this is also done, with some really, extremely, very cool results [cam.ac.uk]. You can migrate a Quake server from machine a to machine b iff you assume some SAN or NAS in 60ms downtime, PRESERVING NETWORK CONNECTIONS. That means if you're remotely logg
It means (Score:3, Interesting)
The other problem here is there are other VM's out there and they all have different requirements for kernel modifications so talk about mess.
The major underl
Re:It means (Score:5, Informative)
In any case, if you really want to learn about the fundamental concepts behind virtualization, I strongly recommend reading the following paper: Formal Requirements for Virtualizable Third Generation Architectures [u-tokyo.ac.jp]
Yes, it was published in 1974, but most of the concepts are still very applicable and make a lot of sense. (though the architecture examples are obviously dated)
This is a very good paper which lays out all the ground rules. Sure, it may sound a bit academic in terminology and explanation, but it is still quite readable.
Re:It means (Score:2)
add some new CPU instructions and another processor run level that
would support virtualisation at the hardware level (which needs to be
the case to do it properly).
Re:It means (Score:5, Informative)
The full x86 architecture is not suitable for virtualization, because there are a few instructions which fail silently when run from user level.
VMware uses various techniques to get around this, including full simulation and binary re-writing.
Xen uses another approach, where they port to an instruction set that is basically x86 without the problematic instructions. This approach requires that the guest OS's be modfied.
This will all change with the new virtualization instructions being added by both AMD and Intel. Once that is in place, Xen will be able to run unmodified guest OS's (such as Windows, for instance). There will be a speed hit though, so modified guests will be prefered if speed is an issue.
Re:Xen into kernel (Score:5, Informative)
Once the Xen0 kernel is running on top of xen, you have basically a normal linux kernel running that does all the hardware support. Then you load up Xen guest machines running the Xen0 kernel and these run their in their own virtual machines complete with their own disk images and linux distro. So xen doesn't really have anything to do with running elf binaries on the other machine. If you ran FreeBSD in the guest, it would run those binaries inside of that OS and that libc. When Xen 3.0 comes out, if you have the new intel or amd chips that support on-chip virtualization, then Windows XP can even run as a guest underneath the linux kernel-Xen0 host.
Re:Xen into kernel (Score:4, Informative)
Nitpicking the "main OS" wording: it's the host OS. I would think in a production server environment you'd keep this OS minimal and not do much in the Xen0 domain so you don't risk crashing or compromising the host environment. On the other hand, if it's a game box it would make sense to have the 3d video drivers in domain 0, and if it's a workstation it may or may not make sense to have the host OS run apps and have the user domains for testing purposes.
Once the Xen0 kernel is running on top of xen, you have basically a normal linux kernel running that does all the hardware support. Then you load up Xen guest machines running the Xen0 kernel and these run their in their own virtual machines complete with their own disk images and linux distro. (my bolding)
Typo: The guests are XenU (user) kernels which typically have no real device drivers and are therefore much smaller. Very well put, though. Note that you can use any block device as a disk image: a file, an LVM volume or even an actual hdd.
Re:Xen into kernel (Score:2)
Does that new hardware support allow running Windows without having to modify it first? For the moment, it appears that no OS will work without modification and support for Windows might not be forthcoming:
Re:Xen into kernel (Score:5, Interesting)
In rough terms:
Admin Cost = N * (H + S)
where N is the number of computers, H is the network and system hardware admin costs for a single machine, and S is the sys admin costs for the machine. Distributing:
Cost = NH + NS
This is a gross simplification, since we all know that complexity is not a linear function of network size, but it will do to be goign on with. Now we take NH and by virtualizing the machine it becomes simply H, so
Admin Cost (multiple virtual machines) = H + NS
Basically, I think it'll be common practice in the future to create virtual machines out of thin air by copying a config file or some directories on a machine with available bandwidth. If the cost of enough surplus hardware is less than (N-1)H, then wouldn't it be cheaper to virtualize?
Of course the complexity is that costs aren't linearly related to N, or for that matter constant in the size and class of machine you are managing. Which is another way of saying YMMV. I think there's clear application in many kinds of situations, for example in software development where we're constantly worried about the various combinations of software our work will have to coexist with. It'd be very convenient to be able to pull a certain system configuration out of a library and have it up in a few minutes, then trash it after a few hours of use. But it may have potential in production environments too.
Xen... (Score:5, Funny)
Re:Xen... (Score:2)
Umm (Score:5, Informative)
Re:Umm (Score:3, Insightful)
Re:Umm (Score:3, Insightful)
As a rather off-topic aside, it strikes me that Sony and Microsoft selling their game consoles at a loss is doing exactly that. Sony makes some profit now on their game division, but Microsoft has lost billions.
From this armchair, that sure looks like classic monopoly abuse.
Re:Umm (Score:2)
Re:Umm (Score:4, Interesting)
Re:Umm (Score:2)
They had to re-engineer Windows to make I.E. replacement simpler.
Re:Umm (Score:3, Informative)
There are basically 2 ways in which you can do that: by dumping or by creating private standards. Dumping is selling stuff below cost price (and thus taking losses) until your competitors are out of business. Private standards can be used to make competing related products incompatible or generally inconvience users of those products and thus try to set up a monopoly, this is at the core of Microsofts business strategy.
For many people
Re:Umm (Score:5, Informative)
Operating system virtualization on the other hand pretty much requires it be hooked into th OS by its very nature.
Re:Umm (Score:4, Informative)
Furthermore, all of their anti-trust problems were in the desktop market. As long as virtualization was positioned as a server feature, and as long as MS didn't threaten any VMWare supporters, I don't think they would have any legal problems.
Re:Umm (Score:3, Informative)
In other words, if MS decides to start shipping software to learn how to speak Italian with every copy of Windows XP, then they would have to allow makers of competing software the same courtesy to ship alongside their own.
If they didn't command a monopoly of th
Re:Umm (Score:2)
Xen has a number of interesting features that require paravirtualization techniques, which would need to be built into the OS.
Two birds, one stone (Score:5, Interesting)
Imagine if you would the ability to use Xen for unlimited operating systems, no licensing cost of the base OS, thinking about it, I would prefer to be in Microsoft's shoes as opposed to VMWare's. Only difference is that Xen when compared to VMWare is a very immature platform and no IT manager is going to take Xen over VMWare just yet (Unless cost is a BIG factor).
I would have to say that this is still very cool, with all the new Virtualization options come out in the new cores shortly and if they can get to market before Microsoft, this is a great way to pick up some customers. Kudos to RedHat and IBM and Intel and everyone else for making this happen.
Re:Two birds, one stone (Score:3, Interesting)
With VM Player and vi you may get a free virtual machine.. see http://b100dian.lx.ro/wordpress/index.php?p=90 [b100dian.lx.ro]
Re:Two birds, one stone (Score:2)
Re:Two birds, one stone (Score:3, Informative)
Imagine (Score:2, Funny)
Are you asking us to imagine what I think you're asking us to imagine?
Re:Two birds, one stone (Score:2)
Have you looked at the instructions to run something with XEN? I don't think this is a threat to VMware, in fact it is probably good for them because they can use it and it will make their product look better (better performance, better integration) and they can continue to sell an application to configure the virtualization system.
Re:Two birds, one stone (Score:2)
I truly don't think it will be that long. The hardware vendors IBM / HP etc will be willing to support this as if the Xen community comes out with extensions for the Virtualization stuff in the new CPU cores (also coming out shortly), then it will help them to push more kit (sell more hardware).
Re:Two birds, one stone (Score:3, Informative)
vmware in linux is the absolute #1 tool useful for reverse engineering. I can pipe the rs232 and usb as well as ethernet ports fo the hosted OS to files or through sniffers on my linux machine and figure out quite quickly how a device is talking to it's hardware or server on the net. Capturing an entire session into a input and an output file makes is trivial to reverse engineer something.
this can not be done with zen or the utter crap that msft is
Re:Two birds, one stone (Score:3, Informative)
Imagine if you would the ability to use Xen for unlimited operating systems, no licensing cost of the base OS, thinking about it, I would prefer to be in Microsoft's shoes as opposed to VMWare's. Only difference is that Xen when compared to VMWare is a very immature platform and no IT manager is going to take Xen over VMWare just yet (Unless cost is a BIG factor).
I've been using xen here in what I call "production development." Its serving several development servers. One of them is running a crappy spam as
Re:Two birds, one stone (Score:2)
Re: (Score:2)
Re:Two birds, one stone... VMware (Score:2)
Here is a garden path, care for a walk?
You never know, VMWare might like the idea of booting MS out of the market all together and keep Xen on the small end of the playing field and build tools to migrate Xen machines to VMWare machines. (I don't know if this exists already)
I remember that Xen although technically capable of emulating a base platform for Windows isn't allowed to now because of patents. With the new virtualization in the chip, I think the Xen
Aggressively pushing? (Score:5, Insightful)
Perhaps it's only me, but this doesn't sound aggressive; this sounds friendly and cooperative.
Re:Aggressively pushing? (Score:4, Insightful)
So, they're "aggressively" pushing Linux instead of Windows as a virtualisation host OS. Six staff members hired to work solely on integrating it into the mainstream kernel is fairly aggressive (toward MS), I would say, as it could lose them a major new market.
Re:Aggressively pushing? (Score:2)
Forking? (Score:4, Insightful)
Re:Forking? (Score:3, Informative)
I know that SUSE 9.3+ has built-in support for Xen, and since there are different kernel packages especially for Xen support, I assume that Suse has accomplished what Red Hat is working towards. Although, I could be completely wrong since I didn't RTFA.
Re:Forking? (Score:3, Interesting)
The irony (Score:5, Interesting)
Re:The irony (Score:5, Insightful)
No, both are quite specific it's the *GPL* they don't like, not Open Source in general.
Re:The irony (Score:5, Insightful)
Re:Irony? (Score:3, Funny)
BALDRICK: Yeah, it's like tinny or brassy, only it's made of iron.
My Bias (Score:4, Funny)
And then the crazy part of me says. "Heh, I can compile modules for the Xbox controller and other weird hardware into the kernel. Maybe useful technology should be in the kernel =]
but then again. I just might have to many voices in my head
Re:My Bias (Score:4, Informative)
IMHO, virtualisation is going to become very important to all sofware developers over the next few years. If it is easy to fire up a Debian system on top of a SUSE and have Mandriva & RedHat running as well then you can test your app on all these platforms at the same time. Hurrah!
Re:My Bias (Score:2)
Because if you're not using it, it shouldn't be there?
So, basicaly what you are saying is "WFT is this rtl ethernet driver doing in kernel, I don't have this fscking network card! Anyone who does should apply patch"?
Typical computer has one CPU, one chipset, one eth... Now, just why do you think kernel contains more than one. Because people don't use the same one hardware you use maybe? Same goes for the needs, you have your own needs just as everybody else has their own. Personally, I
Re:My Bias (Score:3, Insightful)
They've done nothing anti-community since dropping free Red Hat 9 support years ago. Get over it.
-Erwos
Re:My Bias (Score:3, Insightful)
Have they done anything sinister? Not yet. Will they? Who knows. But it's fun to complain about them =]
Re:My Bias (Score:2, Insightful)
That's the retarded part of you. Learn to ignore it.
Re:My Bias (Score:4, Funny)
(c) me, 2005 (Score:5, Funny)
Must remember to patent the idea of trojan/virus that uses visualisation to run a spam/DOS server
in a Windows environment...
Rubs hands with glee as he tries to sell the idea on IRC.
Re:(c) me, 2005 (Score:2)
(Ducking and running)
Just kidding. Java is my friend.
"Innovated" by wb: (Score:2)
I see no problem here... (Score:2)
Re:I see no problem here... (Score:2, Informative)
"Part of the Red Hat emerging technology team's efforts will be to drive the Xen virtualization technologies as part of the Linux kernel rather than as part of a sidebar project, as is currently the case, Stevens said."
JOhn
Sun could be in RedHat's crosshairs (Score:4, Insightful)
Re:Sun could be in RedHat's crosshairs (Score:4, Informative)
This concept likely provides many advantages for system resource management on a server, where you only care about a single operating system. It does not, however, let you run different OSs at the same time.
Why Xen and not vservers? (Score:4, Interesting)
While Xen appears as a neat package, why choose Xen instead of vservers [linux-vserver.org]?
The hardware cost of running multiple copies of the same OS with vservers is smaller than Xen - there is one and only one copy of glibc in memory, one and only scheduler, and so on.
Re:Why Xen and not vservers? (Score:4, Informative)
The hardware cost of running multiple copies of the same OS with vservers is smaller than Xen - there is one and only one copy of glibc in memory, one and only scheduler, and so on."
But part of the purpose of a virtual machine is that you can run a different operating system in each partition, including different schedulers and libc versions.
Re:Why Xen and not vservers? (Score:4, Informative)
Under Xen you can reduce the parent kernel down to obare minimum, reducing the chance of errors.
eg: you want to run an experimental iptable module on one of the virtual servers, no problem, if it crashes, all the other servers keep on trucking.
Essentially Xen provides a better sandbox from a stability/security perspective.
Re:Why Xen and not vservers? (Score:4, Informative)
Also, Xen does things to make that 1 copy of glibc a reality. Arguably, that 1 scheduler is one of the primary reasons you would prefer Xen.
Re:Why Xen and not vservers? (Score:3, Interesting)
Re:Why Xen and not vservers? (Score:3, Informative)
Perhaps because vservers lack some of the neat features of Xen, such as on-the-fly instance migration [cam.ac.uk] and full iptables support?
Furthermore, vservers is, for the foreseeable future, a Linux-only project. So far, NetBSD [netbsd.org] and Solaris [newsforge.com] have been ported to Xen, and basic support [fsmware.com] for FreeBSD as a guest host is available. Once Intel VT and AMD Pacifica are available, Xen will also support [com.com] Windows XP SP2.
Given just these benefits (and Xen has ma
Mandriva 2006 includes Xen (Score:4, Interesting)
I'm running such a box now with a total of three Linux domains (one host domain and two guest)... much easier than manually patching everything.
Re:Mandriva 2006 includes Xen (Score:2)
But unfortunately it's broken on 64-bit machines. (see bug #18432 at qa.mandriva.com; Mandriva says that they will upload the x86_64 glibc-xen packages when they have verified that xen works with their x86_64 kernels, which they currently don't).
All hail the new devil (Score:2, Insightful)
Put differently, how are AMD and Intel going to make it work? Since hardware doesn't like multiple masters (try a PS2 mouse with 4-5 byte protocol, it completely freaks out with a KVM switch), it's going to go haywire if you ha
Re:All hail the new devil (Score:2)
How does virtualization work? (Score:3, Interesting)
But then what do you when two or more OS instances want to monkey around with hardware that has state? For example, if one OS wants the screen resolution to be 640x480, and another OS wants the screen resolution to be 1024x768, you can't very well keep switching the screen between those two resolutions every time you change which OS is getting CPU time. Or another example is with printing: you can't very well interleave the print data streams from two OS's to the printer without hosing the print jobs.
Re:How does virtualization work? (Score:2, Interesting)
As for varying screen resolutions, it just made me think of the Amiga and pulling down a low resolution screen to reveal a high res on behind.
very very cool stuff, but inherantly impractical, I imagine it will most likely be similar to the KVM switching or simple desktop switching?
Re:How does virtualization work? (Score:4, Funny)
Re:How does virtualization work? (Score:4, Interesting)
For printers you can either set up a print server or the printer gets attached to a particular OS instance.
Re:How does virtualization work? (Score:5, Funny)
The Schrodinger Corp. makes special PC cases that can handle those requirements.
Re:How does virtualization work? (Score:3, Insightful)
Re:How does virtualization work? (Score:2)
Sort of.
There is really two parts to virtualization: 1) processor virtualization and 2) hardware virtualization. Processor virtualization allows multiple virtual machines to think they have the whole processor. In this case, there's a lot of interesting tricks you can do (and new hardware support) so that the virtual machine thinks it has the actual underlying pr
Re:How does virtualization work? (Score:2)
A common FreeBSD setup is to run a print server in the host OS and configure jails (FreeBSD's virtualization systems, which are completely unlike Xen) to speak to that server. In short, you treat the jail environments like standalone machines on a network. I'd suspect you'd do something similar under Linux.
Usermode Linux already in the kernel. (Score:3, Informative)
Re:Usermode Linux already in the kernel. (Score:5, Interesting)
Being the first to the party doesn't always mean you're going to the best; see DevFS vs. udev [kerneltrap.org].
Xen has much greater performance [cam.ac.uk] than UML and supports more operating systems. While UML is currently more mature and stable than Xen, it's only a matter of time before Xen surpasses UML as the preferred virtual server technology. Hell, even Linode, a strong proponent of UML technology and virtual server hosting provider is migrating to Xen [linode.com].
FYI, I'm currently running a Xen-based system with 15 virtual server instances for a system administration course at UC Berkeley on a server built with cheap off the shelf components (AMD Athlon 64 2800+, 1 GB RAM) and everything is quite snappy. It'd be difficult to even approach such usability with UML, and I'm using Xen 2.0.7. I can't see what Xen 3.0 will bring.
MS leapfrogging (Score:3, Interesting)
I never even considered virtualization of servers or development environments until I learned about MS Virtual PC and MS Virtual Server. Norton Ghost or dd dumps were all that I knew. So Microsoft is doing something right, and they will be perceived as the innovator and the winner here. They will be selling that you can virtualize servers to save time and money, and companies will buy it. They won't even know that this originated in the *nix world.
I look forward to seeing what the next leap in this technology is. I suspect we are just beginning to see some novel uses for it.
Re:MS leapfrogging (Score:2)
Re:MS leapfrogging (Score:4, Informative)
It didn't. This is mainframe technology. It just didn't work very well on x86 and hence the Windows using world was unaware of it. There is quite a bit of stuff that doesn't exist in the Windows world yet.
SuSE has had Xen since V. 9.3 (Score:4, Informative)
Re:SuSE has had Xen since V. 9.3 (Score:5, Informative)
A problem with Xen has been facing is keeping up with all the changes occuring in 2.6. If Xen is merged into mainline, there's a much better chance that Xen will be able to support the features and bug/security fixes that get added to 2.6 with each release.
For example, the current Xen stable (2.0.7) supports kernel 2.6.11.12. Every time a new security hole is discovered, system administrators using Xen have to manually backport a fix from the latest kernel. Having Xen in mainline should make this process much easier.
It's not personal, it's just marketing (Score:5, Insightful)
What the article actually seems to be saying - it uses the word "agressive" a lot as if this was some kind of virtue - is that Red Hat has a new senior honcho who'd like to make his mark. The issue of incorporating virtualization technologies into the Linux kernel is taken as a given by all parties. Which is hardly news. Chalk one up to the Red Hat marketing department for a nicely planted "news" story about their increased investment in the area (new hires, etc.), perhaps.
Of course (Score:3, Insightful)
Xen Virtualization HW "acceleration" (Score:2)
Re:Xen Virtualization HW "acceleration" (Score:2)
A solution to patches breaking functionality? (Score:2)
Seems like a good idea to me unless I am missing something.
I'd rather see Linux VServer included (Score:5, Interesting)
There are a few problems with Xen. First, it's i386 only. Second (and this is the biggest problem IMO) - Xen is venture-backed, and seems to be extremely eager to show their investors a return. Nothing wrong with that, but it's important to consider the motivation, and the consequence of a funding pull back. If XenSource does not turn out to be a great business, then will Xen still be developed and maintained? Why not wait a little bit, in the open source world quality over quantity matters and time pressure should not influence development.
Also, there is another project that I plug every chance I get - Linux Vserver [linux-vserver.org]. Unlike Xen, this is a purely volunteer effort, and is very innovative and attemtps to solve a difficult issue. Unlike Xen, these guys actually do not want to be in the mainline for now, becuase they think it will slow down development. Because Linux VServer is taking a different approach to virtualization (better known as separation, which was pioneered by FreeBSD jails and is also now supported in Solaris), the end result is cross-platform, i.e. runs on any architecture that Linux runs on.
Now in the past whenever I posted about Linux VServer a lot of folks said that Xen allows you to run multiple operating systems and that that is why it is so useful. I think that in reality running multiple OS's isn't all that valuable - the only case where it may be very useful is software development, but that's a tiny fraction of the Linux users. We've been using Linux VServer for hosting, and we are absolutely convinced that this is the right solution - for using Xen for example would introduce all kinds of problems (starting with resource bloat).
Yet unfortunately the OSS world has become PR driven lately. Very few people are technically capable of looking at things based on its merits and just go after the things that have the most buzz, not realizing that the buzz is artificially generated.
Re:I'd rather see Linux VServer included (Score:4, Informative)
Not true. Today, Xen supports i386, x86_64, and ia64. Xen is currently being ported to PowerPC also.
Second (and this is the biggest problem IMO) - Xen is venture-backed, and seems to be extremely eager to show their investors a return.
XenSource is a company backed by VC. Xen is developed by a much larger community though. There are a ton of press-releases that XenSource puts out that have the typical marketting junk that most Open Source folks despise but whatever, XenSource != Xen. Most of there people aren't even actively working on Xen anyway (they have a product for Xen management),
If XenSource does not turn out to be a great business, then will Xen still be developed and maintained?
Absolutely.
Also, there is another project that I plug every chance I get - Linux Vserver. Unlike Xen, this is a purely volunteer effort, and is very innovative and attemtps to solve a difficult issue. Unlike Xen, these guys actually do not want to be in the mainline for now, becuase they think it will slow down development.
Yup. That's why VServer is not in the kernel--they don't want to be in the kernel. VServer is a cool project, and I would love to see it end up in the kernel. Xen is also a cool project and it would be great to see it in the kernel. The kernel guys *will not* accept crap. Large portions of the Xen Linux port are currently being rewritten to live up to kernel standards. I have a ton of faith in the kernel folks overseeing the process.
Volunteer efforts can fall apart, too (Score:3, Interesting)
Re:What? (Score:2)
It's easier for sysadmins to say "sure, here's root, blow it up", than "no, you can't have root on this box, we'll consider your request for the patch during the next maintinence window"
It also lets you spawn dedicated servers in software.
For my home machine, my "server" can be stripped down to the most bare of bare components while my "sandbox" can have the compiler, web browser and other junk.
Re:What? (Score:5, Informative)
Its mainly an enterprise play.
If you're an old-timer UNIX admin, you may have difficulty understanding the point of server virtualization (i.e. multiple OS instances). In UNIXland, it has been normal and customary for several completely unrelated applications to run under the same OS instance, together servicing thousands of users. That never worked well in Windowsland. That being said, it didn't stop manufacturers from making staggering improvements in performace and capabilities of Intel servers. Companies grew to expect single Intel boxes to perform at the levels of large UNIX servers. The only way to achieve that in recent years has been to use industrial strength virtualization technology (basically, ESX). The boxes are beefier than ever, and ESX isn't cheap, but it works wonders.
For $50k, you can run 50 VMware guests on one very beefy box (not counting SAN), but you'd want a second for failover. For $75k, you can run about 100 guests on 14 blades in 7U (again, not counting SAN) and have the guests automatically migrate to the blades most able to run their workload at that moment in time. Ask a blade to come down for maintenance, and all the guests scatter to other blades before the blade powers off. Replaced a dead blade with a blank? Your systems management policies detect the new blank and automatically install ESX on it so guests can migrate back and evenly spread out the load.
Sounds crazy, I know, but that's a taste of what we're doing in the enteprise space these days.
Re:What? (Score:2)
Re:XEN vs UML (Score:2)
Re:XEN vs UML (Score:2)
Re:Erm ... Competition from VMWare Player? (Score:2)