Slashdot is powered by your submissions, so send in your scoop

 


Forgot your password?
Close
typodupeerror
×
Linux Business IT

Building a Linux Virtual Server 93

Posted by Zonk from the virtual-but-stable dept.
prostoalex writes "NewsForge (an OSTG site) has an article that discusses the details of building a cluster for a virtual server - an environment where one can add and remove machines as needed, to account for high-traffic Web site of intensive database application: 'The main advantage of using LVS is that unlike Microsoft network load-balancing clusters, the LVS allows you to add a node running any operating system that supports TCP/IP to the cluster.'"
This discussion has been archived. No new comments can be posted.

Building a Linux Virtual Server More

Building a Linux Virtual Server

Comments Filter:

  • PC's are not for networking (Score:4, Interesting)

    by gtrubetskoy ( 734033 ) * on Friday June 10, 2005 @05:14PM (#12784939)

    First of all, the Linux Virtual Server project is a misnomer, because a Virtual Server these days means a virtualized operating environment similar to what is provided by VMWare, Xen, Linux Vserver, etc.

    Second, IP Load Balancing is not new and is much better done by a hardware device such as Cisco CSM, Foundry ServerIron, Nortel Alteon, etc. These things boot in seconds, use ASICs, can process orders of magnitudes more packets per second than a Linux box can, have no moving parts (except for the fans), are aesily configured, can be set up in a fail-over configuration, speak routing protocols, etc, etc.

    The Linux Virtual Server is a good project to tinker around with if you would like to understand load-balancing concepts better and do not have the $$$ for a hardware device, but I would not even consider designing a mission-critical solution using LVS instead of a hardware load-balancer.

    I remember back in my ISP days we purchased a small provider with a few T1's all running on Linux-based routers. They had a tendency to lock up and were very awkward to manage for many reasons (e.g. upgrading the kernel was a risky procedure that required way more downtime than necessary) and we replaced them with real routers pretty quickly. I kinda suspect that Linux Virtual Server may be a similar type of experience - PC's just don't make good network devices for some reason.

    Of course I'll probably get flamed for this comment...

    • I agree that PCs do not make very good network devices. But they offer something in price, that none of the other appliances i know of can match.
      • Toss in a 2nd PC, use Mon, and use Linux HA [linux-ha.org] and you've got yourself a high availability cluster that can route almost TCP IP or UDP traffic for far less cost than a "real" load balancer not to mention can do far MORE than a load balancer. We are using LVS and HA at my 9to5 job for load balancing our new Webservers. ITs super reliable. We have it such that within 10 seconds of a web server not being available, its removed from the LB. Once its back up and running, it adds the server back in automatically
      • This brings up the chinese menu of the tech world:
        cheap, reliable, fast - pick two.

    • Re:PC's are not for networking (Score:5, Insightful)

      by Dasein ( 6110 ) <tedc AT codebig DOT com> on Friday June 10, 2005 @05:23PM (#12785041) Homepage Journal
      Many F5 appliances were not much more than FreeBSD PCs

      • Re:PC's are not for networking (Score:1, Informative)

        by Anonymous Coward
        Really true! Our best Loadbalancer appliance, 5 years without one single failure, is actuallty a BSD.

        Also Nokia CheckPoint appliances are BSD!


        My .02E

      • Not only am I an F5 customer and have many friends who work there but I once worked with a company that occupied space formerly occupied by F5.

        Get this, the system that controlled card-key access was a *VERY* early F5 box that they had installed Windows on.

    • Re:PC's are not for networking (Score:4, Informative)

      by Serveert ( 102805 ) on Friday June 10, 2005 @05:23PM (#12785043)
      with keepalived doing health checks / failover and using single CPU LVS boxes, it can handle your mission critical apps just fine. DR(direct route) would be more efficient than NAT since you would cut in half the traffic going through your LVS box. The key is single CPU systems, LVS doesnt like multiple CPUs much and it doesn't add much value (according to certain people in the LVS project).
    • I would have to disagree with you here on a number of points. I am not sure what distro of Linux you where using and why you had all the down time ? Upgrading the kernel should only require a reboot ? So the down time is how long it takes your server to boot up. Did you ever look into the lock up issue ? Could it be bad hardware or driver hardware mismatch ? The only time I have ever locked up a Linux server was when I was using hardware that was not supported or I was using hardware with incorrect modu
    • I want to second your comment as I have had a similar experience. Now, do not get me wrong... I love linux. I have used RedHat and Debian for all kinds of tasks.
      I worked for a small ISP in the mid 90s and there were times, when in a pinch I had to setup a linux router or load balancer. And I got them to work. But there was always a lot of effort involved in getting it to work, and they were always precarious in operation. It was scarey to do upgrades. It was scarey to have hard drives fail.
      It was alw

      • PC's are great for networking (Score:5, Informative)

        by rcw-home ( 122017 ) on Friday June 10, 2005 @09:16PM (#12786452)
        But there was always a lot of effort involved in getting it to work, and they were always precarious in operation. It was scarey to do upgrades. It was scarey to have hard drives fail.

        Here's another approach: put Linux on a CompactFlash card instead of a hard drive. Keep the filesystem mounted read-only for normal operation. Test upgrades on a different computer and CF card. Upgrade by swapping out CF cards. If you can build a PC that doesn't need fans, then you've removed all moving parts from the equation. For smaller installations, single-board computers such as the Soekris [soekris.com] are very reliable. For larger installations, you can usually find a 1U system with the level of performance you need.

        It was always just so much simpler to get a more expensive Cisco box in the long run due to its design, documentation, an performance.

        It always depends on what you're doing, of course, and also what you're familiar with, but for my routing/firewalling/VPN/load balancing/ etc needs I've simply found Linux to be more flexible.

    • Many administrators prefer out-of-the-box solutions. There are good reasons for that. Their simplicity makes them secure, because there are fewer opportunities for mistakes. This is a completely valid point that should always be taken into account when choosing a load balancer.

      That said, Linux Virtual Server is relatively easy to set up (for your average unix sysadmin), very reliable, fast and scalable. It is not inherently slower, less reliable or more dependent on moving parts than any hardware solution.

    • These things boot in seconds,

      Check.

      can process orders of magnitudes more packets per second than a Linux box can,

      How did you benchmark this? What options where you using? I'd like to know. It's hard to argue with a vague blanket statement.

      have no moving parts (except for the fans)

      Check. It's trivially easy to put a Flash HDD in a PC.

      are aesily configured

      Vague and debatable...

      can be set up in a fail-over configuration

      Check.

      speak routing protocols

      Check.

    • I would argue that LVS is a fine alternative to costly hardware based load balancers. I have tried both hardware based load balancers from Cisco and used LVS. Using LVS I have provided sites with 5 9's worth of uptime. With keepalived and two load balancers you achieve a very robust and flexible system. My 2c worth? If you are requiring load balancing, cater to your strengths. If you have a lot of experience running Linux and want something that costs just the simple hardware, use LVS. If you know Cisc
    • I've deployed linux as routers a few times and for the most part in days past your opinion mirrors mine. However kernel 2.4 and 2.6 have greatly improved the useability of linux as a router. They're about on par now with low end hardware networking devices from the bigname companies... they still can't compare to a "real" router though... maybe someday-- after we geta way from moving parts :)

      There are some intersting things you can do with linux as a router that you can't do with a hardware device though
  • Combine that with something like Virtuozzo from SW-Soft or (the now dying) ServerXchange from Ensim (webhosting virtualization software packages) and you could cluster VPSes together and have the added virtuazliation over hardware that would let you move OS images around between machines.

    You could move the systems in and out of the cluster, but you could also move the OS install around on different hardware. Reliability through virtualization, it's a great thing.
    • Ew ServerXchange, actualy Virtuozzo and PEM from SW play rather nicly with load ballancers. You have to make the users data get mirrors any way you can (iSCSI and software raid rock for this) Hsphere has a similar tack but only officualy plays nice with NetApp (works with the OSS version ok though)

      As to load ballancing it's realy a toy when compared to the hardware boxes and as few year old arrowpoint (now cisco) boxes can be had for under 500 used that go faster than any linux box can think of with less
  • Imagine a Beowulf cluster of...never mind.

  • Don't forget the importance of keepalived (Score:3, Informative)

    by Serveert ( 102805 ) on Friday June 10, 2005 @05:18PM (#12784988)
    http://www.keepalived.org/ [keepalived.org]

    What is Keepalived ? ...

    So in short keepalived is a userspace daemon for LVS cluster nodes healthchecks and LVS directors failover.

  • Dubious benefit? (Score:3, Insightful)

    by Sv-Manowar ( 772313 ) on Friday June 10, 2005 @05:19PM (#12785003) Homepage Journal
    The article seems to suggest placing a large number of real servers behind a single virtual server. Surely this is putting a point of weakness in front of a potentially redundant cluster. Load balancing has been done better by specific hardware made by Foundry and the like. I'm not seeing the benefit over 'traditional' clustering with this solution.

  • Sweet ! (Score:2, Funny)

    by ilikeitraw ( 706793 )
    Do you need to buy special glasses ?
  • Can you imagine a Beowulf cluster of these clusters for virtual servers?!?

    • Actually... (Score:3, Interesting)

      by jd ( 1658 )
      ...A cluster is exactly what you want, as opposed to one of these virtual servers.

      A "true" virtual server allows a cluster to be treated as a single machine, from the outside. However, the problem with LVS - and similar technologies - is that they don't do this very well. You often have a single entry point, which means that communication suffers from the bottleneck.

      Ideally, what you want is for ALL the machines to be capable of receiving connections, so that connections can be striped across the farm,

      • A "true" virtual server allows a cluster to be treated as a single machine, from the outside. However, the problem with LVS - and similar technologies - is that they don't do this very well. You often have a single entry point, which means that communication suffers from the bottleneck.

        You are correct. I fail to see what the big deal about this "Linux Virtual Server" is. First of all, it's a terribly chosen name. It seems to imply that you're running some type of "beowulf"-like cluster across multiple
      • What is GAMMA? I found some projects related to gamma radiation, but not load balancing.
        • Freshmeat record [freshmeat.net]


          Main website [unige.it]


          GAMMA is a high-performance, low-latency, non-routable protocol, designed for clusters on a single LAN. The web page isn't that well written, but the code itself seems prety solid and is for the 2.6 kernels.

  • Why does the summary talk about intensive database applications? Not only are databases not mentioned in the article, but having a database server that the Web servers need to talk to would be a huge bottleneck for this configuration.
    • MySQL and some other db servers can handle load balancing across multiple machines using their own methods. You're probably right that the db is often a weak point (which is one reason I don't like cramming unneeded logic into the db and forcing the db to do extra work) but every part of a web server farm should be able to use load balancing in some way or another if you write your web apps right.

      I use a cluster for front-end logic (templating) and another for back-end logic. When I need to I'll do likewis

  • Built one. (Score:5, Insightful)

    by NovySan ( 125848 ) on Friday June 10, 2005 @05:30PM (#12785098) Homepage
    Great for http. Bad for nfs and smb/cifs.
    • I don't think they meant it to be used with protocols like nfs and smb/cifs. This kind of load balancing performs very well with "simple" tcp-based protocols, ones that make a socket connection, exchange some data, and then close it.

      Therefore, it's been used with great success for HTTP, SMTP, IMAP, POP and so on.

  • What about transparancy? (Score:3, Interesting)

    by Jailbrekr ( 73837 ) <jailbrekr@digitaladdiction.net> on Friday June 10, 2005 @05:34PM (#12785139) Homepage
    With any load balancing solution, if a server goes down with an active connection, that connection gets reset. It doesn't matter if you are using LVS, pen or pound, the level of transparancy isn't quite there yet.

    Personally, I am an advocate of pound for http/s load balancing, because its simple, has central logging, and handles ssl authentication itself (thereby keeping the back end servers simple)

    Check it out:

    http://www.apsis.ch/pound [apsis.ch]

  • What the article talks about is a web server farm. The servers aren't migrating data among each other... and since the author suggests using round-robin DNS for redundancy instead of DNS-based wide area load balancing and/or having a failover-style cluster setup so one machine going down activates a standby... well... while it may be useful to get people playing with things on their home network, it's not anything people who want to keep their jobs should be doing unless they're on a near-zero budget and u
  • DEC did a fair bit of research and development into such composite systems back in the late 1980s. That is why today systems like Tru64 UNIX are so adept at virtual networked systems. It's too bad that HP has squandered the gem it picked up from DEC (via Compaq) in Tru64.
  • There's a mention of databases in the article summary (which peaked my interest), but the article was ONLY about web servers (snore). What kind of summary was that?

  • WLBS vs LVS (Score:4, Interesting)

    by markclong ( 575822 ) on Friday June 10, 2005 @06:05PM (#12785404)
    Looking at these images (LVS [newsforge.com] and WLBS [microsoft.com]) it seems with LVS there is a single point of failure with the load balancer box. With Windows load balancing there is no "balancer box" and if any one server fails the cluster continues without missing a beat.

    How does LVS work if the balancer box fails?

    • Re:WLBS vs LVS (Score:1, Informative)

      by Anonymous Coward
      Add a second load balancer box and fail over to it. Connection-wise, it's seamless.

    • Re:WLBS vs LVS (Score:2, Informative)

      by AtlanticGiraffe ( 749719 )
      Microsoft has that SPOF too, they're just calling it a router. Keepalived lets you set up two separate load balancer boxes for redundancy. When your primary fails, the secondary takes over and sends you an email about it. Everyone else will never know.

  • my experience (Score:4, Insightful)

    by buddha42 ( 539539 ) on Friday June 10, 2005 @06:07PM (#12785420)
    Back when I worked for my college's web department I oversaw moving their website from the old server platform to a load balanced cluster of x86 servers using LVS/linux/apache/php/mysql. It is not a minor undertaking in any way. Configuring, tweaking, and troubleshooting all the various elements requires an intense amount of trial and error and confusion. Think of it like sendmail or asterisk (its not quite that bad). Overall, the cost of implimenting it in manhours and downtime from learning-experiences weren't worth it being "free". If I had to do it again, I'd buy real network equipment. Even if used off ebay, just get a failover pair.

    Now if someone came out with a good embedded linux running lvs with a nice web and cli interface, on solid state hardware, that was a decent amount cheaper than brand name equipment, put up a respectably proffessional looking site, and develop a loyal following, I'd consider it.

    • LAMP configuration is pretty easy usually. I assume it was the use of LVS for load balancing that made things diffuclt for you? What kind of issues did you have?

      Did you try just using DNS for load balancing? That's proven easy and reliable for me.

      Did you support users website's too? How did you distribute their files among the different servers?
    • Perhaps. But was the hard part configuring LVS, or configuring your web apps to play nice in a even more-stateless enviroment? Its been a few years, but I suspect that if necessary, I could get up and running a pair of (fall over) LVS director boxes, and configure a handfull of backend systems in a good day. Tweeking the backend systems, testing... thats another story; but that time would be consistant across LVS and commercial solutions.
  • No? You don't even need them physically connected these days, SCSI over IP can do it.

    LVS isn't really an ideal system, the load balancer is bound to be the box that dies.

    For a clustering project :

    http://www.linux-ha.org/ [linux-ha.org]
  • Back in the day you would have been so sensitive about this sort of cross site promotion, but I guess its all about the stock price these days, huh?

  • LVS is not really new news and it is a nice hack and has some good use cases. But in the real world 'dictated and redundant ' load balancers (such as BigIP, etc..) are the only way you are going to run a 24/7 large dynamic web site. Would you REALLY place ONE box (Linux or not) as the gateway/proxy to your web farm, if so, then make so it is a fully redundant box (including CPU) otherwise on the day after the melt down start looking for another job. And if you really place one fully redundant box as your w
    • keepealived allows for hot standby and even active-active setups. Plus LVS syncs state via multicast. Also, no hacking of routing tables needs to be done, the standard IP takeover works. I've used it in production, taken down the primary, failed over seemlessly, state was synced, no lost connections. It worked fine at 1000 connections/second. Have you done this?

  • whine whine; check out ultramonkey (Score:4, Informative)

    by havardi ( 122062 ) on Friday June 10, 2005 @06:24PM (#12785528)
    They have different configurations explained here:
    http://www.ultramonkey.org/3/topologies/ [ultramonkey.org]

    They even have an example where 2 machines act as both real servers and directors:
    http://www.ultramonkey.org/3/topologies/sl-ha-lb-o verview.html [ultramonkey.org]

  • Like the products from F5, Cisco and Coyotepoint. I thought it was going to be like VMWARE or MSVirtual Server.
  • The last time I tried this stuff out was in 2002. I had a minimal setup with two frontend and backend servers. The frontend machines shared a virtual IP address and kept track of oneanother using a heartbeat. However, the main limitation, as far as I was concerned, was the algorithm used to distribute the requests among the backend machines. It was just that, an algorithm: not anything intelligent.

    The problem is that some client requests are 'bigger' than others. If one of my machines got swamped by one o
    • There are several monitoring programs you can use to control what you're talking about... You would probably want to look into keepalived (for basic monitoring of responses from the real servers) or lvs-kiss (for more in-depth monitoring of various stats).

      In a lot of cases, just using an algorithm that watches the number of connections on the servers will cover this... the swamped machine will end up with more pending connections, causing further requests to be balanced out to the other servers instead. U
    • It's called wlc (weighted least connection) algorithm. It does what you want because you can infer the loading of the server by how long it takes to close the connection. Also similar to this is the sed (shortest expected delay) algorithm and the nq (never queue) algorithm.

      The wlc algorithm does have a small drawback though. When a new server is brought into the cluster, the wlc algorithm hammers the new server since it is starting out with zero connections compared to the active servers. The sed ove
  • Round Robin DNS?

    *Yawn* Nothing to see here...

    ps. burn karma burn!!!

  • Lame Article ... Excellent Software (Score:5, Informative)

    by Anonymous Coward on Friday June 10, 2005 @07:57PM (#12786029)
    Unlike most of the posters sharing their 'wisdom' on the merits (or lack thereof) of using PC's for networking ... I can say that I recently setup a load balancer using LVS in front of my Web cluster (of 6 dual Opteron's). It rocks!

    You can easily avoid it becoming a single point of failure by using 2 directors with realtime syncronization and failover. Keepalived works well in this regard, giving subsecond failover performance, and even uses VRRP, Cisco's own hot standby protocol.

    As for the level hardware that you need, a low end or older PC, such as a P3-700 for instance, can easily saturate a 100 mbit connection.

    I used 2 x P4 2.4G systems. So for less than a fraction of the cost of even 1 hardware based boxes, I get hot standby and should be good for way more throughput than I can afford to provide. :)

    The best feature that LVS brings to the table that the hardware based implementations miss out on, is flexibility. This software is being enhanced all the time and nothing is impossible with it. The same cannot be said for the hardware solutions.

    Unfortuneately the article does not touch on any of the nice add-ons, like keepalived, ldirectord, etc., that make LVS a manageable solution. I know that they are trying to keep the examples simple, but showing servers being configured using solely ipvsadm is not too realistic for the average person and make it look undeservedly complicated, which it is not.

    Give LVS a look an see if it might not give you the features you need without paying the big iron prices.

    DC
  • Is it just me or have i missed the point. A virtual OS is something running under something like VMWare??? How is this news???
  • >a cluster for a virtual server - an environment where one can add and remove machines as needed,

    From the very beginning it is obvious that the submitter has no clue about clusters, virtual servers and high-traffic environments.

    >high-traffic Web site of intensive database application:

    What the fuck is this sentence supposed to mean?

    >'The main advantage of using LVS is that unlike Microsoft network load-balancing clusters, the LVS allows you to add a node running any operating system that support
  • This LVS may be free, but it's nowhere as advanced as current load balancers out there.

    For instance, I know of a few that will have the return traffic sent directly from a "real server" directly to the client, rather than that roundabout trip to the scheduler and back out. This is great for offloading network congestion.

  • OpenSSI [openssi.org] seems to be a good solution also


Slashdot Top Deals

C'est magnifique, mais ce n'est pas l'Informatique. -- Bosquet [on seeing the IBM 4341]

Close