No More Need To Reboot Fedora w/ Ksplice 262
An anonymous reader writes "Ksplice, the technology that allows Linux kernel updates without a reboot, is now free for users of the Fedora distribution. Using Ksplice is like 'replacing your car's engine while speeding down the highway,' and it can potentially save your Linux systems from a lot of downtime. Since Fedora users often live on the bleeding edge of Linux development, Ksplice makes it even easier to do so, and without reboots!"
Awesome! (Score:5, Funny)
Re: (Score:3, Interesting)
Re: (Score:2)
Re: (Score:2)
Re:Awesome! (Score:5, Funny)
Re:Awesome! (Score:4, Funny)
Ah, so Windows 6.1^H^H^H7 is all *your* fault!
Hmm... (Score:3, Funny)
Changing your car's oil while driving down the highway could be tricky, too.
Re: (Score:2)
Re: (Score:3, Informative)
It's voilà [wiktionary.org]. How hard is it to not look like a moron?
Re: (Score:2)
You're supposed to spell that "moran" when calling someone else stupid.
Re: (Score:2)
Re: (Score:3, Informative)
As much as I hate to weigh in on a grammar nazi thread, using a split infinitive is not necessarily incorrect.
http://en.wikipedia.org/wiki/Split_infinitive#Current_views [wikipedia.org]
If the meaning is obvious and unambiguous, let it be, I say.
Re:Hmm... (Score:4, Funny)
My tires always rotate while driving down the road. If you had an axle up your ____, you would rotate too.
Now this is even more applicable (Score:5, Funny)
http://www.imagepoop.com/image/660/I-Reboot-As-Much-As-I-Get-Laid.html [imagepoop.com]
Re:Now this is even more applicable (Score:4, Insightful)
The uptime obsession is crazy. Rebooting once in a while is useful, if only to see that you can still get everything running again from a complete stop. Kernel updates in particular can cause all kinds of problems at boot time. If you don't check the boot sequence, you'll almost certainly have forgotten what you changed that killed your cold boot ability when you need it for some other reason (moving servers, power failure, hardware upgrade, ...).
Re: (Score:3, Funny)
I Reboot As Much As I Get Laid
Man, that would be great for Windows users - not to mention they would *REALLY* look forward to Patch Tuesday.
Re: (Score:2)
Well it's an old technique actually. kexec have been there for ages.
How does this compare to kexec actually? I've been using it for a long time on all my Debian machines and while it's not 'instant' skipping all the BIOS, it's damn fast.
Re: (Score:3, Informative)
kexec restarts the entire software stack while leaving hardware running.
From what I can tell, ksplice does not require a software restart or hardware restart. This isn't explicitly stated, but it is implied by the usage instructions: http://www.ksplice.com/uptrack/using [ksplice.com]
Re: (Score:2)
A fellow named John Stumpo wrote a Windows driver called WinKexec [jstump.com] for doing a Kexec from Windows. I compiled it but never did get it to work. Something about the
Scary analogy (Score:5, Insightful)
"Using Ksplice is like 'replacing your car's engine while speeding down the highway,'"
So in other words it's something you'd never want to risk doing because it'd almost certainly cause a crash?
I think they should've thought about a different analogy for this one...
Re:Scary analogy (Score:4, Insightful)
True. The in-place kernel upgrade is somewhat safer than their analogy might imply, but it does lead to an interesting point. Why would you want to do this?
Personally, I'm OK with having to reboot my Linux machine when I change kernels, mostly because it's the only time Linux DOES ask me to reboot. To be fair, Microsoft and especially third party Windows software vendors have gotten a lot better about this in the last few years, so infrequent need to reboot is now a pretty solid feature on both Windows and Linux.
In any case, when I get a new kernel, I can install the new kernel and continue running along on the old one as long as I wish to, then reboot to apply the new kernel at a convenient time. Rebooting Linux Mint takes less than a minute from powerdown to login, and I know I haven't run into any risky process locks or anything during the upgrade process. Plus, I like the fact that the "older" kernel is always available to me on the boot menu in case something goes horribly wrong with the new one.
But I'm not all that uptight about "uptime". It's a home computer. If I have to reboot it once a month or so to apply the latest kernel, I'll reboot it. For my purposes, I don't see any added value for the extra risk (however slight) an "in-place upgrade" would introduce.
If I were running a "must be up 24/7" machine, I could see this as a benefit, but chances are at that point I've load-balanced a couple of machines and the cluster can stand a "rolling reboot" of the machines far better than it could stand a botched upgrade.
I still love the idea, and applaud the folks who managed it, but I don't think I see a real reason for it other than "wow, that's pretty nifty". It doesn't seem possible without introducing at least a little bit of risk, and it doesn't seem that the people who would really need it would be all that tolerant of the risk.
Re:Scary analogy (Score:5, Informative)
Re:Scary analogy (Score:5, Insightful)
And how would you know for sure that it would actually boot correctly the next time you actually *need* to?
There is nothing worse than having an actual unexpected reboot (UPS hiccup, whatever), and finding that the system that has been up for 3 years isn't booting, and not having ANY idea which patch put in place in the intervening time actually broke it.
Not that, ahem, I speak from experience, or anything...
Occasional rebooting is good, if for no other reason than making it happen in a controlled situation so you aren't surprised in an uncontrolled situation. If you really need the 100% uptime, then by all means, design a proper high availability system.
Re: (Score:3, Insightful)
And how would you know for sure that it would actually boot correctly the next time you actually *need* to?
Scheduled reboots.
Now you are going to say - scheduled reboots is when you do your kernel upgrades.
The problem with that approach is overloading due to bitrot. Kernel ugprades are not the only reason a system will fail to boot. By upgrading and rebooting you are combining the two goals of patching the kernel and verifying that the system is still bootable, which means potentially more effort troubleshooting if something goes wrong. In the past you didn't have the choice to separate out those two tasks.
Re:Scary analogy (Score:4, Insightful)
You always had a choice. You could decide to reboot on a certain schedule and only update the kernel half the time.
Re: (Score:3, Insightful)
Somehow, I imagined the following conversation.
"Jimmy, are you ok?! There's blood everywhere!"
"OK?! I just had a huge win! I stain-proofed the carpet, so I don't need to worry about the blood anymore."
"Jimmy, are you hurt? Do you need a doctor?"
"What do doctors have to do with cleaning this mess off the carpet?"
(Hopefully my light-hearted tone is apparent. I have nothing but sympathy for fellow cogs st
Re: (Score:3, Interesting)
I keep telling people NOT to reboot their database servers to fix problems but its a reflex reaction that seem to be hard-wired into the DNA of all admins.
Years of training on Windows systems ... they can't help it. Really, they can't, and it's a legitimate tactic in most Windows environments (well, often it's the only thing you can do.) Yes, it has gotten better in recent years, I must admit, but I still find myself having to reboot more often that I think is reasonable.
I've had users who reflexively re-install the application when they see something they don't like. It doesn't help matters when they do so using a four-year-old version they had lying arou
Re: (Score:2)
Re:Scary analogy (Score:4, Insightful)
Get a little imagination, will ya? Here,I'll boil your objection down into two simple premises, for you:
1) In-place kernel upgrades are inherently RISKY to stability, compared with normal reboot upgrades.
2) Reboot upgrades are a LOW COST operation.
You seem to assume that the risk of #1 (upgrade in-place) will always outweigh the cost of #2 (rebooting to upgrade). At the moment, you MAY be correct in that assumption, but we have no basis for any conclusions, yet.
But Ksplice's current business plan is to get ahold of a massive, low-cost testing infrastructure by getting installed by default on as many popular Linux distros (Ubuntu, Fedora, etc.) as possible. Properly executed, a massive testing and development effort should improve KSplice's quality (read: stability) over time.
At some point, if KSplice does it right, in-place kernel upgrades will become stable enough to no longer entail measurably more risk than traditional reboot upgrades. If/When that happens, you'd be a fool to continue reboot upgrading, right? If there's no practical added risk, why should you have to even put up with the inconvenience of a single minute's delay, or the hassle of closing and re-opening all your SSH sessions?
Hell, it's reasonable to imagine that in-place upgrades could even become MORE stable than reboot upgrades (eventually). If that happens, you'd have to be more than a fool to continue rebooting--you'd have to be some kind of technical cargo-cultist, unwilling to offend the Machine Gods by departing from the correct rituals. (There will probably be at least a few of these people--I know some of them, I think.)
For another perspective, consider these:
- guns vs. bows
- automobile vs. horse-and-buggy
- pen/paper vs. typewriter
- typewriter vs computer
- multiprocessing vs. single CPU
Reasonable people expect that the earliest incarnation of a new technology will be buggy, unstable, dirty, explosive, unreliable, or otherwise potentially hazardous. But given time to iron out the bugs, there's eventually a tipping point where the original technology no longer fulfills its basic purpose as well as the new-fangled competitor.
Re: (Score:3)
You must have some damned fine hardware sir!
my laptop boots Win7 in a lot more than that. It's a Core 2 Duo with 4G of RAM.
Hell, my laptop's barely off the BIOS screen in 15 seconds.
Also you might want to consider that the discussion was about time for reboot after a kernel update. Win 7 shutdown and machine restart is probably approaching that.
Win 7 also does that nasty old MS trick that I hate so very much - it displays the desktop pretty quickly after login but you have to wait another couple of minutes
Re:Scary analogy (Score:5, Funny)
They should have stuck with their original slogan: "Using Ksplice is like updating your kernel without rebooting"
Re: (Score:2)
Re: (Score:2)
Well let's be honest here, the risk/gain isn't exactly working out for stable enterprise uses. They want people that can show off all the crazy things you can do with a computer and are willing to risk that their machine could go down. If they get it working for enough people over time, then it'll spread as people like the convenience of reboot-less upgrades. But right now, I'd say their analogy is just right for the market, it's the nerd version of the teenage drivers who play chicken.
Re: (Score:2)
Well let's be honest here, the risk/gain isn't exactly working out for stable enterprise uses.
Exactly backwards.
This feature replicates what mainframes have been doing for years. Specifically because businesses want zero downtime, if possible.
The difference between mainframes and regular PCs is that one mainframe's role is taken on by many PCs. With proper setup, you should have redundancy between the PCs, so you can reboot them one at a time without affecting service. You can't do that if you only have only one mainframe. Even with two, you can only do it if you can afford to double your load. So this might be needed for mainframes, but not for PCs.
Re: (Score:2)
Maybe the bomb will go off if you drop below 50mph...
Re: (Score:2)
Why would it cause a crash? Not having an engine only means you can't accelerate; you can still brake, turn, etc. It's really a lot like restarting your engine while driving, something that's quite simple and safe: clutch in, turn off engine, restart engine, clutch out.
Re: (Score:3, Interesting)
Re:Scary analogy (Score:5, Funny)
It makes a bloody mess? I don't know about that...
Re: (Score:3, Interesting)
I think they wanted something like "It's like R2-D2 repairing and fortifying your X-Wing while you're in flight," but they probably couldn't get the trademarks.
Well, not unless you shove it up your... (Score:2)
Using Ksplice is like changing your tampon while speeding down the highway
As in it's something that's unnecessary (if not nonsensical) for the majority of drivers/users anyway?
how about is linux with memory leaks? (Score:2, Interesting)
how about is linux with memory leaks? is the base os good? what about X? most of the apps? what about apps get stuck in background that need a reboot to unload?
Re: (Score:3, Insightful)
WTF are you talking about? Kill -9 gets rid of apps if you really need too, rebooting is for windows users.
Re: (Score:3, Insightful)
Unless they are in uninterruptible sleep.
Re:how about is linux with memory leaks? (Score:4, Informative)
Re:how about is linux with memory leaks? (Score:5, Informative)
No, the grandparent means uninterruptible sleep.
Processes sleep in a way that can't be interrupted in some cases. For instance, when writing to a file. The logic of that that if it was possible, the application would have to retry the interrupted call, and since a write is assumed to be uninterruptible nobody tries to check if it was interrupted.
This ocassionally creates problems, like when something in the disk subsystem goes wonky, and a write call never returns, leaving the process sleeping and unkillable forever.
There was a patch [lwn.net] to create a killable state, that allows fatal signals to be processed in such cases, since the process would die immediately anyway. I'm not sure how fully is this integrated, but while I remember unkillable processes in the past, I don't think I had any in the last couple of years.
Re: (Score:2, Interesting)
I've seen this happen to processes fairly recently. (A couple months ago.) It wouldn't have bothered me, but it meant I couldn't replace the files they were writing to. My workflow at the time was sufficiently inflexible that this basically required me to reboot to continue working when it happened. I stopped using the program that caused the problem, and improved my workflow to be more flexible. I'd hate to run into the problem again though. I'd love to see a patch like the one you pointed to become
Re: (Score:3, Informative)
I recall seeing this problem with NFS some years back - don't know if it's still the case. The clients with open connections would freeze solid if the server abruptly dropped the connection, for whatever reason, and would stay
that way until either forcibly rebooted or until the server reconnected.
I think that the server could be mounted in a "soft" state to prevent that but was told there was too much of a performance penalty.
Re: (Score:2)
And you can easily put in an icon that forces a process that owns a Window to quit (it looks like a broken window on Ubuntu/GNOME). The only times I have to reboot is when X.org takes out the desktop *and* the keyboard, or when ACPI fails. Those things things still happens too often.
interesting (Score:4, Interesting)
this may be based on Free software (residing in the machine needing its kernel patched), but it appears that patch preparation is based on a subscription service provided by the Ksplice Uptrack people. That's the part which is (selectively) free-as-in-beer. This isn't organic to the kernel or the normal methods of kernel updating.
That means there's libre-free software and a service provided by a non-distro company which is, for selected distros, gratis-free. For now.
The technical description sounds like the ancient OS patching techniques the old mainframes I used to work on used.
And frankly, I'd still feel a little more comfortable with a reboot, since I'd worry a bit about state consistency of kernel and client processes. But, I guess smarter people than me says it OK, so what do I know?
Re:interesting (Score:5, Interesting)
When you have around 1500 production servers to patch, such as with the memmap 0 bug last year, doing them one-by-one, or even in small batches, remotely over IP KVM takes a long-ass time. This is nice for those types of situations.
Re: (Score:3, Funny)
When you have around 1500 production servers to patch, such as with the memmap 0 bug last year, doing them one-by-one, or even in small batches, remotely over IP KVM takes a long-ass time.
One single line using pssh, dsh, dish, or no lines at all when using a very fancied up puppet configuration?
Do you like toggle in boot code over the IP KVM like a PDP-8 or what?
The ability to do something the hard way, does not prove the lack of existence of an easier way.
Re: (Score:3, Interesting)
OK, you see how all of those things still lead to doing a reboot? Now, imagine automating the process AND using ksplice. And I agree that automating the process would have been super awesome, but unfortunately that's just the sort of design process and forethought which was shunned at the place I worked at that time. So I left.
Re: (Score:2)
unfortunately that's just the sort of design process and forethought which was shunned at the place I worked at that time.
Ouch man, ouch. In the networking world we call that situation trying to solve a simple layer-8 problem using a very complicated layer-1 solution (or various other combos of numbers). I'm guessing rebooting was unacceptable because you had no backups / load balancers / load levelers / checkpointers / heartbeat monitor / hot standby disaster recovery / replication systems. Most places, reboots sound like a great time to test that gear, assuming you have it...
Re:interesting (Score:5, Interesting)
Seriously? I patched 5500 linux servers in 24 hours *by myself*, all the while they were churning through collider data from the LHC. This would be, in my opinion, what I would call a production environment. Shortcuts are nice, but sometimes you don't need them if your environment is engineered properly.
Re:interesting (Score:5, Insightful)
Seriously? I patched 5500 linux servers in 24 hours *by myself*, all the while they were churning through collider data from the LHC. This would be, in my opinion, what I would call a production environment. Shortcuts are nice, but sometimes you don't need them if your environment is engineered properly.
That's slightly different. I assume you're at a CMS or ATLAS T2 center and frankly most of those systems were worker nodes that could be taken down for a minute or too for a reboot as jobs were drained off of them and they went idle. A quick reboot and they'll show up in condor or pbs a minute or two later and start processing jobs. The gatekeepers and gateways for the SE would be more complicated but if you got them up within a minute or two, most if not all of the running jobs wouldn't notice.
Re:interesting (Score:5, Funny)
Your post is accurate =) *shakes fist*
Re: (Score:2)
Small correction: I *was* at a T1 center for one of those detectors. I've recently moved on to a less political/bureaucratic organization.
Re:interesting (Score:5, Informative)
And frankly, I'd still feel a little more comfortable with a reboot, since I'd worry a bit about state consistency of kernel and client processes.
This in theory can be a problem, but each kernel update has to be prepared individually, so someone (once again, this is the theory) has looked at the kernel modifications and made sure it won't cause problems. This isn't an automatic thing that can work with any kernel (don't try to use it to go from a 2.4 kernel to a 2.6 kernel), and if there are major changes, say a new scheduler or something, then someone needs to write code that will move the data from the old scheduler to the new scheduler.
Mainly its used for security updates which are probably a line of code changed, or a function changed, and there is no difficulty with inconsistencies (unless maybe someone is in the middle of trying to exploit the buffer overflow, but they avoid that problem by making sure no threads are in the functions that are being patched). This is my understanding of how it works.
Re: (Score:3, Interesting)
Re: (Score:2)
...it appears that patch preparation is based on a subscription service provided by the Ksplice Uptrack people.
Yeah, I'll use it when my distribution vendor of choice is the one doing the preparation.
Re: (Score:2)
I think that (basic) tools for generating ksplice patches are available as Open Source, it's just that you probably don't want to actually have to generate them yourself (and you ought to have someone qualified look over them). It probably makes most sense for a distro to generate them but as that's not happening yet these guys have their niche.
I ran ksplice on an Ubuntu box for a while without problems.
Re: (Score:2)
etiam, Cartago delenda est.
Re: (Score:2)
Or you just need better educated friends.
Re: (Score:3, Informative)
That means there's libre-free software and a service provided by a non-distro company which is, for selected distros, gratis-free. For now.
I like your Latin-based distinction of "free" better than the free-as-in-beer v.s. free-as-in-speech method. I'll have to remember it for the next time I give a speech on OSS at the Roman senate.
Libre is French. The Latin equivalent is liber.
Ubuntu has it already (Score:3, Informative)
Old hat (Score:5, Informative)
Lisp systems did this 30+ years ago: reload new compiled functions, and keep going. New calls go to the new function, old function becomes garbage when no more threads are executing it.
Re: (Score:2, Insightful)
Did you just equate hot-replacing a kernel with adding a function to a runtime environment? Or did I not quite understand? If I understand, then that would be more like, say, upgrading a program without having to reboot, which is unremarkable.
"Next time you open that app, it launches the new version!"
Re: (Score:3, Insightful)
Re: (Score:2)
Did you just equate hot-replacing a kernel with adding a function to a runtime environment?
No, he equated hot-replacing a kernel with hot-replacing a function in a piece of software while the software was still running.
Re: (Score:2)
Have you ever heard of a LISP Machine? Who says that LISP code is not in the kernel?
Re: (Score:3, Funny)
I never said anything of the kind.
But hey, you got to show off that you know what a lisp machine is, so bully for you.
Re: (Score:3, Insightful)
No, I equated hot-replacing sets of functions in a run-time to hot-replacing a kernel (which is a set of functions).
"Next time you open that app" isn't hot-replacement if you are first required to exit the current instance, such that a new process is started.
Re: (Score:3, Insightful)
Re: (Score:3, Informative)
Incidentally, Smalltalk images work similar. For a fun time, open up a Squeak image and start digging around. Now *that* is open source software.
Re: (Score:3, Interesting)
Almost, but not quite. In Squeak, the VM itself is statically compiled (via C), so you can't modify that at run time. With SqueakNOS, you can modify device drivers, but there are still some bits that you can't modify.
This is actually pretty trivial for late-bound languages in general. With LanguageKit, we can replace methods written in Objective-C with methods written in Smalltalk or JavaScript at run time too.
Re: (Score:3, Interesting)
Bizarre that you got modded flamebait, but...
Almost, but not quite. In Squeak, the VM itself is statically compiled (via C), so you can't modify that at run time.
Eh, that's largely picking nits. Did the Lisp machine let you change the hardware running those lisp expressions? No? Then why would you expect to be able to modify the virtual machine running compiled Smalltalk bytecodes?
With LanguageKit, we can replace methods written in Objective-C with methods written in Smalltalk or JavaScript at run time t
Re: (Score:3, Informative)
Did the Lisp machine let you change the hardware running those lisp expressions? No?
Not exactly, but with a Lisp Machine everything other than the hardware was modifiable. The entire run-time environment was written in Lisp. The Squeak VM includes things like the frame buffer, for example, which are statically compiled.
SqueakNOS is more impressive than Squeak, because everything from the interrupt handler layer and up is written in Smalltalk and can be modified. This is pretty close to being equivalent to a Lisp Machine. The only bits you can't modify at run time are the bits that a
Re:Old hat (Score:5, Funny)
diff mainline patch
)
Debugging in space: a case for dynamic systems. (Score:3, Interesting)
Looks like you might be referring to this http://developers.slashdot.org/comments.pl?sid=130313&cid=10876098 [slashdot.org]
exploitable? (Score:2)
How long before it is used to exploit machines, what could possibly go wrong.
Re: (Score:2, Insightful)
Well, if you manage to get your "updates" accepted by the machine's update process, you pwn the machine after the update anyway, even with conventional rebooting updates.
Finally! (Score:2)
I've been waiting for years!
watch uname -r
(from the man page)
Servers (Score:4, Informative)
That's great... (Score:2)
(It's http://xkcd.com/619/ [xkcd.com] for those of you who still have question marks over your heads)
I don't get the point (Score:2)
Okay, so even suppose this is perfectly reliable. Let's say I'm running a high-availability server and can't stand any downtime. Now when my kernel needs an update, I don't have to reboot, great!
So what about when, say, libc needs an update? As long as programs are still using it, they'll be using the outdated version. Am I supposed to restart all programs using libc? That will cause downtime just like a reboot (although maybe a bit less).
Or what about when I need a hardware upgrade? Or there's a
Re: (Score:2)
Slight motto change:
Free as in prostitute.
Re: (Score:3, Informative)
Re: (Score:2)
If it is REAL WORK then you're doing it wrong. :p
Re: (Score:2)
Re:LOL Linux users (Score:5, Insightful)
well for starters, Apple doesn't officially support using Blades or Virtual Machines (they did "allow" VMWare to do it", but only on Mac Hardware) which are where many enterprise Linux installs are living nowdays on IBM, Dell, or HP farms. Apple hardware doesn't really have an enterprise presence or connections to the type of SAN hardware running in many places. You have to ASK to buy a Mac and not many IT departments would allow that. You don't have to ASK to try out a Linux install, you can beg "forgiveness" later on because generally you won't cost the company monie$$, or at least risks they wouldn't have spent money on in the first place. While Macs are cool, as far as enterprise uses, it is still pretty limited. I have several macs (so I'm not a hater) but I could never get my IT manager to take them seriously.
So... (Score:3, Interesting)
Mac user = more money than brains
Windows user =
Re:So... (Score:5, Funny)
Windows user is middle of the road. He has brains and money but not enough of either.
Re: (Score:2, Funny)
Windows user = a poverty of both
Re: (Score:2)
Ok, so they probably ARE zombies, used for all sorts of nasty DDoS attacks or other types of botnets
Re: (Score:2)
Well, Windows users usually don't directly play this game, but are used as playing figures.
Re: (Score:2)
Linux user = more brains than money ...?
Mac user = more money than brains
Windows user =
moooore braaaaains.......
Re: (Score:2)
Linux user = more brains than money ...?
Mac user = more money than brains
Windows user =
Correct, users of free stuff have less money than brains, and Mac users have waaaaaaaaaay more money.
Re: (Score:2)
So that you can mess around with it for free, get comfortable with it, and decide to use it on servers at work where you're willing to spend the money to avoid rebooting. That's my guess, anyway.
Re: (Score:2)
Q. How long will Ksplice Uptrack for Ubuntu Desktop be freely supported? A. Ksplice Uptrack for Ubuntu Desktop 10.04 Lucid is now available and will be freely supported for as long as Ubuntu Lucid is the newest version of Ubuntu. When the next version of Ubuntu Desktop (10.10 Meerkat) is released, we anticipate freely supporting that next version for as long as it is the newest version of Ubuntu.
Re: (Score:2)
Since that bug with the exec-shield patch that occasionally killed a perfectly innocent process. It will probably just add more bugs which no-one will notice or care about,
Wasn't Fedora always intended to be a test-bed for Red hat Linux anyway?