The New Linux Speed Trick 426
Brainsur quotes a story saying "
Linux kernel 2.6 introduces improved IO scheduling that can increase speed -- "sometimes by 1,000 percent or more, [more] often by 2x" -- for standard desktop workloads, and by as much as 15 percent on many database workloads, according to Andrew Morton of Open Source Development Labs. This increased speed is accomplished by minimizing the disk head movement during concurrent reads.
"
1,000 percent? (Score:2, Insightful)
Linux Devices has an article on the 2.6 network features here http://linuxdevices.com/articles/AT7885999771.html [linuxdevices.com]
Re:SCSI (Score:2, Insightful)
Re:1,000 percent? (Score:4, Insightful)
I suppose that since database data is generally grouped together and read in a big chunk there's less room for improvment.
Re:Cool (Score:2, Insightful)
It's early, but did read/write heads suddenly develop intelligence while I was napping?
A.
Re:Why not combine those two methods? (Score:4, Insightful)
what would you have expected the kernel 2.8 to bring you ?
</joke>
Basically, I think this is like the windows system settings : you either privilegiate front end services (GUI) or back end services (apache, etc) but you cannot do both because some would be optimized for reactivity, the others to handle the workload... like a ferrari and a truck... this doesn't work nor excel in the same way.
But how? (Score:2, Insightful)
I was always under the impression that modern hard drive designs hide the physical disk bits and pieces from the PC. So how can software predict where the heads are?
Re:1,000 percent? (Score:5, Insightful)
Just as 50% is half, but 50% improvement is three halves as good.
Retro is still cool ? (Score:4, Insightful)
All the wonderful stuff like disk seek optimisation, interleaved memory (Even MMU came to the moden computer about 15 years after everyone else had it) were technologies that made systems stand out from each other.
Because of the speed of things these days, lots of that tech has been largely ignored, until now when we're starting to hit hard performance barriers again. Now we have to invent the technology og the '70s all over again. It's nice to see all this stuff comming back though.
what's old is new again (Score:1, Insightful)
Real benefits... (Score:4, Insightful)
Let me start by claiming that optimizing desktop performanceis all about optimizing I/O patterns (contrary to what all Gentoo users think :P). My KDE startup is about three times as fast when I everything is in the disk cache, so it is clear where the bottleneck. (Just try logging in to KDE after boot, then log out and log in again.) A concentrated effort of
There has been a lot of discussion about this on the kde-optimize list (with Andrew Morton participating), so maybe we can hope that KDE 3.3 will offer some improvements.
As an aside, yes, we all hate the windows registry, but I think we should admit that for boot time optimization it is the right thing to do (having everything in one file that is layed out in one contigious block on the disk.)
Speed-ups (Score:3, Insightful)
Alternatively, have multiple read-heads on a single arm. 3 would be a good number. The idea here would be that you could pre-seek either side of the disk, before finishing a read by the currently-active arm.
Re:SCSI (Score:3, Insightful)
Re:Cool (Score:2, Insightful)
I think this would work to minimize the impact of a slow access drive in a heavily multitasking system too.
Oooh.... evil Microsoft must be thwarted! (Score:2, Insightful)
It's kind of sad that the free software advocates sometimes get so carried off by their pathological hatred for Microsoft and corporations that they don't see that they're about to become "the enemy" themselves.
Free is free. If you start to restrict the use and availability of your code by requiring the release of any modifications to the public, it's not free code anymore - no matter what RMS says.
Schedule for Interactivity (Score:2, Insightful)
Surprise, the Mac has the same reactivity problem now thanks to its Unix (Mach) kernel, while the previous Mac OS 9 crashed regularly, couldn't multitask, but has a much snappier user-experience. Apple has been adressing this issue - which they recognize- for 2 years now, and have almost but not quite fixed it with their current Panther release.
It is time we found a way to benchmark a user experience in order to prevent over-optimisation for number-crunching.
Most of my posts get marked down as trolls - think hard: How can you solve a problem if you refuse to admit it exists ?
Re:Oh, come on... (Score:3, Insightful)
This article is just another example of such a case. The anticipatory scheduler algorithm was not published until 2001. With Linux you get these sorts of benefits integrated in reasonble time, with Microsoft you have to wait between W2K and Longhorn to get any of the new goodies.
Slackware! (Score:3, Insightful)
However, I wouldn't even try that on RedHat or Mandrake without having the .config file and a list of distribution specific patches.
This was on a Celeron 1GHz laptop, and honestly, I couldn't tell the difference in speed beyond any custom compile. Custom meaning unnecessary device drivers are removed, and the ones that I need are compiled in (as opposed to remaining modularized).
Re:I've noticed it... (Score:3, Insightful)
From the sound of it you're talking about perceived speed for a desktop user, as opposed to measured server throughput. If this is the case, I imagine the biggest speed increase comes from the fact that (I believe) 2.6 offers far lower latency in the kernel, allowing it to be preempted in more cases. This will make a desktop environment far more responsive.
Re:Amiga Disks (Score:3, Insightful)
I've always wondered why there wasn't something in the OS to force this behaviour, Ie, making sure that App 2 access to the disk is queued until app 1 has finished. Isn't this one of the reasons Windows takes ages to boot? (many processes all competing for the one disk resource?).
You run into a problem where you don't know when app 1 has finished loading in order to start loading app 2. Why, because a loading app looks no different than a running application. You could possibly get around this by having the app set some flag that says "Hey I'm starting up, give me a few extra bashes at the disk before making me yield." The problem with that solution, is what happens if that application never releases the flag. You've got an application that has nearly exclusive access to the disk, and may not be interested in giving it up.
Re:SCSI (Score:3, Insightful)
If I'm writing a user-land program which memory maps a large file, modifies it in memory - then uses msync() to write to disk - what can be safely assumed?
Re:SCSI (Score:3, Insightful)
As for order of pending writes, I don't think you get to have a say on any particular writes, but you can sync after writing to commit everything so far.
See also man 2 sync.
Re:Cool (Score:0, Insightful)
As I've said elsewhere in this story, the linux community can't bitch about not being on everyone's computer if energy is wasted on projects like this. Imagine if everyone who spent time working on things like this vented that energy into making an OS people want to use... Linux would be on everyone's desktop. First things first - get people using the OS. Second step - improve it for them. I mean, no-one even knows if Joe Public is going to want any of these features when they're using their linux box to surf the web. Microsoft developers/hardware providers and their ilk can afford to make custom drivers because millions more people use their software. At the moment, in the linux world, it's common for 10 people to work on a project that will only be used by 10,000 people. On that scale, for Linux to be a world-leading desktop OS, the dev teams would have to be in their millions, and working on the same project. That's obviously not going to happen, so the ratio of developers/users has to be addressed before real growth can happen. Shaving 10% off access times isn't the best way to do that.
Call me a troll or whatever - I don't care. I do care that I can see why linux isn't as popular as it is, and I do care that no-one listens, or is even prepared to engage in a discussion. I'm more than willing to accept I'm wrong, but as soon as I raise points like this I get some tux-licking fanboy triyng to cram an entire O'Reilly library down my throat.
Re:SCSI (Score:5, Insightful)
ATAPI is SCSI-over-IDE however.
I wrote the IDE/ATA drivers for the Amiga. The Amiga SCSI drivers accepted "SCSIDirect" commands from applications. Internally, all IO commands were converted to SCSIDirect commands for execution. To implement ATA, I added a SCSIDirect->ATA translator (which wasn't that hard - about 3 weeks from start to working, booting system - and I implemented just about all SCSI commands even semi-reasonable (all of CCS I think, plus quite a bit).
Doing it this way made implementing support for ATAPI CDROMs (something I did as a contract after Commodore folded) Very Easy.
Re:Real benefits... (Score:2, Insightful)
This is oh-so-wrong in so many ways. Let me name just a couple of them:
1. Because damned near everything causes changes in the registry, the registry is NEVER layed out in one contigious block on the disk. Instead, the constant editing forces it to be scattered across the entire disk.
2. I can make a better argument that it actually slows the boot process because, in order to get the parameters associated with just booting, the system must read thousands upon thousands of other registry entries that have nothing to do with the boot process! I would much prefer a smaller file that only has the parameters need for booting and NOTHING ELSE!
Coupled with the inherent risk of "placing all your eggs in one basket", the registry most decidedly is NOT the right thing to do!
Re:Yawn (Score:2, Insightful)
Re:SCSI (Score:3, Insightful)
Not sure what happens if you try this on a network file system, whether it forces the hosting computer to flush to disk, or if it only forces the local computer to flush to the host.
Depends on the server - you can request it, but the server isn't obligated to comply.
Re:Cool (Score:3, Insightful)
In general, kernel hackers don't write pretty GUI's or design highly usable interfaces, and HCI experts don't optimise low-level scheduling algorithms. These are orthogonal parts of Linux, and your belief that improving the kernel's low-level efficiency somehow makes the end-user experience worse is frankly ridiculous.
Of course maybe you're just a very good troll and you're successfully wasting mine and others' time
Re:SCSI (Score:3, Insightful)
Trying to do all the reordering in the OS (as suggested in several posts here) seems like a good idea, but ignores some issues:
Tagged queuing in SATA/etc drives is a step in the right direction, though last I checked it wasn't equal to TQ in SCSI. Native Command Queuing in SATA will probably give similar performance to TQ in SCSI.
This PDF on Native Command Queuing [serialata.org] is even more interesting.