The Completely Fair Scheduler's Impact On Games

eldavojohn writes "We've heard a bit about the completely fair scheduler previously, but now Kernel Trap looks at the implications this new scheduler has for 3D games in Linux. Linus Torvalds noted, 'I don't think any scheduler is perfect, and almost all of the time, the RightAnswer(tm) ends up being not one or the other, but somewhere in between. But at the same time, no technical decision is ever written in stone. It's all a balancing act. I've replaced the scheduler before, I'm 100% sure we'll replace it again. Schedulers are actually not at all that important in the end: they are a very very small detail in the kernel.' The posts that follow the brief article, reveal that Linus seems quite confident that he made the right choice in his decision to merge CFS with the Linux kernel. One thing's for certain, gaming on Linux can't suffer any more setbacks or it may be many years before we see FOSS games rival the commercial world."

Article is misleading

Funny, in the article those framerates for Quake III show CFS beating the pants off of SD.

Besides, the biggest barrier to 3d games in Linux is video card drivers (ATI, I'm looking at you!) as 3D drivers in Linux, even the proprietary ones, have tended to be unstable.

Linus is right one this one, the scheduler is a small part.

Re:Article is misleading

Yeah, I was just gonna post 'Long story short: CFS is better. -yawn-'

Why not just say that, instead of trying to get a bunch of ./ers to RTFA?

Re:Article is misleading

Neither the summary nor the FA did a great job of summarizing the issues.

I agree to some extent. Notably the test specified in the article is "open a game and then sit there without hitting the keyboard." In my mind, this means the game isn't responding to any I/O, so gets pushed to the background, so adding more tasks just means it gets 1/tasks timeslices. Seems reasonable. I'm not sure why the CFS would keep the game running more often than SD if there was no I/O. An interesting comparison would be to see not only the FPS/CPU usage for the game but also for the "loop" tasks. (Those tasks also are not I/O bound.)

Fundamentally I think the name CFS is a little bit odd - how does one define "fair"? In fact, I probably don't want my scheduler to be fair at all - I want it to run the stuff I want fast, and the other stuff it can run slow. That's not very fair.

So, I would say there is not enough information given in the article to tell exactly why the systems had different FPS performance for different schedulers - just looking at that number doesn't tell how it's splitting the time among all the processes.

Re:Article is misleading

Fundamentally I think the name CFS is a little bit odd - how does one define "fair"?

Fair, in this context, means that the scheduler will give all the running tasks CPU time in proportion to their priority (nice level). It follows from this that all the tasks in a given nice level are given equal amount of CPU time, and a higher-priority task (lower nice level) is given more CPU time than a lower-priority one.

SD scheduler (but not CFK, AFAIK) also had idle priority, which means a task that only runs if nothing else at any nice level wants to run. Very useful for running FoldingAtHome.

In fact, I probably don't want my scheduler to be fair at all - I want it to run the stuff I want fast, and the other stuff it can run slow. That's not very fair.

That's what "nice" is for. A fair scheduler respects nice levels, as stated above.

Re:Article is misleading

All schedulers attempt to do that. However the old scheduler design collected statistics based on what process was running at each clock tick. So if a process always yielded right before the clock tick the scheduler would mark it as having used 0 ticks worth of clock time. So it would always stay near the top of the list, and could monopolize the CPU.

A fair scheduler basically times the actual CPU usage. It starts timing when it gives control to the process, and stops timing when the process yields or the scheduler decides to interrupt the process. it tracks processes not by ticks but by actual time used. (This post is based on my understanding of the issue. I may be incorrect.)

Re:Article is misleading

The performance of the "patched SD" mentioned near the bottom show SD to be slightly better than CFS on the test system. However, a few FPS one way or the another really amounts to testing "noise" -- it doesn't mean anything. If there are any problems with 3D, it obviously isn't common to all systems, which suggests to me that the scheduler isn't the problem. Not unless the problem systems have some background job running that the others don't, which is messing up CFS in some way (and this seems unlikely).

Re:Article is misleading

FPS is a poor measure of the feel of a game. I know it's what all the graphics card benchmarks use, and it does do a good job of measuring the total processor and video card throughput, but that's not the most important thing.

The most important thing is the time between you pressing a key and the changed game state being reflected on your screen and how consistent that delay is.

One of the arguments that CK has made about kernel development is that kernel developers have become obsessed with throughput to the exclusion of all else and that this leads to very poor desktop performance because throughput is a poor measure of 'interactivity'. Someone posting 3D game framerates as evidence of one scheduler being better than another is exhibiting exactly this bias.

IMHO latency is a better measure, but still not perfect and it can be hard to measure in some cases.

I don't know enough about the scheduler to know which one is better or which one exhibits particular properties. But I can see that the throughput bias is evidenced in force in the thread the article points to.

And CK is also right that big iron shops care more about overall throughput than any measure of 'interactivity'. IMHO there ought to be some kind of pluggable scheduler system that allows you to completely change the algorithm to reflect the preferred behavior of the computer you're using.

Re:Article is misleading

I believe that you can swap the scheduler out. I might require you to rebuild the kernel but there is nothing stopping a distro, you or anyone else from using a different scheduler.

And actually, If people think this is a problem, Distro's already heavily customize the kernels so switching this out in their particular kernel shouldn't be much of a problem.

Re:Article is misleading

FPS is a poor measure of the feel of a game. I know it's what all the graphics card benchmarks use, and it does do a good job of measuring the total processor and video card throughput, but that's not the most important thing.

I disagree. Responsiveness is important, but I've never encountered a situation where the frame rate was good and the machine couldn't read my keyboard clicks fast enough.

Re:Article is misleading

You are on to something here. CK uses arrays and has a lower context switch time whereas CFS uses red-black trees. What this means is that doing a bunch of "while :; do done" loops that always use their full timeslice is a test that favors CFS as much as possible really.

As an aside, what kind of retard benchmarks a scheduler using a game that is doing nothing and 100% cpu tasks? Put some disk access in there, maybe a set of folders that will easily fit in cache and then find . them. Have some fixed-seed random busy / sleeps of different ratios. Have the game play a demo reel on repeat and record avg *and* min/max fps. Come on Ingo must be somewhat familiar with CK so he must know that these tests where CK is roughly the same are biased toward CFS to begin with. If you are going to say 'look I did these benchmarks and it's a wash' and use that as a justification then at least do good benchmarks.

I think this more than anything else confirms my impression than Ingo is just hacking shit until it kinda works ok. Note that this is exactly the same kind of rationale Linus gave for diss'ing Con so flame off.

Re:Article is misleading

Besides, the biggest barrier to 3d games in Linux is video card drivers (ATI, I'm looking at you!) as 3D drivers in Linux, even the proprietary ones, have tended to be unstable.

I would think that the biggest barrier to 3d games in Linux would be the inherent paradox concerning the lack of 3d games.

No gamers = No profit for game companies = No games being produced.
No games = No gamers = No profit for game companies.

The one thing that I would agree on is that video card support brings game developers and gamers closer to a certain extent. Having better drivers might get both gamers and developers to consider Linux a *little* more. However, even if Linux had terrific video card drivers that were just as good or better than the Windows drivers I still wouldn't consider Linux for games just because there's very few good games available.

Better drivers can only help. But I can't consider that the "biggest" problem. The biggest problem is that there are too few people who use Linux. So video card manufacturers don't care about Linux. Game developers don't care about Linux and lastly (most) gamers don't care about Linux.

I realize there was a lot of bad management decisions involved, but look at what happened to the last company that tried to make a business out of porting titles to Linux (*cough* Loki *cough). I have just about every Loki title that was developed and I really wish they had stayed afloat. Maybe it was bad business decisions and maybe it was just that there was no profit in porting titles to Linux. The situation might be different today and I hope that someone has the desire, balls and money to step up and try what Loki tried 7 or 8 years ago. But Loki's fate did send a clear message. There's no profit in Linux games. John Carmack also said back then that releasing Q3A for Linux saw no profit.

Hopefully as more desktop companies, like Dell, jump on board and push Linux then maybe both the game developers and video card manufacturers will start to see the potential for profit and a result gamers will jump on board. But even Mac has suffered from the same problem for 20 years, and there's way more profit in developing for Mac than Linux. And it shows. There are more commercial Mac games than there are Linux. But both Linux and Mac have next to no games at all when you compare to the titles available for Windows.

Re:Article is misleading

I made the switch from w2k to redhat v8 or 9 about the time that XP came out. At the time I a serious q3 player practising about 5-6 hours a day, playing in leagues, etc. One thing everybody playing did was lower their resolution and raise the refresh rate up to 120 or 125hz, you get smoother view of the game. In both operating systems my machine would easily sustain the 125fps you need in q3, but there were subtle differences in the game.

In w2k at 125hz other players would appear to be moving smoothly. In redhat they would have a constant stutter, like the other players positions were only being updated every 2 or 3 frames, rather than every frame as they appeared to on w2k. This made a difference when playing the game, I ended up moving around distros until I found the preemptive, and low latency patches made the stuttering go away.

For me, fps wasn't ever the problem. It was something else.

FOSS games

Aren't going to happen until artists in the medium, 'good' artists rather, decide to start working for free the same way coders do. Some artists will work for publicity alone, bu they seem to be by far in the minority. On a technical level, I've not seen much problem with linux. Ogre, for example, runs quite smoothly for me.

Re:FOSS games

It depends on the artist.

Are you saying all of "Snow White" was drawn by one person? All 24 fps, 83 minutes worth?

Re:FOSS games

Aren't going to happen until artists in the medium, 'good' artists rather, decide to start working for free the same way coders do. Some artists will work for publicity alone, bu they seem to be by far in the minority.

Of course they're in the minority. For them, there's nothing to be gained in providing their services for free.

The publicity for working on games is almost nonexistent. For example, can you, name any artist that worked on any one of the most popular games? I can, but I know a bunch of artists that work in the games industry.

Besides, artwork doesn't work as FOSS. Unlike code, artwork for games isn't inherently "sharable" - it's designed for the purposes of that game and that game only. Game engines can be used for multiple different kinds of game. Artwork almost always can't. It may be used for sequels (but generally isn't as the requirements change from game to game) but it can't be used across different types of games.

Re:FOSS games

Artists usually have a huge archive of unused material, either done in their free time for practice and fun or for games that never used it due to a redesign or cancelation. Even some of the dummy objects most artists produce, to give the coders something to work with, can be better than actual graphics made by a hobbyist.
I always wondered why they just wouldn't contribute at least some early works to the open source community? Is it maybe just the lack of a good website where stuff like this could be indexed or isn't there a good enough standard license model to release something like that for free? I thought the Creatice Commons license [creativecommons.org] would be quite suitabel for it.

Everybody who spent some time finding a good textured and rigged low-poly character model, preferably with basic animations, on the net for use in an open source game, knows that there is next to nothing available. Well, at least not when I could have needed one about two years ago.
It really doesn't have to be that professional or finished - even that untextured rat someone made a decade ago to have something to shoot at, later to replaced by some creatures, could maybe be of use to someone; and if he textures it, and maybe do a simple animation and perhaps record some sounds, and then uses it in his project, he should give the additional stuff he made to other developers as well.
Soon there will be a nice looking 3D rat with some textures to choose from, various sounds, walking and death animations, etc. and everybody did his part. That's the open source way - why does it seem it's not very common among artits and only coders?

Anyway, I think it could be really just the right website that is missing - some Sourceforge-style page with a nice upload-frontend, where stuff gets properly indexed based on categories, tags and styles and with a feedback option, where contributers can see which projects are using their works. Add some voting to rank it, karma, apply a fair license to it upon upload and I think something like this could really take off.

You won't get good games until you get marketshare

This is irrelevant to the gaming front.

The limit to games on Linux is market share. Its not (much) easier to develop a good 3D game for linux as it is Windows, so why code for 2% of the market when you can code for 92% of the market?

Thus you will only get games where the developer has gone out of their way to ensure complete portibility and provides a port mostly out of courtousy.

The scheduler details are irrelevant for this: what Linux Games need is 10%+ marketshare on the desktop.

Re:You won't get good games until you get marketsh

Sounds like an evil circle.

To get Market Share you need games.
To get Games you need Market Share.

Should it matter?

I would think that 3D games would be considered a 'real time task' (i.e. you must draw the next frame within 1/30th of a second or else it won't look right), and therefore you would want to run them using the real time scheduler (SCHED_FIFO or SCHED_RR). Given that, it wouldn't much matter what fancy scheduling algorithms the non-real-time tasks were using.... your game would always get the cycles it needs when it needs them (up to to the CPU's capacity, of course).

Perhaps people just don't want to run their games as root though....

Modular Kernel

One of the reasons I like the linux kernel is because it's very modular. Why can't both schedulers be included in the kernel, and the person compiling the kernel set which one they want? Kinda like how you could select between Alsa or OSS, or a myriad of other feature that are different but serve a similiar purpose?

Re:Modular Kernel GREAT QUESTION jshriverWVU

Actually the quote from the in the summary from Linus is part of a larger email where hes dismissing the idea of using the plugscheduler (I can't seem to recall the exact name) that would make the CPU scheduler plugin based like the IO scheduler currently is. His reasoning against it were largely BECAUSE what they learned from having the IO scheduler plugin based and its something Linus as well as the subsystem maintainers DON'T want to repeat.

Anyways, you can still just apply Con's patch to the kernel to use his scheduler instead of the old scheduler (and if he keeps maintaining it, you'll be able to use SD instead of CFS). Don't forget that we haven't even had a kernel released using CFS!

Games.. only thing keeping me from linux full-time

I've used linux quite a bit before.. but it is always stuck on a 2nd or 3rd box I have around. I love the new Ubuntu...

except.. gaming support on linux is shitty. The games I have run on linux worked okay - but it took hours to set them up properly. Unreal Tournament 2003 is the last one I played on linux that had a linux port available..

I play games about 10% of the time I use my computer.. nonetheless, bad gaming support is what keeps me from using Ubuntu 100% of the time. I do not feel like having 2 operating systems intalled on my computers.

I'm not a 'gamer' by any means.. but games are important enough to me to keep me using Windows.. I would love to switch.. but I'll switch when the linux coders decide to push for a more compatible gaming system.

Pluggable

That is why Linus should have listened to Con Kolivas when he tried to introduce a pluggable scheduler system. With a modular system we could have CFS and the staircase scheduler and both problems solved.

Re:Pluggable

Linus has already explained why he doesn't like the idea of a pluggable scheduler system. See http://kerneltrap.org/node/14019 [kerneltrap.org]

Multiple choice schedulers

Personally, I'd like things like schedulers to be pluggable. But that's just me. Or maybe not [kerneltrap.org].

Scheduler Nanokernel

Schedulers are actually not at all that important in the end: they are a very very small detail in the kernel - Torvalds
Actually, I'd like to see the OS kernel consist entirely of only the scheduler and the thinnest APIs to secure drivers granting access to the HW. Everything else, including IPC, could be in userspace.

That would make distributing the OS a lot easier. And the simplicity could be a lot easier to secure, to develop for, to customize a deployment for minimum HW (like eg. a "self-winding" 10mW Bluetooth ring with "accessory" features). Practically every device could run the same "OS", with modules bolted on for increased functionality on heavier HW.

Setbacks?

One thing's for certain, gaming on Linux can't suffer any more setbacks or it may be many years before we see FOSS games rival the commercial world.

Linus mentioned somewhere that CFS and SD are both superior to the old scheduler, so in what way can choosing CFS be considered a setback?

Seems like Linux / FOSS should be focusing on...

...something more probable, like making GIMP a killer Photoshop clone or something. Games is just such a competitive and tricky market, and I wonder if there would really be that many Linux people willing to pay for them. Seems like many Linux people in the desktop space are high school and college kids.

Linux and Games

I've figured out how to get games running with Linux!

1. Set up your Linux system. Use onboard video and don't overspend on your processor.
2. Buy a PS2, a Wii, or a 360.
3. Play games on your game console and do everything else on Linux.

Be glad they chose CFS over SD then.

Sayeth Kenneth.

Alright, Just got done with some testing of UT2004 between 2.6.23-rc1 CFS and 2.6.22-ck1 SD. This series of tests was run by spawning in a map while not moving at all and always facing the same direction, while slowing increasing the number of loops.

CFS generally seemed a lot smoother as the load increased, while SD broke down to a highly unstable fps count that fluctuated massively around the third loop. Seems like I will stick to CFS for gaming now.

--
Kenneth Prugh

Sayeth Matthew

My experience was quite similar. I noticed after launching the second loop that the FPS stuck down to 15 for about 20 seconds, then climbed back up to 48. After that it went rapidly downhill. This is similar to other benchmarks I've done of SD versus CFS in the past. At a "normal" load they're fairly similar but SD breaks down under pressure.

The only other thing of interest is that the -ck kernel had the WM menus appear in about 3 seconds rather than 5-8 under the other two.

The only chap that says otherwise doesn't post his results.

From TFA [kerneltrap.org]

This was already a known issue

Con's SD scheduler is light years ahead in terms of desktop computing and gaming. The "Completely Fair" Scheduler is not. Linus made a poor political decision that once again favors the multi-CPU, gobs of memory, server architecture that normal people don't give two shits about.

Vague generalizations on OS suitability

Windows is for games and office work.

OS X is for artists and people really into style.

Linux is for hackers and various niche environments.

Linux does not need to support gaming. Windows does that quite well. Anyone that wants to game can dual-boot with Windows, or buy a console. Linux will not support gaming, for the same reasons AIX or Solaris are not chock full of gaming goodness. It isn't required or desired, and the OS is far more suitable for other, often more "serious," applications.

any *more* setcbacks?!?!?!

gaming on Linux can't suffer any more setbacks

In the same way that the pope can't get cervical cancer?

The past is the map to the future

When there was more than one OS that ran on different HARDWARE, games could be differentiated and selling points could be made for buying an Amiga, or an Atari over the Mac and PC of the time.

Other than Security, to the average user who is using Windows there just is'nt the "Whoahh" that people used to get when somebody back then saw the Amiga or Atari. Compelling reasons just don't exist for the average user to switch from windows to LINUX except maybe fear of viruses and malware.

For games to take off on LINUX there needs to be a game done on LINUX for LINUX ONLY that everybody wants and does something on LINUX that CAN'T BE DONE on windows. When I saw what the Amiga could do commpared to the PC I KNEW why I wanted one.

Granted the Amiga was as much a hardware advancement as OS, but the point was the average user could understand why they needed it.

The X Factor

I keep wondering...X is a single threaded server, communicating with a (generally) single threaded game. Worse, wine inserts the wineserver process, so I have three single threaded things trying to synchronize to get interactivity. A low latency event like a keypress might require all three processes to be scheduled in succession, to get a response on the screen. A poor man's way to do this is with the kernel's scheduler, but a far superior way to do it is to have multiple threads in the X server. Scheduling an interactive event isn't hard. Getting crap on the screen in the same scheduling timeslice is hard (impossible?) since it requires a second scheduling point. As I understand, this is how BeOS achieved substantial interactivity in the presence of load -- my having a multi-threaded graphics server *and* kernel.

So, how much can be gained by rewriting X, or going to a different graphics server? Or do I completely misunderstand the effect of X?

-- Bob

The Scheduler That Lived will live

Both SD and CFS are superior to the old one. Between the two, the one that gets merged into mainline will be the best eventually.

There were a lot of testers when SD came out, because it clearly beat the pants off the old one, and that was exactly why Ingo went ahead to throw his own version of a fair scheduler - otherwise his code would not survive.

Which one is better, SD or CFS? Technically, it was hard to say, but it's not about technology - it's like the browser war, the one with the bigger market share wins. CFS has been merged into mainline and gets the biggest exposure, so there are hell a lot of more testers and developers attention, and with a genius like Ingo, it's only a matter of time that CFS will perform better than SD - the latter has been simply ignored since CFS was out.

So arguing which one is technically better is not the point. It's like people kept arguing Netscape was better than IE. But the fact is IE eventually outperformed Netscape even technically.

A lot of people who watched the whole thing on LKML consider the decision of merging CFS was unfair, but few really worried that we'd have a worse scheduler in the long run. Get your Cause and Consequence right.

Jitter is important

Usability research has shown, that variation in waiting time is actually a bigger irritation for users than waiting time itself.

I have seen several projects, where user interface response time problems have been "improved" by making adding a minimum response time. The average response time increases, but variation decreases, and the user often reports the program as having become faster... the logic to this seems to be, that the user wants the user interface to have a predictable response.

I think the reference for this is Søren Lauesens books about usability programming, but I cannot remember for sure right now.

CFS's impact is.. none!

Yeah they are too busy bitching about the difference between CFS and SD.

And then the news post here, says "Linux cannot suffer any setbacks in gaming". I think you'll find that compared to the original scheduler, CFS pretty much rocks for gaming. As much or less than SD, who the fuck cares?

It's better than the original scheduler, so where's the setback?

If it's not as good as SD, oh well, cry me a river. I don't agree with Linus' "there is no maintainer" idea, but more the concept that CFS removes more lines from the kernel than it replaces, and does a better job, whereas SD adds complexity for roughly the same effect. What could be a perfectly good technical reason in previous LKML posts got turned into politiking.

Difference between SD and CFS.. fractions of a frame per second. WOW. That really means Linus made the wrong decision! The impact on games, where 1/500th of a second really MAKES A DIFFERENCE is too high! Put the old scheduler back you fucking crazy-ass Finn!!

Let me say it again: Ingo sucks.

Lets not forget the intellectual theft perpetrated by Ingo Molnar. No matter how things turn out the guy is a liar, a thief, and a cheat. He sucks.

Ingo fought long and hard for the status quo. Complaints against the current O(1) scheduler (author: Ingo) were met with flames and ignorance. Con spent a year developing a new scheduler and proving that it was better than the scheduler that Ingo wrote. Ingo spent the entire year ignoring and denying Con's scheduler, but one day he saw the light and wrote his own knockoff in "62 hours" and gets it merged the next week. This is bullshit.

The end result is that Con has been driven from Kernel development by asses like Ingo. Whether CFS is better than SD is irrelevant. What has happened is the guy who spawned the entire process of getting an improved scheduler in the kernel and spent a year overseeing and improving it has his concept stolen by Ingo. Ingo did not want a new scheduler, he wanted his name to be on the current scheduler. When he saw that disappearing he had to take drastic action and he "authored" a new, improved scheduler. All for his own glory.

Ingo sucks.

Obstacles to commercial gaming on Linux.

The real obstacles with developing a commercial game on Linux isn't the performance or technicalities of developing the game like it was years ago; it's distribution and support.

Linux has plenty of horsepower and driver support these days to run even the most demanding 3D games and the development platform is consistent enough with Windows to make porting issues or cross-platform development a manageable, almost transparent task.

The big obstacle is distributing and supporting a game for Linux. For a third party developer such as a commercial game developer, packaging and releasing a game for Windows is a relatively easy, standardized task. A commercial developer knows that once they develop the game for Windows it's going to work as expected on the greater majority of consumer Windows machines in existence; even those running older versions of Windows and most likely will run on future versions of Windows. The commercial game developer generally does not have to worry about new hardware coming not working because of lack of drivers or new releases of the Windows OS coming out every 6 months that causes their product to stop functioning and needing fixed constantly. The commercial game developer doesn't have to be concerned with what other Windows developers are doing and whether changes going on in another area of the OS or applications being developed are going to effect their game development. Furthermore, the commercial game developer doesn't really have to spend extra effort dealing with multiple versions of Windows; for the most part Windows is Windows.

Contrast this with Linux, where there are literally dozens of recognizable distributions, none of which have an appreciable majority marketshare and many of which change dramatically every few months. Trying to maintain a game and keep it working from one version to the next of a single distribution is a sizeable task; trying to do that for dozens of distributions becomes prohibitive which is what a commercial game developer would have to do if they truly intended to support their game properly. And after all this effort, what would the effective sales increase of their game really be? The economics of properly distributing and supporting a game just isn't worth it.

If Linux really wants to be a practical target for commercial game development, it needs to bolster itself in the areas of distribution, support and platform compatibility stability. This will in turn not only make it more feasible for commercial game developers to consider targeting Linux as a valuable platform but improving these same factors would apply to all application developers and increase marketshare and commercial viability of Linux. If that is the desired effect, of course.

Why scheduler debates might be so popular.

I think one of the reasons scheduler debates might be so popular, is that it's one just a few parts of an operating system that are easy to understand and also heavily emphasized in introductory computer science classes. Schedulers do matter in terms of user-experience, but in terms of overall performance they theoretically shouldn't make that much of a difference(except for the cost of switching tasks). I think that's what Linus is getting at when he says it's a small part. Previous schedulers worked fine, but this one allows a little bit "smoother" user experience.

That said, here is one example of how schedulers affect the lives of users:

I tend to play with the nice values of shared servers at my work place(It's nice to be the administrator.) when I'm compiling on the same machine as someone else who is running a simulation.. The compile time for my primary project(on an unburdened system) is around 20 minutes for a full recompile, while the simulation might take days to finish. If my compile process is equally nice as the simulation process it takes approximately double the time to complete, so I end up reading Slashdot for an extra 20 minutes and the simulation runs for about an extra 20 minutes. That's great on a day when I don't have deadlines to meet. When I give my compilation priority, I may see the compilation finish in 25 to 30 minutes and the simulation will run an extra 30-35 minutes as a result of my compilation. So I save 15 minutes on my compile while my coworker never notices the difference even if I do a full compile 10 times in one day. So the scheduler allows me (the interactive user) to work more productively.

Anyway, I don't know if that example was needed, but I hope it was helpful for someone.

That big a deal?

Did Linus all the sudden lock out all other potential schedulers? Was the choice taken away? Root usually needs to install such games, and it's no big deal to change them at runtime, so relax, aye?

Other OS schedulers?

Does anyone know what types of scheduling Windows XP and OS X use? It seems OS X grants highest CPU priority to the application process whose window is in focus, which seems to work very well. Would any type of communication between X.org and the linux kernel be possible to enhance process scheduling? Or is this already done? It seems natural that the (user) interface should talk to the kernel, even at a low level, to ensure overall OS responsiveness. After all, strictly speaking the user never really uses the operating system-- they only use the interface.

Priority Queueing and Hashing as an Option

It might be nice if there was a mechanism whereby a priority queue was maintained with pointers into the red-black tree or other data structures used by the CFS module so that the superuser could designate that certain programs, identified via cryptographic hash, are special and should receive first consideration for scheduling while they are running. For example, during normal operations the priority queue would be empty and the CFS would handle scheduling in the usual fashion. However, once certain processes were loaded into the CFS *and* put into the priority queue (not every process goes into the priority queue, only the designated special ones), the CFS would know to check the priority queue when scheduling to see if there were any special processes running so that it could give those processes first crack at the processor and other resources in order of their priority (there could be multiple special processes running at the same time as well). In gaming, for example, it is usually the case that the user is focused entirely on the game and not on other processes running in the background so it would make sense for the game program's hash to be in the special priority queue so that it would receive top billing while it was running. It would also be possible to use this mechanism for other more usual processes (say on a server), but the CFS alone would probably be sufficient for most server type scenarios. The special scheduler add-on would really only be useful for special scenarios such as gaming, graphics rendering, and other long running and high demand processes. Perhaps something similar already exists, but I am not a Linux kernel programmer so maybe I just don't know about it...

What about "nice?"

Of course a fair scheduler will cause a single process to yield its CPU time- that's the point. It tries to spread the system resources out among as many similarly-niced processes as possible. Now if you want your game to run better than something else, why not renice the game -1 or renice the other stuff +1? I do this all the time to be able to run extremely compute-intensive processes in the background on my system and it makes them pretty much invisible to me (as long as they don't eat up all the RAM or do a lot of HDD I/O, of course.)

Absolute Priorities

Is there an option or a patch for Linux that runs the highest priority tasks first and then the lowest ones? i.e. task PuttingOutFires gets all the cpu even if MakingDinner and TakingOutTrash want time?

Well...

The thing about Linux is, why doesn't someone just make a pro-gamer distro? It's open source and GPL, so there is no reason why this can't be done.

Optimise everything for 3D acceleration etc. Bundle a bunch of open source games for the heck of it.

Or is there already such a thing? If so, what is it?

alternative solution

Here's an idea: why doesn't someone in the know create a 'game development' library base which could be dropped somewhere in /usr (or /opt or whatever's standard today) and be used as a stable-as-debian, cross-linux library base for games. Include all the standard things like SD, and optionally have them built to local base libraries - or whatever else is needed, I'm not really up on specific gaming libraries. But, at the least, it'd create a stable environment which game developers (and proprietary application developers, for that matter) could develop towards in the same fashion that they do for Windows.

Another idea, or possibility: while I'm not 100% sure it's possible or preferable, why not use a 'scheduler scheduler' to allow for active switching between scheduler profiles in a manner similar to how ACPI and APM work? I'm not going to want the same behavior while doing 'office' work as I would while watching a movie or gaming. It'd not make much sense on a server, I don't think, but on a workstation the ability to switch between task-oriented micro-schedulers seems like a pretty good idea, even if there's a several-percentage point processor performance hit. I imagine the loss would be made up in more task-appropriate performance if the user could be allowed to customize it: use x scheduler when x1 and x2 binaries are running, and y when y1 and y2 are running...

nice?

I didn't see any mention of using nice to adjust the priority. It would make a big difference to set the nice level of X11 and the game to -20, explicitly telling your scheduler that your game is important, unless of course, you think it's important to have your email server responsive while playing Q3.

Re:Hmm

The CFS (Completely Fair Scheduler) won't, but maybe the new FCS (Flux Capacitance Scheduler) might. Or already did.

Strange, I've been gaming in Linux for years.

I have Windows XP and Gentoo Linux running side by side, and strangely, Gentoo scores 10 to 12 FPS faster in World of Warcraft, Warcraft III and even Doom 3. Granted they are commercial games, but if they can run in WINE that fast, I wonder what a direct Linux implementation would do. I just love seeing folks buying the headlines instead of blazing their own paths.

That's why the world is in the shape its in... the majority is always waiting for someone to save the day. You want desktop Linux? Then make it your desktop. Otherwise stop bitching and post some valid comments.

Re:Strange, I've been gaming in Linux for years.

WOW DO have native linux ports

Uh...no. WoW does run nicely under Wine, but that's not a port.

Re:Strange, I've been gaming in Linux for years.

Performance improvements could come from i.e. unsupported and thus unvirtualized eye-candy DirectX features which would have a negative impact on performance under normal circumstances. Anyway, afaik all the mentioned games have OpenGL renderers, but I assume that you use the stanard DirectX renderer in World of Warcraft.

On the other hand, performance drawbacks because of wine's virtualization are very small but naturally they do exist. Adding an extra layer of wrapping takes time. Of course, maybe wine's handling of win32-specific calls and systems is more efficient than Microsoft's implementation in their operating systems ;-)

Re:Strange, I've been gaming in Linux for years.

Well, the issue at hand here isn't throughput, it is consistency.

An efficient scheduler gives processes in general the most bandwidth. A fair scheduler gives processes in a priority class the most equal bandwidth shares. A real time scheduler gives any given process the most predictable wait for bandwidth.

Each of these notions is somewhat different. Achieving a high frame rate over the course of a test on an unloaded system tells you nothing about the scheduler, other than perhaps it is not truly awful. On a moderately loaded system, the scheduler may be giving your game more than its share of CPU time, but if from time to time your game seizes up for a fraction of a second, it would be an irritation, even if on average it's getting enough bandwidth to give you a good playing experience. At the same time, this situation would be fine for data processing applications like image analysis, where an operation might take several seconds, or even minutes to complete. As long as the process gets plenty of cycles over the course of the operation, it's ok, even though your operation might have "frozen" for up to a second in the process.

Re:Strange, I've been gaming in Linux for years.

*** I wonder what a direct Linux implementation would do***

A perfectly reasonable question, but the answer may well be "about the same". The NE in wiNE istands for "Not an Emulator". In a sense, WINE *IS* a native Linux graphics implementation albeit aided or hindered by using the Windows API interfaces. If I recall the WINE documentation correctly it says that WINE is sometimes faster than Windows on the same hardware and application and sometimes slower.

Here's a link http://linuxhelp.blogspot.com/2006/02/wine-vs-wind ows-xp-benchmarks.html [blogspot.com] that seems to say roughly the same thing.

Re:Um

Ya think?

Of course, because as we all know, it is only possible to write FOSS for Linux, and not for any other OS. Like, say, Windows, for instance.

Re:Mods seem kind of *cranky* today

That's some funny shit. My line of thought started out the same but --- 'hit someone over the head with it?! Lucky you're on slashdot else your GF would be pissed!