The Completely Fair Scheduler's Impact On Games

eldavojohn writes "We've heard a bit about the completely fair scheduler previously, but now Kernel Trap looks at the implications this new scheduler has for 3D games in Linux. Linus Torvalds noted, 'I don't think any scheduler is perfect, and almost all of the time, the RightAnswer(tm) ends up being not one or the other, but somewhere in between. But at the same time, no technical decision is ever written in stone. It's all a balancing act. I've replaced the scheduler before, I'm 100% sure we'll replace it again. Schedulers are actually not at all that important in the end: they are a very very small detail in the kernel.' The posts that follow the brief article, reveal that Linus seems quite confident that he made the right choice in his decision to merge CFS with the Linux kernel. One thing's for certain, gaming on Linux can't suffer any more setbacks or it may be many years before we see FOSS games rival the commercial world."

Article is misleading

[RailGunner (554645)]

Funny, in the article those framerates for Quake III show CFS beating the pants off of SD.

Besides, the biggest barrier to 3d games in Linux is video card drivers (ATI, I'm looking at you!) as 3D drivers in Linux, even the proprietary ones, have tended to be unstable.

Linus is right one this one, the scheduler is a small part.

Re:Article is misleading

[Aladrin (926209)]

Yeah, I was just gonna post 'Long story short: CFS is better. -yawn-'

Why not just say that, instead of trying to get a bunch of ./ers to RTFA?

Re:Article is misleading

[ThosLives (686517)]

Neither the summary nor the FA did a great job of summarizing the issues.

I agree to some extent. Notably the test specified in the article is "open a game and then sit there without hitting the keyboard." In my mind, this means the game isn't responding to any I/O, so gets pushed to the background, so adding more tasks just means it gets 1/tasks timeslices. Seems reasonable. I'm not sure why the CFS would keep the game running more often than SD if there was no I/O. An interesting comparison would be to see not only the FPS/CPU usage for the game but also for the "loop" tasks. (Those tasks also are not I/O bound.)

Fundamentally I think the name CFS is a little bit odd - how does one define "fair"? In fact, I probably don't want my scheduler to be fair at all - I want it to run the stuff I want fast, and the other stuff it can run slow. That's not very fair.

So, I would say there is not enough information given in the article to tell exactly why the systems had different FPS performance for different schedulers - just looking at that number doesn't tell how it's splitting the time among all the processes.

Re:Article is misleading

[ultranova (717540)]

Fundamentally I think the name CFS is a little bit odd - how does one define "fair"?

Fair, in this context, means that the scheduler will give all the running tasks CPU time in proportion to their priority (nice level). It follows from this that all the tasks in a given nice level are given equal amount of CPU time, and a higher-priority task (lower nice level) is given more CPU time than a lower-priority one.

SD scheduler (but not CFK, AFAIK) also had idle priority, which means a task that only runs if nothing else at any nice level wants to run. Very useful for running FoldingAtHome.

In fact, I probably don't want my scheduler to be fair at all - I want it to run the stuff I want fast, and the other stuff it can run slow. That's not very fair.

That's what "nice" is for. A fair scheduler respects nice levels, as stated above.

Re:Article is misleading

[Tacvek (948259)]

All schedulers attempt to do that. However the old scheduler design collected statistics based on what process was running at each clock tick. So if a process always yielded right before the clock tick the scheduler would mark it as having used 0 ticks worth of clock time. So it would always stay near the top of the list, and could monopolize the CPU.

A fair scheduler basically times the actual CPU usage. It starts timing when it gives control to the process, and stops timing when the process yields or the scheduler decides to interrupt the process. it tracks processes not by ticks but by actual time used. (This post is based on my understanding of the issue. I may be incorrect.)

Re:Article is misleading

[complexmath (449417)]

The performance of the "patched SD" mentioned near the bottom show SD to be slightly better than CFS on the test system. However, a few FPS one way or the another really amounts to testing "noise" -- it doesn't mean anything. If there are any problems with 3D, it obviously isn't common to all systems, which suggests to me that the scheduler isn't the problem. Not unless the problem systems have some background job running that the others don't, which is messing up CFS in some way (and this seems unlikely).

Re:Article is misleading

[Omnifarious (11933)]

FPS is a poor measure of the feel of a game. I know it's what all the graphics card benchmarks use, and it does do a good job of measuring the total processor and video card throughput, but that's not the most important thing.

The most important thing is the time between you pressing a key and the changed game state being reflected on your screen and how consistent that delay is.

One of the arguments that CK has made about kernel development is that kernel developers have become obsessed with throughput to the exclusion of all else and that this leads to very poor desktop performance because throughput is a poor measure of 'interactivity'. Someone posting 3D game framerates as evidence of one scheduler being better than another is exhibiting exactly this bias.

IMHO latency is a better measure, but still not perfect and it can be hard to measure in some cases.

I don't know enough about the scheduler to know which one is better or which one exhibits particular properties. But I can see that the throughput bias is evidenced in force in the thread the article points to.

And CK is also right that big iron shops care more about overall throughput than any measure of 'interactivity'. IMHO there ought to be some kind of pluggable scheduler system that allows you to completely change the algorithm to reflect the preferred behavior of the computer you're using.

Re:Article is misleading

[sumdumass (711423)]

I believe that you can swap the scheduler out. I might require you to rebuild the kernel but there is nothing stopping a distro, you or anyone else from using a different scheduler.

And actually, If people think this is a problem, Distro's already heavily customize the kernels so switching this out in their particular kernel shouldn't be much of a problem.

Re:Article is misleading

[Bob-taro (996889)]

FPS is a poor measure of the feel of a game. I know it's what all the graphics card benchmarks use, and it does do a good job of measuring the total processor and video card throughput, but that's not the most important thing.

I disagree. Responsiveness is important, but I've never encountered a situation where the frame rate was good and the machine couldn't read my keyboard clicks fast enough.

Re:Article is misleading

[by Anonymous Coward]

You are on to something here. CK uses arrays and has a lower context switch time whereas CFS uses red-black trees. What this means is that doing a bunch of "while :; do done" loops that always use their full timeslice is a test that favors CFS as much as possible really.

As an aside, what kind of retard benchmarks a scheduler using a game that is doing nothing and 100% cpu tasks? Put some disk access in there, maybe a set of folders that will easily fit in cache and then find . them. Have some fixed-seed random busy / sleeps of different ratios. Have the game play a demo reel on repeat and record avg *and* min/max fps. Come on Ingo must be somewhat familiar with CK so he must know that these tests where CK is roughly the same are biased toward CFS to begin with. If you are going to say 'look I did these benchmarks and it's a wash' and use that as a justification then at least do good benchmarks.

I think this more than anything else confirms my impression than Ingo is just hacking shit until it kinda works ok. Note that this is exactly the same kind of rationale Linus gave for diss'ing Con so flame off.

Re:Article is misleading

[garett_spencley (193892)]

Besides, the biggest barrier to 3d games in Linux is video card drivers (ATI, I'm looking at you!) as 3D drivers in Linux, even the proprietary ones, have tended to be unstable.

I would think that the biggest barrier to 3d games in Linux would be the inherent paradox concerning the lack of 3d games.

No gamers = No profit for game companies = No games being produced.
No games = No gamers = No profit for game companies.

The one thing that I would agree on is that video card support brings game developers and gamers closer to a certain extent. Having better drivers might get both gamers and developers to consider Linux a *little* more. However, even if Linux had terrific video card drivers that were just as good or better than the Windows drivers I still wouldn't consider Linux for games just because there's very few good games available.

Better drivers can only help. But I can't consider that the "biggest" problem. The biggest problem is that there are too few people who use Linux. So video card manufacturers don't care about Linux. Game developers don't care about Linux and lastly (most) gamers don't care about Linux.

I realize there was a lot of bad management decisions involved, but look at what happened to the last company that tried to make a business out of porting titles to Linux (*cough* Loki *cough). I have just about every Loki title that was developed and I really wish they had stayed afloat. Maybe it was bad business decisions and maybe it was just that there was no profit in porting titles to Linux. The situation might be different today and I hope that someone has the desire, balls and money to step up and try what Loki tried 7 or 8 years ago. But Loki's fate did send a clear message. There's no profit in Linux games. John Carmack also said back then that releasing Q3A for Linux saw no profit.

Hopefully as more desktop companies, like Dell, jump on board and push Linux then maybe both the game developers and video card manufacturers will start to see the potential for profit and a result gamers will jump on board. But even Mac has suffered from the same problem for 20 years, and there's way more profit in developing for Mac than Linux. And it shows. There are more commercial Mac games than there are Linux. But both Linux and Mac have next to no games at all when you compare to the titles available for Windows.

You won't get good games until you get marketshare

[nweaver (113078)]

This is irrelevant to the gaming front.

The limit to games on Linux is market share. Its not (much) easier to develop a good 3D game for linux as it is Windows, so why code for 2% of the market when you can code for 92% of the market?

Thus you will only get games where the developer has gone out of their way to ensure complete portibility and provides a port mostly out of courtousy.

The scheduler details are irrelevant for this: what Linux Games need is 10%+ marketshare on the desktop.

Re:You won't get good games until you get marketsh

[Zironic (1112127)]

Sounds like an evil circle.

To get Market Share you need games.
To get Games you need Market Share.

Pluggable

[cerelib (903469)]

That is why Linus should have listened to Con Kolivas when he tried to introduce a pluggable scheduler system. With a modular system we could have CFS and the staircase scheduler and both problems solved.

Re:Pluggable

[andrewd18 (989408)]

Linus has already explained why he doesn't like the idea of a pluggable scheduler system. See http://kerneltrap.org/node/14019 [kerneltrap.org]

Setbacks?

[ichigo 2.0 (900288)]

One thing's for certain, gaming on Linux can't suffer any more setbacks or it may be many years before we see FOSS games rival the commercial world.

Linus mentioned somewhere that CFS and SD are both superior to the old scheduler, so in what way can choosing CFS be considered a setback?

Linux and Games

[CopaceticOpus (965603)]

I've figured out how to get games running with Linux!

1. Set up your Linux system. Use onboard video and don't overspend on your processor.
2. Buy a PS2, a Wii, or a 360.
3. Play games on your game console and do everything else on Linux.

Be glad they chose CFS over SD then.

[delire (809063)]

Sayeth Kenneth.

Alright, Just got done with some testing of UT2004 between 2.6.23-rc1 CFS and 2.6.22-ck1 SD. This series of tests was run by spawning in a map while not moving at all and always facing the same direction, while slowing increasing the number of loops.

CFS generally seemed a lot smoother as the load increased, while SD broke down to a highly unstable fps count that fluctuated massively around the third loop. Seems like I will stick to CFS for gaming now.

--
Kenneth Prugh

Sayeth Matthew

My experience was quite similar. I noticed after launching the second loop that the FPS stuck down to 15 for about 20 seconds, then climbed back up to 48. After that it went rapidly downhill. This is similar to other benchmarks I've done of SD versus CFS in the past. At a "normal" load they're fairly similar but SD breaks down under pressure.

The only other thing of interest is that the -ck kernel had the WM menus appear in about 3 seconds rather than 5-8 under the other two.

The only chap that says otherwise doesn't post his results.

From TFA [kerneltrap.org]

The X Factor

[mcelrath (8027)]

I keep wondering...X is a single threaded server, communicating with a (generally) single threaded game. Worse, wine inserts the wineserver process, so I have three single threaded things trying to synchronize to get interactivity. A low latency event like a keypress might require all three processes to be scheduled in succession, to get a response on the screen. A poor man's way to do this is with the kernel's scheduler, but a far superior way to do it is to have multiple threads in the X server. Scheduling an interactive event isn't hard. Getting crap on the screen in the same scheduling timeslice is hard (impossible?) since it requires a second scheduling point. As I understand, this is how BeOS achieved substantial interactivity in the presence of load -- my having a multi-threaded graphics server *and* kernel.

So, how much can be gained by rewriting X, or going to a different graphics server? Or do I completely misunderstand the effect of X?

-- Bob

Jitter is important

[dybdahl (80720)]

Usability research has shown, that variation in waiting time is actually a bigger irritation for users than waiting time itself.

I have seen several projects, where user interface response time problems have been "improved" by making adding a minimum response time. The average response time increases, but variation decreases, and the user often reports the program as having become faster... the logic to this seems to be, that the user wants the user interface to have a predictable response.

I think the reference for this is Søren Lauesens books about usability programming, but I cannot remember for sure right now.

Re:Hmm

[by Anonymous Coward]

The CFS (Completely Fair Scheduler) won't, but maybe the new FCS (Flux Capacitance Scheduler) might. Or already did.

Strange, I've been gaming in Linux for years.

[DaedalusHKX (660194)]

I have Windows XP and Gentoo Linux running side by side, and strangely, Gentoo scores 10 to 12 FPS faster in World of Warcraft, Warcraft III and even Doom 3. Granted they are commercial games, but if they can run in WINE that fast, I wonder what a direct Linux implementation would do. I just love seeing folks buying the headlines instead of blazing their own paths.

That's why the world is in the shape its in... the majority is always waiting for someone to save the day. You want desktop Linux? Then make it your desktop. Otherwise stop bitching and post some valid comments.

Re:Strange, I've been gaming in Linux for years.

[vtcodger (957785)]

*** I wonder what a direct Linux implementation would do***

A perfectly reasonable question, but the answer may well be "about the same". The NE in wiNE istands for "Not an Emulator". In a sense, WINE *IS* a native Linux graphics implementation albeit aided or hindered by using the Windows API interfaces. If I recall the WINE documentation correctly it says that WINE is sometimes faster than Windows on the same hardware and application and sometimes slower.

Here's a link http://linuxhelp.blogspot.com/2006/02/wine-vs-wind ows-xp-benchmarks.html [blogspot.com] that seems to say roughly the same thing.

Re:Strange, I've been gaming in Linux for years.

[Nasarius (593729)]

WOW DO have native linux ports

Uh...no. WoW does run nicely under Wine, but that's not a port.

Re:FOSS games

[OldeTimeGeek (725417)]

Aren't going to happen until artists in the medium, 'good' artists rather, decide to start working for free the same way coders do. Some artists will work for publicity alone, bu they seem to be by far in the minority.

Of course they're in the minority. For them, there's nothing to be gained in providing their services for free.

The publicity for working on games is almost nonexistent. For example, can you, name any artist that worked on any one of the most popular games? I can, but I know a bunch of artists that work in the games industry.

Besides, artwork doesn't work as FOSS. Unlike code, artwork for games isn't inherently "sharable" - it's designed for the purposes of that game and that game only. Game engines can be used for multiple different kinds of game. Artwork almost always can't. It may be used for sequels (but generally isn't as the requirements change from game to game) but it can't be used across different types of games.