High Performance Linux Kernel Project — LinuxDNA 173
Thaidog submits word of a high-performance Linux kernel project called "LinuxDNA," writing "I am heading up a project to get a current kernel version to compile with the Intel ICC compiler and we have finally had success in creating a kernel! All the instructions to compile the kernel are there (geared towards Gentoo, but obviously it can work on any Linux) and it is relatively easy for anyone with the skills to compile a kernel to get it working. We see this as a great project for high performance clusters, gaming and scientific computing. The hopes are to maintain a kernel source along side the current kernel ... the mirror has 2.6.22 on it currently, because there are a few changes after .22 that make compiling a little harder for the average Joe (but not impossible). Here is our first story in Linux Journal."
Re:GCC compatibility (Score:2, Informative)
Amazing. You have no idea what you're talking about :D
C99 doesn't stop you writing interrupt code OR threaded code.
Re:Will this kernel run fast on AMD processors? (Score:2, Informative)
Nope, they have not changed that, and I think it is quite bad behavior for Intel.
However the do _not_ insert _bad_ code. What they do is that they prevent code optimized for the newest Intel CPUs to run on non-Intel CPUs, even if all the used instructions is present. I think -xW (use SSE, SSE2, optimize for Pentium4) is the highest that will run on AMD.
However in almost all cases the Intel compilers will still produce the fastest binaries on AMD. Not only compared to GCC, but also compared to other commercial compilers like PGI (with have specific optimization flags for the latest AMD CPUs.)
Re:letme google that for ya. (Score:3, Informative)
It severely cripples maintenance. Any optimisation, especially one that forks you into multiple parallel implementations (raw C, x86 asm, amd64 asm, amd64 ASM with SSE4, PPC, ....), has to be carefully weighed against its extra maintenance cost.
The parts that do benefit from optimisation, such as RAID parity calculation, symmetric encryption, etc. are already optimised. At any rate I think the kernel developers know a lot more about this than you or I do.
Re:We're going about the problem the wrong way (Score:3, Informative)
That's being done too. GCC 4.3 with Profile Guided Optimisation is SWEET. I don't think plain PGO can be run on a kernel (but that would be an awesome project), but it would definitely close the gap between ICC and GCC. ICC's PGO is not as good, or rather, ICC itself is better at making the kind of fuzzy predictions that PGO makes definite.
Re:GCC compatibility (Score:4, Informative)
(Hint: there is no standard way)
Re:GCC compatibility (Score:4, Informative)
To a large extent, they have. ICC really no longer has the performance lead that it once did over gcc. There was absolutely a time when the difference was consistent, and significant. But, a lot has changed since gcc 2.95, when egcs existed. The 4.x branch in particular has been about improving the optimisation capabilities of the compiler. These days, I generally reccomend just going with gcc to anybody who asks me.
Re:compilers? (Score:3, Informative)
The GCC team has to work on ARM/MIPS/SPARC/whatever while ICC only need to work on x86.
ICC supports IA-32, Itanium 1 & 2, x86-64, and xscale. Not that it kicks too much of a leg from your argument, but if you are going to argue the point you should at least make it accurate. Ah yeah almost forgot to mention all the extended instruction sets too... SSE, SSE2, SSE3, MMX, MMX2, etc...
Re:GCC compatibility (Score:5, Informative)
There isn't one, so what you do is use pragmas (I remember #pragma pack(1)) or attributes (__attribute__((packed)) or something similar.
Of course they're compiler-specific but there's no reason that code can't be written wrapped in defines or typedefs to stop compiler-specific stuff getting into real production code nested 10 directories down in a codebase with 40,000,000 lines.
Linux does an okay job of this - but since coders usually reference the compiler manual to use these esoteric pragmas and types, they are usually told "this is specific to GCC" (GCC does a good job of this anyway) so they should be wrapping them by default to help their application be portable and maintainable to future compilers (especially if they change the attribute name or the way it works - as has been done on many a GCC, let alone other compilers).
What usually nukes it (and why linux-dna has a compiler wrapper) is because they're hardcoding options and doing other weird GCC-specific crap. This is not because they are lazy but because the Linux kernel has a "we use GCC so support that, who gives a crap about other compilers?" development policy and it usually takes some convincing - or a fork, as linux-dna is - to get these patches into mainline.
Re:GCC compatibility - Time to move to Java? (Score:3, Informative)
Re:Thank you, I look forward to trying this. (Score:2, Informative)
I'm afraid the boost of kernel code won't help you much. Since you're doing fluid physics, I guess the hotspots are in the floating point math computation, and your code doesn't do context switching often. In that case, kernel speed isn't that important.
Well, I'm just saying it. I hope I'm wrong :)
Re:GCC compatibility - Time to move to Java? (Score:4, Informative)
Java is not a "systems language", meaning you don't write operating systems and systems level code in it for very good reasons.
Funny cause Sun already did that like 13 years ago.
One of them being, name me a processor that can run Java bytecode nativly.
The ARM9E.
Re:Will this kernel run fast on AMD processors? (Score:4, Informative)
> Its their compiler, they are damn well allowed to do what they want - call me when AMD pour that kind of resource into having their own compiler.
ARM put money into GCC. That's far better than them trying to make their own compiler.
Re:GCC compatibility (Score:5, Informative)
Depends on the CPU... gcc has reasonable performance on x86, but on ia64 or ppc the vendor supplied compilers have a big advantage. even on x86 icc leads by a considerable margin in some areas, especially on very new processors.
Re:GCC compatibility (Score:5, Informative)
On the contrary - there is no support for threads or interrupts whatsoever in C99. Sure, there's pthreads and the like - but those are not part of C99, nor can you implement them in pure C99.
C itself (all versions) tries very hard to avoid tying itself to any specific hardware or OS. It even supports weird things like platforms with more than 8 bits in a char, or with reserved bits in their integers. But as a result, it has only the bare minimum featureset common across all platforms imaginable, and this is why it's very hard to write anything useful with only pure C. (No networking, no listing the contents of a directory, no executing any other programs except via system()...)
For most userland applications, C plus some OS-dependent libraries are good enough, of course. Things like the POSIX API can't be implemented in regular C (at some level you have an assembly call to the OS's syscall interface), but if you treat it as opaque, no problem.
But for an OS kernel, things aren't that easy. In the quest for high performance, Linux does all kinds of neat hacks, including things like inlining assembly code into C functions - and later rewriting that code on the fly (google for 'smp alternatives' for more information). It also makes use of CPU-level atomic operations - and exactly which ones are available depend on the architecture. Because of these kinds of hacks, which produce noticeable speed improvements, it is utterly impossible to stick purely to standards like C99.
Re:GCC compatibility - Time to move to Java? (Score:1, Informative)
What a complete load of garbage:
Don't get me wrong, I love Java and I'm a full time Java developer but here's why what you just said is a complete load of Tosh:
1. JIT compilation is just doing a similar job to a C/C++ compiler only at run time. It's certainly not any better than the excellent ICC compiler and for an operating system which you would want to boot quickly would you really want it compiling half the OS at boot up?
2. Most of Java's IO operations are just JNI wrappers for native C system calls anyway. Because Java has to be compatible with the APIs for all the different operating systems they make some nasty compromises in order to present a common interface. That means that Java IO is almost always alot slower than native IO.
3. It's just a terrible idea! Java is a wonderful language but not for operating systems.
Re:Portability.. (Score:3, Informative)
Oh, wait a second, I see the problem here.
You are a moron.
First up, personal attacks to the parent does not an argument prove, all it does is lessen your credibility.
By supporting a range of compilers we help make the kernel MORE robust to such changes, and these are both highly competent compilers, so the 'intersection' of features is actually most of the C/C++ specs..
Of course the intersection of features are the specs.. because they are the only standardized thing that makes it c, but as has been said, C leaves a LOT to the implementer in order to be flexible, the standard does not specify everything, and operating systems need to run at such a low level that what they deal with is NOT covered in said specs,
Furthermore, as for being 'more robust' to breakage when compiler changes occur by supporting more changes, bollocks, now instead of supporting one standard compiler that worked for platforms x,y and z with portable coding practices, you now have to monitor a second compilers changes just so it doesn't break on platform x aswell
Also you have to monitor that any changes you do to support the second compiler, don't break the building of the three platforms on the first.
Sounds like a lot of work for essentially no gains, and a lot of added complexity and room for errors that simply doesn't need to be there.
Of course you obviously have zero experience of such things. You seem to think 'better' means more highly tuned code - try maintaining a major project for more than 6 months, and you may well learn a thing or two.
By trying to support multiple compilers by changing source code to suit, you really are killing maintainability, with normal programs it might not be so bad, but as said the kernel relies on a LOT of things not standardized between compilers(inline assembly, etc etc), writing compiler specific portions for every aspect of it (and maintaining it) would be hell.
Again, you dont seem to know what you are talking about, do you perhaps measure compiler 'goodness' by Dhrystone mips?
I agree performance is never the be all and end all, however going the other route and saying performance is nothing is just as absurd.