SuSE and Siemens Release Linux Memory Extension 150
hussar noted that SuSE and Siemens have developed a memory extension that will allow Linux to use up to 4GB of memory. Linus has reportedly approved its inclusion in kernel 2.3.15. The strangest part is excite has taken to posting about Linux Kernel Patches. Pretty crazy stuff.
Re:This is great! (Score:1)
That kind of memory is pretty common place - and Linux needs to support it.
Has anyone played with SGI's journaled file system for Linux? I am glad to see this major shortcoming being looked at.
Re:Excite covering kernel patches (Score:2)
Yes, it was key to supporting IRIX.
What's important to note and understand is that everyone, and I mean EVERYONE, has a differnet idea of what is and isn't a good feature. And believe me, I get a fair amount of feedback on my articles and I rarely can please everyone. There's always the person that just isn't happy until the PC internal speaker is supported, or until the kernel can do ISA PnP (it can now), or until it can do some other completely bizarre task that seems trivial to us at first, but them you realize how critical this can be under
That's what makes Linux great: we have such a selection of great things that we have to please everyone a little; but we'll never please everyone entirely.
As for me, this 4G thing seems pretty silly.
Joe
...absolutely not! (Score:2)
Part of the FreeBSD kernel are actually under GNU...
Most certainly not!
Berlin-- http://www.berlin-consortium.org [berlin-consortium.org]
Re:What limit have other OS-es? (Score:1)
HP-UX 11, being a 64-bit OS, breaks the 4 Gig limit, but IIRC the actual limit varies a bit between "patch levels". They're moving it upwards all the time, I seem to recall.
--
Re:Excite covering kernel patches (Score:1)
Re:More info? (Score:1)
Re:Okay, So when will it be able to access all of (Score:1)
Re:NT has had 4GB addressable for a while. (Score:1)
Yes, NT (unless you turn on the right boot flag, which might work even in non-enterprise editions) has 2GB of the virtual address space of the machine available to user-mode code and 2GB for kernel-mode code, just as had been the case for Linux all along.
However, I'm not sure that limited you to 2GB of physical RAM on NT, as NT might not require that all of physical memory be mapped into the kernel's address space.
Linux has, for a long time, supported "unlimited user space" in the sense that user-mode code can map stuff into and out of its address space, so that the no-more-than-4GB window you get on a processor with 32-bit linear addresses (e.g., an x86 processor) can be moved around a more-than-4GB set of data. The patch doesn't change that.
However, that's true of most OSes that run on x86 processors, these days.
Re:Somewhat related question: (Score:1)
You don't need to use segmentation to get at >4GB of physical memory.
In fact, segmentation doesn't even help, given that the x86 MMU maps 48-bit segmented addresses to 32-bit linear addresses before running said addresses through the page table and translating them to 32-bit or 36-bit physical addresses.
However, for any single process to access more than 4GB, it does have to do something that amounts to bank switching, i.e. map stuff into and out of its no-more-than-4GB linear address space as necessary; the same applies to kernel-mode code and the kernel-mode portion of the address space.
Re:this is how companies work (Score:1)
NT 4.0, or NT 5.0^H^H^H^H^H^HW2K? I don't think NT 4.0 supports more than 4GB of physical memory, but W2K, at least in the DataCenter Server edition, will.
See, for example, this note on "Address Windows Extensions and Windows 2000 DataCenter Server" [microsoft.com], and this press-release-like document [microsoft.com] which indicates that this is new in W2K and not in NT 4.0, but also seems to imply that memory above 4GB will be available to the page pool (I'd seen stuff that gave me the impression that it would be wired-down memory that had to be specially mapped into a process's address space, but perhaps that's not the case).
Re:Somewhat related question: (Score:1)
No, it doesn't. It just has a 32-bit base linear address, a 20-bit length (which can either be in units of bytes or 4K pages), and a bunch of other flags. A 48-bit far address gets translated to a 32-bit linear address, and that is what gets translated to a 32-bit or 36-bit physical address via the page table.
See (this page that has a link to the PDF document) Intel Architecture Software Developers Manual Volume 3: System Programming [intel.com] for the full story.
Re:This is great! (Score:1)
Actually I do have a couple of servers with a small amount of disk on them (16GB each as I recall) and the main problem is how slow NFS is.
still kernal level appletalk support is nice. Is all swings and roundabouts.
Re:windows98 (Score:1)
Re:BSD (Score:1)
I really expected better from the user base than a bad case of Linux Envy...
Re:Excite covering kernel patches (Score:2)
Re:this is how companies work (Score:2)
Yes, P6's have 36-bit physical memory addressing; this does not mean that virtual (linear) addresses are 36 bits. (Some other 32-bit processors, e.g. 32-bit SPARC processors or processor modules with the SPARC Reference MMU, also supported more than 32 bits of physical address. Heck, PDP-11s with MMUs often supported either 18 or 22 bits of physical address, even though they only supported 16 - or 17, sort of, if you count split I&D space - bits of virtual address.)
Dynix, on Sequent's servers, might - they support much more than 4GB of physical memory, and if the NUMA part of "NUMA-Q" just means that accessing "other people's" memory is slower, not that you have to play bus-mapping games to get at it, they may use the 36-bit features to do that - although I thought I saw something indicating that some of their machines supported 128GB, which requires 37 bits.
Re:What limit have other OS-es? (Score:1)
Re:What limit have other OS-es? (Score:1)
on a completely unrelated point, I wonder if this new 4GB support is a compile-time option, or if it can be somehow enabled at runtime without a performance hit when it's disabled. that would be neat, but it does sound a bit improbable for a change like this.
Re:More info? (Score:1)
Re:Any details? (Score:1)
Re:What limit have other OS-es? (Score:1)
This is what linux needs to remian competitive (Score:1)
Re:Isn't this a bit late? (Score:1)
--
Michael Hasenstein
http://www.csn.tu-chemnitz.de/~mha/ [tu-chemnitz.de]
Excite covering kernel patches (Score:3)
My best example of this was when I saw a rehash of Pravenich's 2.4 kernel thing on a tech news site - they'd duplicated his mistakes, and were brainwashing the masses with the stuff! ;)
Re:Walnut Creek/FreeBSD (Score:1)
And *when did I ever say bsd has no drivers*.
Re:This is what linux needs to remian competitive (Score:1)
Re:Top 5 Reasons (Score:1)
Office 2000 can now run on Linux with 4GB.
W2K only supports 128M?
Re:This is what linux needs to remian competitive (Score:1)
The second and more important IMHO is app support. You need to be able to run things on this nifty OS (whatever OS that is). Now i favor a more specialized approach, using the right OS for the job. Realistically I could care less if I can use a word processor on a web server. And for a desktop only machine I can care less if I can run a web server.
The 4GB memory capability is something that has been needed. A webserver or any server that needs high performance really doesn't need to be bogged down by the I/O subsystem.
Re:Windows 2000 supports 64GB (Score:1)
Re:This is what linux needs to remian competitive (Score:1)
Things are on the up and up indeed.
What limit have other OS-es? (Score:1)
I'm interested if anybody knows what are the limits of Win 9x/NT, BEOS, *BSD, MacOS,...
Linux rox!!!
Re:What limit have other OS-es? (Score:2)
The current MacOS can access up to 1.5 gigs (even thought the new high end G4s will let you put 2 gigs in).
I thinkNT will let you access 4 gigs. At least that is what I understood from the whole Midcraft fiasco...
I'm not familiar enough with the others to answer...
this is how companies work (Score:3)
Now, that someone has come up with a bigmem patch that Linus will live with, THAT is news!
This is a big deal for some users. A real shocker will be if someone comes up with a patch to use the 36 bit addressing on the P6 cpu, for up to 64GB ram on Intel machines.
Re:IA64 (Score:1)
Re:What limit have other OS-es? (Score:3)
I'm not sure, given that the "enterprise" edition lets it use only 1GB, but note that this is virtual memory, not physical memory - this doesn't mean NT is consuming 2GB of physical memory.
I infer from the stuff I've seen in the "Inside Windows NT" books that it has a common page/buffer cache; the first edition of Inside Windows NT says
which is similar to what some UNIX systems (e.g., SunOS 4.x, SVR4-based systems, FreeBSD, etc.) do; I have the impression that if the Linux kernel doesn't yet have a unified page/buffer cache in a stable kernel, it's going into the 2.3[.x] line.
Most of memory is, in effect, a page cache on these systems, unless you count only pages not currently being used as cached, in which case most of memory not used by pages currently being used is a page cache. In addition, the buffer cache (in the sense of the cache of pages read in from files) is just part of the page cache.
Linux kernel story not unusual for excite (Score:2)
For the non-journalists/editors in the crowd, I'll point out that this story was A) from a newswire service (check the DATELINE--ITS IN ALL CAPS LIKE THIS) and B) the story was obviously a press release (witness the "About SuSE" section of the article -- an obvious shameless plug for the company putting out the press release).
Excite's news section consists almost entirely of newswires because its cheaper than writing your own stories and they generally don't require too much editing (because they are written by professionals who know what newspaper editors like to see in an article in terms of structure, content, syntax etc.)
FWIW, I used to design/edit/publish several newspapers for non-profit veterans groups like the AMVETS and PVA...at least until I got a "real" job in the IT field.
Kernel development driven by people (Score:3)
Re:Somewhat related question: (Score:1)
I vaguely remember reading that Solaris now supports it, although finding detailed technical information on Sun's shiny new Marketing-Driven(TM) Web site looks as if it'd demand more patience than I have.
It's probably a question of whether any "commodity" x86 machines support it; if not, then Solaris for Intel, and UnixWare, may not support it either.
Re:Somewhat related question: (Score:1)
Not in the case of P6-core machines; segmentation turns a 48-bit segmented address into a 32-bit linear address. They're using paging, instead, i.e. the page table entries, in one of the 36-bit-physical-address modes, generate more than 32 bits of physical address from 32 bits of (linear) virtual address.
Any one process would have to map stuff into and out of its address space to use more than 4GB (or 4GB minus what kernel-mode code takes) of physical memory, but
Kernel-mode code could also map stuff into and out of its part of the address space.
And then there's SGI. (Score:3)
Re:Kernel development driven by people (Score:1)
Contributions from hardware vendors (Score:3)
Re:Excite covering kernel patches (Score:1)
Ok, so I'm posting too much in this non-thread.
Re:This is great! (Score:1)
Well ... it depends what you do. Where I work (in the EDA industry), most of our boxes are 2Gb, and we have a few 4Gb boxes, and that's not enough... We still swap like crazy for some jobs.
I unfortunately can't find more than 4Gb on UltraSparcs (our main platform) without going to a large server. 8(
Re:2^64 - 2^43(?) (Score:1)
39-bit = 549755813888 = 512 gigabytes.
42-bit = 4398046511104 = 4 terabytes.
45-bit = 35184372088832 = 32 terabytes.
You only need 4 terabytes of memory if you're altavista or fedex and you want to keep your entire database in memory at once. Otherwise a couple-hundred gigabytes should do you fine.
(I wish I could remember the specific figure...)
36 bit and 32 bit (Score:1)
--
Re:36 bit and 32 bit (Score:3)
True, in the sense that at any given instant of time, the linear address space of a process can be no larger than 4GB (and, as segmented addresses get translated to 32-bit linear addresses before being run through the page table, segmentation doesn't help).
However, as I noted in another post, a process could, if the OS lets it (and most OSes you'd run on big machines do, these days), map stuff into and out of its address space, and more than one process could exist on the machine, so you can use it for userland code, userland data, kernel code, kernel data, mapped files, buffer cache, whatever, just as you can use the first 4GB - it's just a question of what the OS does or lets applications do.
Re:This is great! (Score:1)
Unfortunately, my boss did not like the idea of running MSSQL server that way, and the ServerRAID would not work (IBM has since release drivers, however.)
Re:Excite covering kernel patches (Score:1)
which it certainly couldn't this time last year.
Not true. BW is a commercial service that distributes unedited press releases. Excite runs all of the BW content. (It may be crap, or it may not
The avenue has always been open to Linux. All it takes is a public-relations department. This is not a case of Linux becoming important enough for the general media to cover; it is a case of a Linux company becoming sufficiently aware of How Things Work to get the word out through commonly available channels.
Re:What limit have other OS-es? (Score:1)
I had read that recent revisions of 7 would support 36 GB on P6 machines; however I guess that functionality was removed.
Clarification (Score:3)
test %eax,0x8000000
je user_mem_label
I think they have worked on 3GB prior to this. SOrry, been a while since I checked the kernel lists.
Anyhow, this is only of practical use to database developers. maybe some but not many. In any case, you might as well use a 64-bit architecture.
Hasdi
It's a company press release... (Score:1)
Re:This is great! (Score:1)
Re:Kernel development driven by people (Score:1)
Re:Kernel development driven by people (Score:2)
http://boudicca.tux.org/hypermail/linux-kernel/19
Rob make an "Update" and give credit to people behind those companies.
For Italian people: Andrea will talk in Padova check http://meeting.pluto.linux.it/ for more
Size of swap partitions/files (Score:1)
Has this annoying restriction of 128MB been removed, or can we use more swap spaces (lets see, 4 GB divided by 128 MB gives way too much swap spaces to be practical), or are we not supposed to use virtual memory any more ?
If the restriction of 128 MB per swap space still exists, is there anybody working on removing this so Linux can become practical for modern computers? If the restriction is removed, is it possible to create a single 8GB swap space ?
Re:windows98 (Score:2)
Re:Size of swap partitions/files (Score:1)
Since Kernel 2.2 the 128MB limitation is gone. You only need a recent mkswap/swapon to use the new big swap partitions. I dont know if there is another limit, i'm happily using 1GB swap partitions on computers with 1GB RAM.
-- Anderl
4Gb? (Score:1)
Why the difficulty (Score:1)
How is it that FreeBSD handles more RAM? Do they merely take a performance hit or do they have better kernel architecture?
Re:Okay, So when will it be able to access all of (Score:1)
Re:windows98 (Score:1)
What type of motherboard were you using?
Try the same eact setup with linux, you'll probably have problems there too.
Re:What limit have other OS-es? (Score:1)
though apps themselves can only use 999 megs
Re:Any details? (Score:1)
Re:this is how companies work (Score:1)
The enterprise edition of Windows NT Server does make use of this. They are doing the bankswitching thing, as with EMM.
I thought Linux *already* supported 4 gigs? (Score:1)
Re:What limit have other OS-es? (Score:1)
So I'll bet you'll see it SOONER then you think
Re:More info? (Score:1)
Re:What limit have other OS-es? (Score:1)
Re:What limit have other OS-es? (Score:1)
Re:What limit have other OS-es? (Score:1)
But this is going to change in MacOS X.
The maximum memory every G2 Mac (7500,7300,8500,7600,8600,9500,9600) and G3 Mac (G3, G3bw, iMac, G3series powerbook, G4) supports is 4 Gb.
This is going to rise to 1 Tb when Apple is going to switch to the G5 which is 64 bits.
Re:Somewhat related question: (Score:1)
I'm surprised to see this sort of thing again. I can remember the first time I saw this kind of stuff, on the old Atari 130xe: 128 Kbytes through 'bank switching'...
chris
Re:What limit have other OS-es? (Score:1)
Re: (Score:2)
Re:What limit have other OS-es? (Score:2)
Re:Excite covering kernel patches (Score:1)
I could give you a dozen more examples, I'm sure.
I have found that, in my experience, the media always seems to pick up on the "errors" and less frequently on the real features. Ah well. I say that it's a "draft" for a reason.
Joe
Top 5 Reasons (Score:5)
5) Office 2000 memory requirements now supported by Linux, making port much easier.
4) Enhances sales potential of Windows 2000 -- WINE now able to run W2K under Linux.
3) Yet another fun Linux feature to deny and obfuscate.
2) Can complain before tech-unclued journalists about Linux's memory requirements -- 4G compared to W2K's 128M.
1) Now that Linux supports 4G of RAM, it will be competition on the everyday Joe's desktop, thus making MS-DOJ trial irrelevant.
(Darn Excite, slashdotted again.)
Re:What limit have other OS-es? (Score:1)
Re:What limit have other OS-es? (Score:2)
Re:Somewhat related question: (Score:5)
The Intel P6 line has a 36bit physical address bus allowing the chip to address up to 64GB of physical memory. However, since it is a 32-bit processor, you can only see 4GB of this memory at any time. You can change what you can see by playing games with PTEs (page table entries), and/or segment registers.
To address >4GB of memory requires use of either wide (64-bit) PTEs, or the strange mode Intel added to the PII, that allows large pages with narrow PTEs.
The Sequent Dynix/ptx OS supports up to 64GB of physical memory. The hard part is conserving enough kernel virtual address space.
Re:this is how companies work (Score:1)
probably not.... (Score:1)
"Subtle mind control? Why do all these HTML buttons say 'Submit' ?"
Re:Somewhat related question: (Score:1)
They're still there, albiert in a different form.
In 32-bit code, near addresses are much more common than far addresses, but far addresses are possible. 32-bit far addresses consist of a 16-bit segment and a 32-bit offset. The segment refers to a page table, a begining, and an ending. The page tables then refer to physical ram, which has 36 address bits these days.
Environments that have all segments more or less equivalent are called 32-bit flat mode (as opposed to 32-bit segmented mode).
Re:Somewhat related question: (Score:1)
Gahhhh! Segment registers, yuck. You'd think we'd be done with them by now, but they're creeping back.
I did a little 8088 programming back in the Eighties, and all I can say is "Never Again". I don't mind at all on a machine with a flat memory model, but I don't want to touch segments ever again if I can help it. It didn't help that those segments on the 8088 were only 64K in size.
Let's just move to 64-bit processors, shall we?
Re:Clarification (Score:2)
The issue is not to enlarge the virtual address space of a process, but to enlarge the total probable physical memory to the kernel. The processes still do see only 3GB of virtual memory.
The difference is, that previously all physical memory had to be mapped into the kernel virtual address space (hence max. 3GB and then 1GB virtual memory to processes), now this is no longer true. Now we have two sorts of memory, normal memory that is always mapped into the kernel virtual address space and the high memory (all above 3GB but to have 3GB virtual address space to processes its usually all above 1GB) which is mapped on-demand to a special virtual address range in the kernel (which is also the overhead of this model).
Richard.
what does this have to do with anything? (Score:1)
"Subtle mind control? Why do all these HTML buttons say 'Submit' ?"
Re:Excite covering kernel patches (Score:2)
For probably 99.9% of people, it is pretty silly. Most people/companies can't afford that much memory even if they had a motherboard that could handle it (most motherboards are limited to somewhere between 512M and 2G). This limit matters strictly for the very highest end 32 bit systems.
I think it is mainly being done to eliminate one of Microsoft/Mindcraft's 'checklist' items, and there is certainly a level of justification for doing that sort of thing.
However, there is also always that 0.1% that it really does matter to, I guess. In general, I would imagine that this probably just won't matter to most of us, but it will remove one of the percieved barriers against high end Linux deployment.
The difficult thing may be getting people/press to realize that things have changed. Witness the ongoing examples of people who still claim that Linux can't support multiple CPUs at all, when it has had at least rudimentary SMP support for quite some time. And of course they would never mention that Linux SMP support has improved some recently, and is going to be improving further in the next couple of major kernel versions.
Re:Windows 2000 supports 64GB (Score:1)
Re:Clarification (Score:1)
In any case, the limiting factor is the kernel/user memory model rather than the kernel per se. I still think we be better off with a 64-bit architecture.
Hasdi
[Some Versions of] Windows 2000 support 64GB (Score:1)
That means your heavy-duty stand-alone applications require you to purchase a very expensive product, which is tuned exclusively for network server and database work. You'll get an expensive set of server apps and licenses "for free" which you'll never use, and wait several weeks after the initial release for each and every service pack (they're always delayed for the high-end NT versions).
If you're interested in spending this kind of money, run Linux or *BSD on hardware that's designed to have 64Gb of memory and 12 CPUs (eg UltraSparc or Alpha) and don't worry about stupid kludges from Intel or any other 32-bit vendor. When even Intel are telling you that IA32 is a dead-end, it's time to get off.
In fact, if you've really got this type of job to do, your priority is probably scalability and performance, in which case a Proprietary Unix on it's own native hardware is going to look much more attractive. Want to buy some E450s?
Nick.
Re:[Some Versions of] Windows 2000 support 64GB (Score:1)
The fact that it's in a different product is because of marketing.
Hey, why not charge more for that sortta stuff....Sun etc do already.
Re:This is great! (Score:1)
2GB system, 2GB user...same with 98.
I've run 95 on 256 - so it definitely isn't anywhere near as low as you think.
Re:And then there's SGI. (Score:1)
We have worked with an engineer at SGI on implementing the SGI 4GB patch which has been out for some time.
There were some other issues with Oracle shared memory, but we finally resolved those as well. In other words, 4GB is alive and well and in production on our web site right now. Id expect a writeup in the linux journal in a couple of months on exactly how we accomplished this and implications.
Re:What limit have other OS-es? (Score:1)
-sporty
I like to moo it moo it.
Re:What limit have other OS-es? (Score:2)
Re:Somewhat related question: (Score:1)
(1) PPro and Xenon for sure. Dunno about PII and PIII. Certain PowerPCs also have this.
Re:Excite covering kernel patches (Score:2)
Maybe it's interesting that Siemens would consider a kernel patch significant enough to warrant a press release, although this doesn't seem surprising to me. It's a pretty significant new feature.
Re:This is great! (Score:1)
http://www.bombcar.com It's where it is at.
Re:Somewhat related question: (Score:1)
If you remember back in the old days of the 8086 and its friends, you had two values to address everything. Segment register, and index. Now some braindead monkey decided that a segment would be equal to 16 bytes (4 bits!) effectively cutting 12 bits off of what could have been 32-bit addressing on those old 16-bit machines. (16-bit segment, 16-bit offset.) Of course the reason was it was much easier to write 16-bit code if you didn't have to worry about wrapping your segment register in the middle of arrays and stuff, so as long as we used under 1 meg of ram, it seemed like a good thing.
When they made the 80386, the mistake wasn't made twice. Now the 16-bit segment register points to a virtual page table, which means each 16-bit segment register 32-bit index pair is theoretically capable of addressing 4 gigs. It would be a bitch and a half I think, to use more than 4 gigs in a single process, but that doesn't rule out different processes using drastically different address spaces. So think about having 4 gigs per process and see how well you could use that.
Frankly, I have no freaking clue what you'd do with it. Maybe I'd increase the size of my memory cache in Netscape...
Also, I don't know what the implications would be for writing an operating system, as I have never done so. But I assume it would make things tricky. I'll just wait for the 64-bit machines, and enjoy the [insert huge number I don't know here] bytes of directly accessible RAM.
Re:Somewhat related question: (Score:2)
The (impossible?) trick is to make segmented memory access "clean". Neither Linux nor *BSD has chosen to embrace the hoops needed to use large memory P6 machines. NT doesn't get full advantage of >4G either, you know: that RAM does have a 32-bit string attached. Offhand I don't know about Solaris, SCO, or anyone else. I'd be a little surprised if Sun didn't do it, although I'd also understand if they left it out to encourage UltraSparc sales.
The K5, K6, K6-2, K6-3, M2, C6, et al have no such extensions, of course -- just Intel's patented P6 processors and buses.