Linux Breaks 100 Petabyte Ceiling

*no comment* writes: "Linux has broken the barrier with the 100 petabyte ceiling, and doing it at 144 petabytes." And this is even more impressive in pebibytes, too.

Re:Ok...

[Anonymous Coward]

How about the people who filled their 30mb drives in seconds a few years ago? The more things a computer can do, the bigger the files get. All I need now is a slightly bigger hard drive to take advantage...

watchit

[mr_exit]

remember when 640k was enough for everybody?
well i for one am scared by the fact that oneday soon 144pentabyte files will seem small

- Lord of the Rings is boring. There is a distinct lack of giant robots in it. Good movies have giant robots, bad movies don't. -james

Re:Ok...

[astrophysics]

There are about 10^10 solar masses of mass in a large galaxy like our own. At ~10^33 g/ solar maxx, and 10^23 atoms per gram, That's 10^66~2^219 particles in our galaxy. Beleive me, scientists will make use of as much computing power, RAM, and storage space as they can get their hands on. If only the limiting factor were operating system limitations rather than the more practicalities realities of funding and costs of hardware.

Re:Ok...

[PyroMosh]

No, not really. But you must remember, that if you have a need and you wait until you have that need to develop a solution for it, you have developed the solution too late. Remember "640K sould be enough for anyone?" That's an example of not planning for the future adequately. So, no it you're looking for a here and now reason for it, you're not going to find it. But remember, that that's not the point.

OK this is great...

[TheMMaster]

Now, I can really imagine someone that buys a 144Pb drive (array) and will use IDE?? I would personally go for SCSI there ;-)

What I am really wondering is: is there at the current moment ANY company/application/whatever that required this amount of storage? I thought that even a large bank could manage with a few TB's
Not intended as a flame, just interested

but still, this is a Good Thing (r)

144 PB, not really

[tap]

Sounds like all they are saying is that the new
IDE driver can support 48 bit addressing. With 2^48 seconds of 512 bytes, you get 144 PB. But there are a LOT of other barriers to huge filesystems or files.

For instance, the Linux SCSI driver has always support 32 bit addressing, good enough for 2 terabytes on a single drive. But until recently, you couldn't have a file larger than 2 gigabytes (1024x smaller) in Linux. I think that the ext2 filesystem still has a limit of 4 TB for a single partition.

So while the IDE driver may be able to deal with a hard drive 144 PB in size, you would still have to chop it into 4 TB partition.

Finally something is done right....

[Anonymous Coward]

The drive size limitations for IDE drives has been driving me nuts for years. First we had 0.5G, then 2G, then 8G, 32G and finally 128G (or 137G actually)... every time the barrier was moved forward to be 4 times larger than before, which meant we needed a knew kludge every 2 years. At least now it'll take a little bit longer, before we need yet another addressing scheme. By the way, this would be an excellent opportunity to nuke the old DOS partition table format (happily I know *BSD never needed it) once and for all, as well.

Btw, don't get messed up with two distinct things: 1) Being able to address 2^48 sectors on an IDE disc, and 2) having a filesystem that can handle files as large as 2^48 sectors.

Re:Ok...

[Anonymous Coward]

Heavy elements are pretty rare compared with hydrogen & helium, so an average atomic weight of 6 is feasible imho.

I would still like to see where the figures come from though.

Re:Somebody will probably correct me ...

[Anonymous Coward]

Of course, this was probably salescrap. Does anyone know the truth on this?

The BABAR experiment [stanford.edu] at SLAC [stanford.edu] is using Objectivity for data storage. Unfortunately, I cannot find a publicly available web page about computing at BABAR right now.

The amount of data BABAR produces is in the order of magnitude of 10's of terabytes per year (maybe a hundered), and even storing this amount in Objectivity is not without problems. The LHC [web.cern.ch], which is currently under construction, will generate much more data than BABAR, but even if they reach 10 petabytes per year one day, I very much doubt that they will be able to store this in Objectivity.

Re:Ok...

[digitalunity]

Think *Cinema*.
Current codecs already do a pretty decent job of compression of smaller(resolution) streams. However, what if I want my linux box feeding my HDTV projector at high resolution? This might be one more step in my vision of the ultimate entertainment center.

I wouldn't say they are the first/only to do this.

[Leimy]

http://www.freebsd.org/news/newsflash.html#2001Nov ember3:1

Hey ... look at that 48 bit addressing ATA drivers are now working? Wow... maybe FreeBSD people should run around making bogus claims too. FreeBSD invented the Question Mark! Wooo hoo.

I also think you can use Vinum to mount such a petabyte sized file system fairily easilly.

Really FreeBSD doesn't get enough credit for work that's been done. I know linux has a lot of good marketing for technical features but you also have to believe everything you read to fall for it.

1.44 petabytes is half a lifetime

[mrogers]

The size of information storage devices and the bandwidth of networks are approaching meaningful limits: the size and bandwidth of human experience. Tor Norretranders claims in his book The User Illusion [amazon.com] that the amount of information absorbed by the senses is around 11 Mbits per second. In other words, a totally immersive virtual experience with sight, sound, smell, taste, touch and motion could be transmitted over a standard Ethernet connection. An entire day of a human life could be recorded in perfect detail (with no compression) on a 120 GB disk. So there is a limit to how much information you could ever want to store. In your entire life you will experience less than 3.5 petabytes of information. 1.44 petabytes will never seem small to a human being.

However, there might one day be information processing systems to which 1.44 petabytes is a small amount of information. In a sense, these systems will have a richer experience of the world than human beings. I wonder if human consciousness would seem marvellous or valuable to such a machine.