Replacing Atime With Relatime in the Kernel 416
eldavojohn writes "Our friend Jeremy at the Kernal Trap has dug up some interesting criticism of atime from Linus Torvalds. As Linus submitted patches to improve relatime he noted: 'I cannot over-emphasize how much of a deal it is in practice. Atime updates are by far the biggest IO performance deficiency that Linux has today. Getting rid of atime updates would give us more everyday Linux performance than all the pagecache speedups of the past 10 years, _combined_.' And later severely beat atime about the head with a pointed stick: 'It's also perhaps the most stupid Unix design idea of all times. Unix is really nice and well done, but think about this a bit: 'For every file that is read from the disk, lets do a ... write to the disk! And, for every file that is already cached and which we read from the cache ... do a write to the disk!'" Well, I guess I can expect my Linux machine to become a little bit faster!"
Personally (Score:5, Interesting)
Re:Personally (Score:5, Informative)
It's funny the kernel devs are just now talking about this, I discovered this almost 6 months ago on the Ubuntu forums while glancing around for a fix for my laptop's ridiculous sound issues.
Re:Personally (Score:5, Informative)
I'm surprised Linus is just cluing in now about atime updates: The noatime workaround is listed in all kinds of performance-tuning guides.
Re:Personally (Score:5, Informative)
Is this just a Linux issue? (Score:4, Interesting)
I'm particularly curious as to whether it's an issue on Mac OS X with the HFS filesystem also, and whether it would be possible / advisable to force Mac OS X to mount the root HFS partition as noatime/nodiratime.
OS X doesn't use a traditional UNIX-style fstab, so it's not immediately clear to me how you'd change the mount options (last time I checked disk mounting was all just in
If this doesn't occur in other OSes (I picked OS X because it's the other OS that I use frequently, and it uses a default fileystem that's pretty different in design from ext2/3), it seems like it might be worthwhile to look at why that is, and what tradeoffs other OSes have made to avoid the same issue.
Re: (Score:3, Interesting)
atime is useful for deleting files from working directories, I use it on my laptop graphic design partition. Anything untouched for 2 months is deleted by a shell script to free up space. The backups and archives are also mounted with atime, its usefulness to me in this role far outweighs the performance penalty.
As a flash fs writer... (Score:5, Informative)
If you're using a desktop system with a hard disk you'll hardly notice any difference unless you hammer the system really hard.
Remember though that most Linux systems are either embedded (using mainly flash) or servers. In both these cases atime updates can be very damaging to performance and should be avoided unless there's a very good reason to turn it on.
Re: (Score:3, Informative)
Specs:
AMD 3000+, 512MB ram, 1x 80GB IDE, 2x 500GB SATA (Raid)
MASSIVE difference in performance, even just doing an ls on a directory with a lot of files/directories in it.
Re:Personally (Score:4, Informative)
Re: (Score:3, Interesting)
I ran a 'find . >
The first run took 50s, then it quickly stabilized between 1.5s and 6.5s, mostly around 4s. The cache obviously made a huge difference.
Then I remounted with noatime and reran the test. It was very consistently at just under 0.7s.
So, between 1.08 times faster and 6.4 times depending on if th
Ummm.. (Score:2, Funny)
Thanks.
Re: (Score:3, Informative)
Re:Ummm.. (Score:5, Informative)
becomes
Re:Ummm.. (Score:4, Interesting)
FWIW, the Relative Access Time (relatime) patch simply doesn't update the access time unless the file has been modified since the last atime write. That allows ancient applications like MUTT to still synchronize on various files. Synchronization that does not work with noatime set.
Of course, I have to question why they're still using something as ancient as MUTT. A nice event system would be 1000x more efficient than trying to synchronize on flat files stored in your home directory on the file system. Of course, that would require designing OSes beyond the standard UNIX/POSIX philosophy and design. So I doubt we'll see that in Linux any time soon.
Re:Ummm.. (Score:5, Insightful)
However Mutt's use of atime simply is cheap enough that there's simply never been a reason to change it all the time most people have had atime updates on anyway. If it ain't broke, don't fix it.
Re:Ummm.. (Score:5, Interesting)
The thing I really *really* like about Mutt is that it uses Unix mailbox files. These are not just human-readable, they can also be manipulated using tools like 'cat'. I periodically archive my working mail files into a backup directory by just concatenating the working files onto the archive files of the same name. The resulting archive mail files are still fully usable with Mutt, even though some are 100Mb+ in size with mail going back to 1997. I could also use them with even older mail clients such as Pine, but that would be like using vi when you've got vim.
When I initially began using Linux, I used the Netscape email software, got fed up with it, and tried a few other mail clients. But none of them came close to the flexibility of Mutt. They all used their own mailbox formats, which could not be archived in the way I just described. I suspect that this is still true. I'm not going to trust Opera, Sylpheed, Thunderbird or Evolution with my mail, because (a) I doubt present-day mail files will still work reliably in 10 years time, (b) I can't easily migrate to another mail reader, and (c) I can't efficiently archive my email because the database files are not plain text.
That's why I like Mutt, anyway.
Re:Ummm.. (Score:4, Informative)
Maildir [wikipedia.org] is my preferred format - each message is a separate plain text file, which means no locking issues, and simultaneous access from multiple machines. You can just tar the directory up if you want to move it, or cat all the files together if you want mbox back. You can use maildir with mutt [elho.net] (and lots of other software).
Re:Ummm.. (Score:4, Interesting)
It's also commonly used by cleanup scripts to delete rarely accessed temp files and such. I'm sure there are plenty of scripts in production dotted around doing things like find -atime +7 -delete, some probably quite recent.
If you read the article further... (Score:3, Informative)
They were also concerned about breakage of other systems that are not open source (HSMs) which use atime for usage tracking. Noting that HSMs and atime-sensitive maint
Re: (Score:2)
Not much of a commitment, there. The summary implied that notime was not supported. Do you disagree that it doesn't?
Go find bigger fish to fry.
Re: (Score:2)
Re: (Score:3, Informative)
Actually, noatime is a superset of nodiratime, so there is no need to specify both.
Re: (Score:3, Insightful)
I did know, but not off the top of my head. I haven't mucked around at that level for quite a while now. Given that the summary seemed to imply that Linux did *not* have noatime support (which would be quite an odd thing indeed) I was not ready to challenge it until I actually read the article. Of course, it was a case of bad-summary-itis. (Again.)
As it so happens, TFA is an exercise in the danger of pulling from a conversation with lit
Re: (Score:3, Informative)
Re:Ummm.. (Score:5, Informative)
Laptop power issue. (Score:3, Informative)
I have noatime set on my laptop.
Writing atime to the disk every time data is read from the cache keeps the disk spinning, burning battery power. "noatime" means the disk can shut down much of the time, spinning up only when writes must be synced or data not yet in cache must be read.
Re:Laptop power issue. (Score:5, Informative)
It's also worth mentioning that you *can* have atime enabled with properly configured laptop-mode and laptop-mode-tools and still see almost as much power savings -- The atime writes will still happen, but at least they will be buffered for when the HDD actually needs to spin up and do a lot of other more pressing IO.
Re:Laptop power issue. (Score:5, Informative)
Re:Ummm.. (Score:4, Interesting)
Ummmm... if you are doing regular backups then the atime will be changing every time so you can't tell if the file has not *really* been read in three years.
So atime is only useful in the case that you havn't backed that file up in three years and it has never even been accessed in that time... at which point I have to ask why you are only now deciding to archive it to offline media, if its relevence is really that low?
Re:Ummm.. (Score:5, Funny)
I don't think you can fully realize how poetic a phrase such as the one above can be until you read it with absolutely no understanding, as I do.
Thank you, ArcherB and thank you, Slashdot.
Re: (Score:2)
You don't really need to change any kernel settings, just set the noatime flag for your disk if you don't need the functionality.
Re:Ummm.. (Score:5, Insightful)
It's long been standard practice to disable this on, eg, news/mail spools (anything with large numbers of read-mostly small files). The new plan of, if I understand correctly, only updating atime when the file is modified or if the new atime is more than 24 hours after the current atime should provide nearly the same functionality with practically none of the performance hit.
Re: (Score:2)
Re: (Score:3, Interesting)
It's good for finding files that are not being accessed anymore (doh). This may not be a big issue on a personal desktop, but in a corporate network with tons of centrally installed tools - and even more versions thereof - it very much is. I've made in the past extensive use of it for exactly that purpose.
It's also good for automatically getting rid of the many defunct temporary files people tend to leave behind in shared directories such as /tmp. Again think multi-user machines.
I've also used it as
Re:Ummm.. (Score:5, Funny)
How can anyone write that with a straight face?
Re:Ummm.. (Score:5, Funny)
I'm sure it was written with a gay face.
Probably overblown (Score:5, Insightful)
Re: (Score:3, Insightful)
Seriously. Many have recommended mounting filesystems with the "noatime" parameter if you don't need to know atime for many years now,
Except for the fact that there are programs that rely on atime, so distributions don't want to set the default mount option on a filesystems to noatime.
For the vast majority of users who aren't interested in screwing around with their filesystem (and don't even know what a filesystem IS), that means that they'll suffer this performance penalty. That's why this middle ground
Re: (Score:3, Insightful)
"Do you wish to log whenever a file is written or read? So every time a file is used, it has to write an entry on the HDD, slowing performance, but it can have uses, like in forensics, security or backups (if a file has not been read in three years, it's probably safe to archive and move off the drive)."
With the default answer being "no" and only a single click required to make it "yes".
Re:Probably overblown (Score:5, Insightful)
The only real question is whether noatime or relatime make for a more sensible default.
Re:Probably overblown (Score:5, Interesting)
It is no option for the kernel to make noatime standard, as it would brake POSIX compliance. But without noatime, the OS suffers a large penalty compared to some other OSs. The magnitude of the penalty has been made clear in the quote from the article.
So a solution, which is POSIX compliant, but doesn't suffer as much a penalty as the current solution is sought for.
Re: (Score:3, Interesting)
Nothing about POSIX requires that it be impossible to get higher-resolution time stamps. You just can't get them with POSIX-only code.
For instance, OS X (and possibly other BSD-flavored UN*Xes) either defines the time stamps, in "struct stat", as struct timespec st_[amc]timespec; and #defines st_[amc]time as st_[amc]timespec.tv_sec, or puts a long st_[amc]timensec; field after each time_t st_[amc]time; field, depending on whether _POSIX_C_SOURCE is defined - POSIX-only code will work, and non-POSIX-only
Re: (Score:2)
Re:Probably overblown (Score:4, Interesting)
Really? I can't see why. In the past, I didn't really see the point of multiple partitions but with the choice of filesystems now, they make a huge difference. Putting my
Re: (Score:2)
Re: (Score:3, Informative)
Re: (Score:2)
there was no "noatime" option before? (Score:2, Insightful)
Re: (Score:3, Informative)
For instance on the mail server do this in the fstab (not necessary on root and the like...just use it on the heavy traffic mounts):
Re: (Score:2, Informative)
Re: (Score:3, Interesting)
I wish Unix (including Linux) would be more proactive about culling options which are not used by the majority and have such a detrimental effect. If the disk performance leaps by 10% by not using atime, it seems a no-brainer that no
Atime is used extensively (Score:3, Insightful)
Re: (Score:3, Interesting)
I see no reason why atime updates can't be postponed until some moment other metadata has to be flushed, or once a minute, whatever comes first.
The exactness of atime might suffer, but nobody will notice.
That said, I agree the noatime mount option covers most needs.
Almost nothing actually uses atime. The Relatime patch therefore only updates atime if mtime or ctime is newer (to show that the file has been accessed since it has been updated or created) or the current atime is over 1 day old. This increases IO usage only slightly compared to noatime, but is less likely to break those few applications that do use atime.
Re: (Score:3, Funny)
mount noatime? (Score:2, Redundant)
man mount says:
Is there something I'm missing?
doh. (Score:2)
And then I read the article. Very intersting, but the noatime trick will work till they get realtime sorted out. (unless you use mutt)
Re:mount noatime? (Score:4, Informative)
atime updates actually help a LOT in some circumstances. I use it to help with security checks (even though you can hack the atime record), to help debug (what input file caused it to hang?), and all sorts of things where it's useful to know when a file was read.
I could cope with not having them, but in many cases I love the extra information atime gives.
Ok, I'm missing something here (Score:3, Insightful)
Uh, why? Or rather, why not cache the atime update until disk usage is clear, THEN do the write, at least for already cached files?
Re: (Score:2)
It just seems like common sense to me. Yes, access times can be useful data but I think they are minor enough not to be done immediately. I'd default to having them cached and have an option for them to be applied immediately for t
Re: (Score:2)
I don't think trusting the access times is too critical in most cases, the cases where it really seems to matter are in software that checks for the files that have the most activity or programs that monitor f
weaker atime might be adequate (Score:2)
Another might be a "weakatime" option, to update the atimes in the inode cache but mark them as something weaker than "dirty" ("sumdged"?) when the data is re-read from cache. Opening, last closing, or
Re: (Score:2)
if you lose power you're in trouble because now you have all those pending (how many?) atimes that you just lost. and you don't know how many and for what files so if you rely on atime for something then you suddenly no longer can for any of those files (but which ones? - you have no way of knowing)..
Which is why there would be an option for immediately applying the access times. For many people they really don't need to have the access times and having it applied immediately is a huge performance hit. Those people would have the option set to delay the writing of access times, if a crash happens and they lose the access times then it's not a huge deal for them, it's not critical data.
For the people who rely on access times they could set the option so that access times are written out immediately.
Linus did not say that! (Score:5, Informative)
Re:Linus did not say that! (Score:5, Funny)
You fell victim to one of the classic blunders! (Score:5, Funny)
Re:You fell victim to one of the classic blunders! (Score:5, Funny)
atime vs ctime (Score:5, Insightful)
Re: (Score:3, Interesting)
Re:atime vs ctime (Score:5, Interesting)
A lot of the time, modification of a file... isn't a modification of a file. Instead, the program will delete the existing file and create a new one in its place. (There is sometimes other operations in there, like saving to a temp file, deleting the original, then renaming the temp file to the original file name.)
This means that storing the real creation time of a file means that it won't be what you expect, because the file that you think is the same file actually isn't.
(MS-DOS/Windows have something called filesystem tunneling [msdn.com] to attempt to get around this problem. If a file is deleted and a new one created in its place (see the MSDN article linked to from there for details) within a default 15 seconds, the creation time of the old file is carried over. This technique exchanges purity and absolute correctness (not that metadata times are reliable against tamper anyway) for utility.)
Re:atime vs ctime (Score:5, Interesting)
The linked site explains all this, but I know the propensity of
NTFS (Score:4, Insightful)
You can disable it in a few different ways Once this is done, the Last Access Time attribute for newly created files will simply be their File Creation Time. Disabling Last Access Time may affect the operation of backup programs that use the Remote Storage service. YMMV
Or there's a registry key, which requires a reboot:
Re: (Score:3, Informative)
Re: (Score:2)
But really, Create Time would be nice... even if they call it Summoning Time or Ethereal Conjuring Time.
Re: (Score:3, Insightful)
Name one single use.
There is a file on my drive, and I want to find out when it was created, just out of curiosity?
Summary please? (Score:2)
Re:Summary please? (Score:5, Informative)
Why not just fix the filesystem instead? (Score:2, Interesting)
Re: (Score:3, Informative)
I think relative-lazy-atime optimizes this further by only scheduling updates for atime if the old atime is older than ctime or mtime. If the old atime is newer than ctime or mtime then it silently forgets if you've accessed the file. If you are updating ctime or mtime, then I think atime is updated anyhow.
This optimization still allows certain programs to tell if files have been read once since they were last modified (probably the only non-securit
Not Realtime As In Real-Time System (Score:2)
If you read carefully, you will see it says rELAtime, not rEALtime.
noatime and rsync (Score:2)
C'mon guys -- no "itsatrap" tag? (Score:2)
would this help the "locate" utility? (Score:2)
Re: (Score:2, Informative)
You could schedule the update (updatedb) to run at a more convenient time.
Locate is a pretty useful command - locate file, and it shows you instantly where it is. I don't use it a lot but when I do it's VERY handy to have.
Wow (Score:3, Insightful)
Bit of trivia -- NTFS does the exact same thing with regards to access times. There's a registry entry to disable it buried somewhere deep that I don't remember at the moment, but if you're stuck on Windows it might be worth looking up to improve I/O performance.
Re: (Score:3, Informative)
Re: (Score:3, Informative)
Bit of additional trivia, Vista already disables it by default, and the whole access timestamp mechanism has changed, removing this performance penalty all together.
latest relatime patch (Score:5, Informative)
Hey, Slashdot posted an article about me! [ They also renamed me to Linus - what more can a geek ask for? ;-) ]
In any case, the latest version of the better-relatime patch can be picked up from:
http://redhat.com/~mingo/relatime-patches/
Apply it, build it, reboot into the new kernel and enjoy a faster (and lower latency) desktop. (no fstab twiddling needed)
Re:latest relatime patch (Score:5, Funny)
I kid, I kid.
Re:latest relatime patch (Score:5, Informative)
Firstly, the patch is mainly about modifying relatime behavior to make it more compatible and more usable.
The fact that you dont have to change fstab is no big deal, provided you have the right util-linux package installed, with the relatime user-space patch applied which not even the latest distro devel repositories have included.
If you dont have that then adding "relatime" to your fstab might leave you with a read-only mounted root filesystem and some commandline (or rescue-image) tinkering to do.
People prefer all-in-one kernel patches that just turns on the feature they are interested in. You'd be surprised how many people are willing to try almost arbitrary kernel patches but loathe to touch their user-space environment in any way.
And ... it's also kind of ironic that this relatively small patch often brings more practical benefits to the desktop than all the "big" desktop interactivity/latency features (cfs, swap-prefetch, -rt kernel) combined.
Re: (Score:3, Interesting)
And ... it's also kind of ironic that this relatively small patch often brings more practical benefits to the desktop than all the "big" desktop interactivity/latency features (cfs, swap-prefetch, -rt kernel) combined.
Talking a bout desktop interactivity, it seems you missed to mention Con's scheduler in gamming scenarios, curious eh?.
noatime on Mac OS X (Score:2)
Is this reason why we cant spin down disks? (Score:4, Interesting)
Ive been told that the kernel constantly needs to access the disk...
Is this the reason of something else prevents the disks from spinning down?
Re:Is this reason why we cant spin down disks? (Score:5, Informative)
And if you don't believe any of that you shouldn't have any trouble using google to find Admins who tell horror stories about having to reboot a drive and losing the entire drive because the bearings were shot to the point that once the disks stopped the motor couldn't generate enough force to restart them. But the disk could have lasted years more as long as it wasn't stopped. In fact in companies where a lot of data is stored the disks are put on their own power source at least partially because the disks don't have to be stopped if a server needs to be rebooted because of failure or updates. This is also one of the reasons to be wary of purchasing used storage arrays. Might have worked great when they shut it off, but you might be able to restart the array.
Re: (Score:3, Informative)
Syslog is often the culprit so if you can get it to stop writing log files all the time your disk can sleep longer. There is a syslog out there somewhere that limits it's writes to disk to when it fills a cache - the quick hack otherwise with an unchanged system is to get it to dump logs to a ram disk (kernel parameter on boot to enable ram disks) and sync every day or whenever.
Another thing is cron - work out
Mount noatime (Score:3)
Problem solved. Many older/slower machines [laptops] can be sped up considerably by this step.
atime on EXT3, huh? (Score:5, Funny)
$ time for i in `seq 1 10000`; do touch file1.dat; done
real 0m15.231s
user 0m3.075s
sys 0m11.970s
$ time for i in `seq 1 10000`; do cat file1.dat >>/dev/null; done
real 0m14.326s
user 0m2.928s
sys 0m11.172s
(remount without noatime flag)
$ time for i in `seq 1 10000`; do touch file1.dat; done
real 0m12.629s
user 0m2.687s
sys 0m9.772s
$ time for i in `seq 1 10000`; do cat file1.dat >>/dev/null; done
real 0m12.401s
user 0m2.624s
sys 0m9.624s
Yes I think I'll stick with atime for now, thanks Linus.
Re: (Score:3, Informative)
Re: (Score:3, Funny)
Accounting is a nuisance in general (Score:5, Interesting)
But sometimes you need it... Whether it is to project your savings or to figure out, if a particular file was read within the last year.
My problem with atime is that it is not universal enough. For example, reading a file via mmap() or sending it directly to a socket via sendfile() (both methods widely used by web-servers) will not update its atime. The access-timestamp should be updated every time a file is opened for reading, rather than when a read() is issued on it...
So, when I wanted to report, when my little piece of software was last downloaded (via HTTP), I could not, unfortunately, rely on the file's atime...
Re: (Score:3, Funny)
If you don't believe me RTFA. I just wish the OP had actually read the article before submitting it.