New Approach To Malware Modifies Linux Kernel

Hugh Pickens writes "Professor Avishai Wool has unveiled a program to watch for malware on servers with a modification to the Linux kernel. 'We modified the kernel in the system's operating system so that it monitors and tracks the behavior of the programs installed on it,' says Wool. Essentially, Wool says, his software team has built a model that predicts how software running on a server should work (pdf). If the kernel senses abnormal activity, it stops the program from working before malicious actions occur. 'When we see a deviation, we know for sure there's something bad going on,' Wool explains. Wool cites problems with costly anti-virus protection. 'Our methods are much more efficient and don't chew up the computer's resources.'"

Help!

It's stopped me from running Vista in a VM...

Re:

How is this off-topic? The mods must have been infected!

Re:Help!

That's nothing. I ran it in a virtual machine on Vista. It broke out, took over the machine, and put vista in a vm.

premise to shutdown

Is this not the very premise that caused the Amazon cloud shutdown? A failure to communicate back proper activity illogically deduced that there was an improper activity?

Re:premise to shutdown

This has greatly increased the online sales of weight-loss products although mostly from browsers identifying themselves as Internet Explorer

Linux users were terminated by their modified kernel after it detected that they were exercising ;)

selinux

Great, sounds exactly like what people have been doing with selinux and capabilities. But selinux acknowledges we don't always do the same things with our computers as the next guy... Will this approach be as flexible?

I don't want to boohoo his research, it's probably fine, but the article summary just gets my goat. Malware is a lot more complicated than most anti-malware software authors make them sound, and false positives are the biggest/most complicated problem they have to deal with, especially in automated systems that block like this...

Re:selinux

If you can detect and isolate a program that doesnt do what it is supposed to be doing and attracts attention with unusual behaviour, like scanning certain ports or trying to send packets etc, then it may be something that you dont have to have on your server. Most security programs today arent looking up what software does. They look what ports are used, what traffic is routed and so on.

These malware programs today try to hide themselves into deep that you just couldnt find them if you dont know what you are looking for. This system here as i understand it tries to identify what are the normal parameters for a certain program to work. If the program doesnt do or behave like a normal software, then there must be something wrong with it and alarms go off, lights are blinking and all hell brokes lose.

Oh crap, red lights and i hear noises. Oh it's only the cops.

Re:selinux

Well, from my basic reading of the paper it sounds like it won't have false positives but it also will miss many negatives. Essentially when you build it it'll make a map of what system calls can be made and in what sequence. If an application makes a system call it never calls or never can call in that order because it's been hijacked then this thing will stop it. If you manage to do your nasty business using the system calls it normally uses, it won't. Think of it as a auto-hardening system turning off any syscalls or combinations that the application doesn't use anyway. One of the downsides is that if you know this system is in place, you can probably add dummy syscall patterns to your exploit to match the application's behavior unless it's a syscall it never does. Still, there's little reason assuming an attacker is perfect and this is a worthwhile protection for the cases where it does work.

Re:

If an application makes a system call it never calls or never can call in that order because it's been hijacked then this thing will stop it.

The problem is that, for any non-trivial program, it is impossible for a static analyser to decide whether the software will call or not and in what order a specific trap. For instance, that static analyser is not able to tell you in advance at what time the exit syscall will be called.

Re:selinux

Or the application has been legitimately updated to do new things...

Well that part is handled. The call map is made when the application is compiled, so new source equals new map. That's at least one advantage over SELinux, this is completely automatic and it's always correct in that sense. Though if you want to get this with precompiled binaries, you also have to get precompiled maps so the distros have to help you out - it's not something you can set up on existing binaries on your own.

Re:What about plug-in based systems?

How does this thing deal with plug-in/add-on based systems like Firefox or Eclipse, where new capabilities get added to the executable through dlls (or java classes, I guess, in the case of Eclipse? - Although, with regards to Java, I wonder if this system would work at all, since I think the kernel never exactly 'sees' Java programs or classes as executables, but only the JRE, which already has all the system calls built into it?)

It's about servers here, I personally think one should really think thrice before installing plug-ins and add-ons on a server, and rather go browsing on a desktop machine. Regarding Java, I can see your point.

Re:selinux

This is actually quite different from selinux, although they both hinge on controlling what syscalls a program can make. The major difference here is that this system automatically builds an equivalent to the selinux rules at compile time, and therefore avoids a lot of the problems caused by poorly written selinux rules. An associated advantage is the ability to track whether a chain of syscalls is legitimate. Such context analysis could be very powerful if used properly.

However, you've probably already spotted the major flaws with this approach: the first is that it only works on compiled programs, which strikes me as a serious problem when you're talking about webservers. The second is that it doesn't work on certain classes of programs whose execution pattern is extremely difficult to predict: self-modifying code, highly dynamic code (longjmp is not allowed), etc. Another limitation is that it only works on statically linked libraries. Finally, it is totally dependent on GCC and friends, which could be a problem for it moving forward, and in groups where the intel compiler is preferred.

As for false positives, the entire point of this is that inside of these admittedly limited confines, it has no possibility for false positives. This system is not statistical in nature. It depends entirely upon the program itself to determine correct syscall behavior.

All in all, while it is a long way from a practical security model, it does offer the promise of powerful, accurate protection from certain classes of attacks. When combined with selinux and pcap on a system with a slim attack profile it could help to narrow the gap between being a zero-day compromise and having full protection.

Oh great.

They recently unveiled a unique new program called the "Korset" to stop malware on Linux...and once it reaches its full potential it could put anti-virus software companies out of business.

Doesn't our economy have enough problems? Do we really need to put Linux anti-virus vendors out of business? Next we'll probably drive the ice vendors in Alaska to bankruptcy!

Re:Oh great.

Oh, don't worry. Things are just warming up for the ice vendors...

How it works

From the papter: "The resulting model is an automaton that represents the legitimate order of system calls that an application may issue. This automaton is then enforced by Korset's monitoring agent, which is built into the Linux kernel, by simulating every emitted system call."

This is not likely to work for scriptable applications (Apache, Java-based servers, etc.) The order of calls is determined by the script, not the underlying executable.

Ummmm.....

....I thought that was the philosophy behind AppArmor (http://en.opensuse.org/Apparmor).

It's been deployed in SuSE products for years.

Re:Ummmm.....

....I thought that was the philosophy behind AppArmor (http://en.opensuse.org/Apparmor).

It's been deployed in SuSE products for years.

Apparmor seems to be a relatively sophisticated least-privilege system, i.e. the idea that if a BIND DNS server should never need to (for example) modify the routing table, then it also should not be able to modify the routing table. That way, if an attacker compromises said DNS server, he won't be able to do very much with it that isn't directly related to serving DNS requests (this is why I would personally refer to such a system as damage control, useful for containing/limiting an attacker who has already compromised something). The system discussed in the article is different in that it seems to be less concerned with what specific tasks a program should or should not be doing and more concerned with whether the code that is executed and the way that it is executed is what you would expect from the program's source. That way, if someone exploits i.e. a buffer overflow and inserts their own shellcode, it would deviate from the pattern that you would have expected from the exploited program and this deviation would be detected.

Both can be compared to systems like PaX [grsecurity.net] (kernel) and SSP [ibm.com] (userspace) which are intended to make sure that an attacker will fail to exploit an existing vulnerability, such as an unpatched buffer overflow, in the first place.

I'm definitely not an expert...

But this looks a lot like SElinux or AppArmor, except that the application profiles are constructed by static analysis of program code, rather than by hand, or by observing the app during a "training" period. The linked paper indicates that it is still in a rather rough state; but it looks quite promising.

Not a good article, but an interesting paper.

OK, what this is doing is watching for code injection attacks (buffer overflows, stack smashing, etcetera) by building a model of how each specific application is going to operate, and blocking system calls that the model of the application would never make. It seems like an interesting approach, though it may not be as useful on Windows where there's not such a formal distinction between system calls and other kinds of calls.

It won't do anything about interpreter code injection (eg, SQL injection or shell code injection) or script privilege escalation attacks (eg, ActiveX and other "cross zone" attacks in Internet Explorer), or attacks that involve complete executable code drops.

Still, this is useful and not nearly as dodgy as the article made it sound.

I might give it a spin

When & if Linus Torvalds (or whoever the benevolent dictator of the kernel is nowadays) includes it in to the main kernel source tree...

Sounds like a good idea to me, I just want to see what the Linux kernel pros think of it...

It's kind of like Vista

Sounds from the summary at least (hey, it's slashdot, I haven't read the article) that it's similar in some ways to the service profiling in Vista. The service profiling means that the dev looked at what the service needed to do to be able to run and gave it only those permissions, restricting the damage it could do if it were compromised. This seems to extend that to give the kernel the intelligence to baseline the services itself, and then restrict activity when the baseline activity changes.

Re:Heuristic scanning v2.0?

this sounds like the heuristic scanning features that has been in Norton Antivirus and other A/V utilities for almost a decade now, where it searches for out of the norm items and reports or blocks them, such as a program deciding to write to the MBR, or a program using raw disk I/O to write to the hard disk.

Wow, those "heuristics" sound like a simple blacklist of "virus-like" activities.

No, what this does is cleverer. It creates (at compile time) an automaton representing the system call activity of the program, and if the program tries to make a syscall that does not have a matching edge in the automaton, it kills it. Basically, if there is not a code path that should lead to execution of a certain syscall, the program gets killed.

Re:Just trying to understand

It creates (at compile time) an automaton representing the system call activity of the program

At compile time of the program? So in addition to a modified kernel you need a modified gcc and to compile everything from source or have a specialised distro? It doesn't surprise me that the summary should be lacking such details, but it would be nice if for once it gave a decent overview.

I agree that this was a poor summary but instead of complaining about the summary you could always do something crazy like read the article.

Re:Just trying to understand

I wonder if a fix for the buffer overflow (besides languages that make it harder) would be to separate the stacks used for local variables and return addresses.

The problem is that when a call is made to a function the compiler pushes the return address onto the stack. Then the function allocates space for its own variables on top of the same stack. If one of those variables overflows it can hit the return address. That essentially is a mixing of code and data. If you had two stacks then the processor could trigger an exception if anything writes to one of them except via a call or return. You could probably accomplish this via changes to the compiler without a processor change - the processor will always use the regular stack but a compiler could be designed to maintain a separate stack for local variables. You wouldn't have that read-only protection on the regular stack, but the two would be in different segments making an overflow impossible.

Other tricks that are used are things like canarys - values written onto the stack and then checked before a return - if there was an overflow the canary would not be intact. GCC has an option to do this which works most of the time.

Re:Completely incorrect basic assumptions

You're right. You can't exactly predict the behavior of a program without running it.

But that's not what this package is trying to do. Instead, it's trying to rule out large swaths of the behavior space of a program based on static analysis. Of course there will be false negatives -- i.e., malicious actions that remain undetected. But I don't really see how false positives would be a danger, modulo bugs in the static analyzer.

I imagine this package would be nearly useless for something like firefox, which does many varied tasks. But for programs like exim, or bind, or vsftp -- which do one task over and over again -- the degree of protection should be pretty good because there's a lot these programs don't do.