More Info on Debian.org Security Breach

mbanck writes "James Troup (part of the Debian System administration team) has published more information on the recent compromise of four debian.org machines. The attack vector seemed to be a sniffed password of an unprivileged account, from which the attacker somehow managed to gain root and install the suckit rootkit and crack the other machines. As the machines were fairly uptodate with respect to security, an as-of-yet unknown local root exploit might be in the wild, so keep an eye on your boxen.Note that the main ftp archive running on a sparc machine was not compromised, so the exploit might not yet be ported to non-i386 architectures."

Great

[headbulb]

Right as I am downloading Debian.
I will check the md5sum.

Anyways Something to be said about passwords.. I am getting sick of passwords.. I have looked at the RSA keychains, But they cost too much.

So I ask are there any good one time password systems out there. That are opensource.. I have looked at going with smart cards but again with the money. (not to mention overkill for me)

I have found a few but none with a keychain.. I don't mind paying for a keychain, but I want the software to be opensource.

Root password

[phorm]

Once an infiltrator is in a machine, it is often just a matter of time before he acquires root access - unless monitoring or disablement are standard procedure.

Depending on the power of the box and the time from which the lower-level account was compromized, it could just be that a password-cracking procedure gained root access. Of course, it's also possible that the attacker managed to nab control of a process running as root, but again the initial compromise still required cracking a password to gain access to the machine.

First rule, secure your passwords... and it's probably not a bad idea to use a password cracklib to ensure that any semi-privileged (can SSH) users have somewhat secure passwords as well.

Re:But wait....

[Aussie]

When has windowsupdate ever been compromised?

Windows Update is served by Akamai ie Linux.
Now what was your point ?

local root == remote root

[Markus Registrada]

This is a good demonstration that the distinction always made between local privilege-elevation bugs and remote exploits is academic hair-splitting. It's rarely difficult to get unprivileged access through a buggy non-privileged service. (Web-server plug-ins are a reliable source of entry points.) Once you're in, privilege elevation takes you the rest of the way.

Certainly the distinction is useful to security students and analysts, but it's misleading for everybody else. "Oh, that one's just a local exploit; not so bad." The OpenBSD advocates promote the fallacy: "only one remote exploit in this millennium!" (or something like that), encouraging us to ignore almost equally damaging exploits in non-core services that provide access to local accounts and more damaging attacks.

There's a similar fallacy in distinguishing security holes from other bugs. Without a depth of analysis that hardly anybody can ever afford, almost any bug might actually be a security hole, too. The OpenBSD people get this one right -- to them, any bug is a security hole until proven otherwise, and they encourage running latest versions -- but almost everybody else gets it wrong. When I fixed a double-free segfault in lib[mumble], nobody posted security warnings about every program that relies on it. despite that double-free bugs can often be exploited.

Debian gets this wrong, and very selectively backports only proven security holes, ignoring the myriad bugfixes that might just as easily be security holes as well. To find holes in stable-branch services, just look for bug fixes in later versions, particularly in libraries used by those services. Failing that, look at new features added shortly before the library-version used. Chances are the last new feature added has bugs that haven't been noted yet, and that might be exploitable.

This might be a good place to mention that the CVS codebase is almost irreparably insecure. The practical implications are: (1) A remotely-accessible CVS server should never be run on a host that does anything else that matters, or that has access to anything else; (2) An anonymous CVS server should never be the same CVS server that is used for checkins, or even run on the same machine. The pserver should be a slave that only gets read access to a copy of the archive. (3) Checkins on remotely-accessible servers should result in patches logged to another archive kept on another, not-remotely-accessible machine. Patches from that server should be posted to the mailing list.

A quick question...

[kinema]

Just out of curriousity, how much logging data would servers like the ones in question produce per day/week/month? It's got to be quite a bit with all the various packages like an httpd, ftpd, MTA and all the typical syslog data.

Sad day for Debian

[swordsaintzero]

As long as a machine is connected to the internet there is going to be a method to compromise it. My question is this why Debian? They are the only Linux distribution that is truly built by volunteers to gain any mindshare of real note. (not sure about slack so please dont sick bob dobs on me) This is not imhop the work of rank amatuer crackers with there first root kit. These were servers being run by experienced admins using a distro known for stability which when patched and up to date usually means somewhat difficult to hack. I seriously doubt these guys were running winders attempting this either. Wtf is happening to the community when people with talent are attacking a distro that yet again imhop doesnt suck. These guys need to be found and buried. Not by the police but by the commmunity. Last but not least (places tinfoil hat on head) could this have been funded by M$ trying to discredit linux. I cant see the glory angle so its got to be money or power. (no glory in getting called a dick when you tell your friends what you did)

ldap?

[rsax]

Obviously we can't continue without LDAP accounts for very long either.

Can someone who's familiar with system administration on those debian boxes clarify the above statement? Have they disabled LDAP accounts or was it implied that they're going to set up authentication with a ldap backend in the future. If it's the latter then I'm curious as to how having ldap in the equation would have made cracking those system accounts harder.

Re:One recommendation

[suss]

One of the first things that get wiped in an intrusion are the logs

Try wiping logs printed out on a matrix printer...

Re:#1 on Ten Immutable Laws of Security

[SamNmaX]

I wish Microsoft had more to say about this "Law #1". I think the personal computer is in dire need of a serious security overhaul. Running a program shouldn't equate to giving it total access to your computer. Obviously it's impossible to cover for all issues, but something as simple as double clicking on an application (or attachment) SHOULD NOT be able to wreak havok on your computer. Most programs only access files that the user explicitly asks for in a dialog box, so why should they get access to everything? All programs should by default be around the equivalent of a sandbox, mixed with a UI that implicitly gives a program access only to files the user chooses in some manner. Only programs that absolutely need access to the entire system should get it, such as virus scanners. (which hopefully wouldn't be needed in such a system)

Re:Sad day for Debian

[trick-knee]

> Wtf is happening to the community when people with talent are attacking
> a distro that yet again imhop doesnt suck. These guys need to be found and
> buried. Not by the police but by the commmunity.

hear, hear.

it's not a sad day for Debian so much as it is for the community. if Debian can find this supposed new exploit, fix it and publish details, then Debian will rise a little higher in people's esteem.

but why crack Debian in the first place? here I am stumped, but then I've never fully understood the cracker mentality.

What's up with these anti-Linux attacks?

[PurpleFloyd]

To me, this attack and the recent attempt to insert an exploit into the Linux kernel [iu.edu] seem like possible evidence of a disturbing trend: skilled attacks against high-profile Linux sites (you can't get much higher-profile than kernel.org or debian.org). I'm pretty sure that these systems were secured against all known local root exploits; if they weren't, this probably would have happened long ago.

So, what's going on here? Are these simply two unrelated attacks? Is it an attempt by an immature highschooler with some cracking talent to boast to his friends "LOL 1 hax0rred debian.org!?" Is it an attempt by some sort of anti-Linux commandoes to undermine Linux's public image? I almost suspect the latter, but the prime suspect there is Microsoft, who have far too much to lose by going that route and plenty of money for traditional FUD that will make it into "traditional" news channels better anyway. SCO might be crazy enough to do it, but they probably wouldn't want to divert resources away from spewing lawsuits at everyone in existence.

From what I understand of the cracker community, Linux is held in fairly high regard (although I admit I don't try to keep up on the latest in the cracker community). You'd think that black-hats, who tend to be rather immature, when armed with a brand new exploit, would attack a site seen by the general public and post goatse.cx images on the front page, rather than subtly changing Debian packages. So, who's behind all this?

Re:But wait....

[Anonymous Coward]

the Debian mess is actually more like Microsoft's Windowsupdate, Officeupdate, and MSDN servers all getting rooted at once. Akamai uses Window servers too, as a heterogeneous network is more resistant to this kind of attack. If the root attack that was used is portable across architectures (and there's only circumstantial evidence that it's not) then Debian is doing users a disservice by not running a heterogeneous network.
Given the resources, the Debian admins should run OpenBSD and Windows 2003 machines alongside the (evidently insecure) Debian servers.

Re:Human Error

[Anonymous Coward]

In practice, a single local privelage escalation attack is all it takes. Maybe this will end up being a good thing in the end, we get to find a previously unknown local root exploit, fix it and improve the Debian security practices, all in one move.

So when an exploit is found in Windows, it is considered a bad thing that shows how lame of an OS it is.. but when it is found (or not?) in Linux it is a good thing?

Ooooook. I know I know, I must be new here.

Re:In a nutshell - somehow

[Anonymous Coward]

For some reason, everyone else seems to be overlooking the fact that there is (or appears to be) an unknown root exploit out there.

Uhm, did you read James' post? Here's a quote:

Unfortunately due to the fact there is (I believe) an unknown local root exploit in the wild, we can't yet unlock the Debian accounts.

Surely this constitues something else than "overlooking" the root exploit? Deciding to keep the Debian accounts disables effectively stops the entire developement of Debian. Nobody has been able to upload packages in the last week, and lots of services are down.

James could have unlocked the accounts to make the developement pick up again rapidly (which would probably would be the only option in a corporate setting -- there's a release schedule that must be kept at all costs), but the admins are being thorough on this one.

In summary: James (and the other admins) are keeping the entire Debian Project in suspense for the purpose of tracking down this local root compromise and preventing it from being exploited again. You might want to think about that for a second, and see if "overlooking [the] unknown root exploit" is applicable here.

Debian physical site security?

[identity0]

Okay, I read the article and it said that at least one machine was at a remote location that couldn't be accessed - can anyone tell me what kind of physical setup debian project uses? I always get the impression that they're based out of some dude's dorm or basement, like in this OpenBSD image [openbsd.org]. Do they have any physical security measures at all around their boxes?

Re:What's up with these anti-Linux attacks?

[Anonymous Coward]

Guess what? People are assholes. There are plenty of people in the world who want to make a quick buck or have a laugh at someone else's expense. They don't give a fuck about open source moral high-ground, they don't give a fuck about Microsoft's hegemony, they don't give a fuck about you. They want to break your shit, take your identity, steal your money and send kiddie porn spam from your box.

Wake up. Linux has become more popular, more mainstream and there is much to be gained by owning your boxes. The most surprising thing about all this is that people are surprised it's happening. It sure was nice when Microsoft was the only target in the world but times have changed. It's time to grow up and realize that this is everyone's problem.

Re:Human Error

[dasunt]

Er, the problem with biometric identification is that (1) its not testing who you are, just that the digital input matches some value and (2) you can't change what its testing.

You can't change who you are. Thus, once the key is compromised, it stays compromised.

Re:Diebold, take note

[oo_waratah]

"Viewed from this perspective, I don't think we need to keep an eye on our boxen just the backup tapes / disks/ CDs."

But how will you know unless you monitor it? Being able to recover a problem is a long way from identification.

Re:Sad day for Debian

[Anonymous Coward]

Blaiming debian crack on M$ tactics = paranoid.
Blaiming Linux kernel CVS hack on M$ = paranoid.
Blaiming SCO shenanigans on M$ = paranoid.

Put the three together and maybe we shouldn't think it's paranoia, after all, who has the most to gain by discrediting Linux?

suckit ...

[Pegasus]

This reminds me of a shit we had back in the april at the place where i work. We got a couple of production server r00ted with suckit, with the only possible attack vector being apache/php (only port 80 was open in the firewall), that were latest versions back then. The only way to stop it was to recompile a kernel without modules support and some minor patches to deny writes to /dev/kmem in any possible way ... therefore killing the method suckit uses to load itself. See point 6 here [phrack.org] and here [epita.fr].
There were quite a lot of similiar reports from the folks all aronud at that time ...

My big hairy conspiracy theory would be in the line of super zonda type of organization hiring some of the most skilled crackers and r00ting the boxen all around ... for spamming, ddosing or whatever ... welcome to the Wild Wild Net.

Re:Human Error

[Xerithane]

A good method: Easy mental ciphers.

You pick a passphrase that you use for all of your systems. You then pick a unique seed for each system. Then, you do some quick mental math on it (pick an algo of your choice, just make it simple) and then you have the effective security of two passwords + unknown algorithm. It will make all of your passwords invulnerable to dictionary attacks (unless a rare circumstance has your resulting password being "password" or something)

For example, if you have a pass phrase of "MYBOXISSECURE" then you can use the box name as a seed, lets call the box "DEBIAN" and have the algorithm block the seed and then subtract, modulo 26.

MYBOXISSECURE
DEBIANDEBIAND
-------------
I'm too tired to do this and I'm on my windows sytem without perl.

Then reverse it or something. Walla! Pseudo-random passwords. Works great, and after a few times you will memorize the keystrokes and you won't need to do it by hand. You can even have a standard system for the passphrases amongst an entire group for the root password, so each system can have a different root password that everybody can just figure out as long as they know the passphrase. In addition, if you want to remove someone from the loop, just change the passphrases and redistribute to the trusted source.

It's a hack solution for the weak-password problem.

Re:Human Error

[swillden]

The biometric information is not used *AS* the encryption key.

And there's a good reason for that: It wouldn't work. Every time a biometric is scanned, the result is different. Biometric matching is hard because it's a process of evaluating the "closeness" of the livescan to the stored template and then deciding whether the two are close enough to be considered the same.

This means that trying to extract a set of bits from the scan which you could be sure would be the same every time is very difficult, and likely wouldn't net you many bits to use as a key. A set of bits that changes a little every time doesn't make a useful key.

Given some sort of a secure processor, you can store the key and the biometric template in there, and program it to refuse to use the key until it has been presented with a biometric scan which it considers to be close enough to the template. That gets you about half way to security, now you just need to find a way for the secure processor to verify that the livescan it receives is fresh, and not replayed. Oh, and it would be good if you could also be sure the livescan is a *live* scan. And don't forget to secure that template database well.

Making biometrics secure is hard. In practice, this means biometrics are only useful in two situations. The first is very low security, where the biometric is being used to raise the level of security from very, very low to very low. The second is very high security, where the biometric is to augment some other authentication methods, or when verification is only done in a very controlled environment, i.e. where you're watched closely by a human guard who knows how to ensure you're not trying to fool the scanner.

time of attack

[Anonymous Coward]

take a look at the time of the attacks.

The attacker stoped at 19:00h and started back at 5:00h.

Now from this time, and assuming the attacker is like most computer guys I know, he would sleep around 2-3 AM, and wake up around 11-12.
This would place the attacker at 6-8+ GMT hours.

that is, in china, Mongolia, Rusia or Indonesia.

Re:Human Error

[orcrist]

For some bizzare reason, I haven't found it necessary to be able to do that. All you need to do is learn how to make hard-to-guess, easy-to-remember passwords:
Choose a quote or sentence, take the first (or second if you really want it to be hard) letter of each word, use numbers instead of letters for words like 'to', and alternate capitalization for the rest:

"To be or not to be, that is the question" becomes
"2bOn2BtItQ" which should defeat any dictionary based attacks, and is incredibly easy to remember. Of course I also choose somewhat more obscure quotes or make up an interesting sentence.

-Chris