Follow Slashdot blog updates by subscribing to our blog RSS feed

 



Forgot your password?
typodupeerror
×
Linux

University of Minnesota Researchers Send Apology to Linux Kernel Mailing List (kernel.org) 208

Earlier this week Greg Kroah-Hartman of the Linux kernel development team banned the University of Minnesota from contributing after researchers there submitted what he called "obviously-incorrect patches" believed to be part of a research project into whether buggy code would be accepted.

Today the professor in charge of that project, as well as two of its researchers, sent an email to the Linux kernel mailing list saying they "sincerely apologize for any harm our research group did to the Linux kernel community." Our goal was to identify issues with the patching process and ways to address them, and we are very sorry that the method used in the "hypocrite commits" paper was inappropriate. As many observers have pointed out to us, we made a mistake by not finding a way to consult with the community and obtain permission before running this study; we did that because we knew we could not ask the maintainers of Linux for permission, or they would be on the lookout for the hypocrite patches. While our goal was to improve the security of Linux, we now understand that it was hurtful to the community to make it a subject of our research, and to waste its effort reviewing these patches without its knowledge or permission.

We just want you to know that we would never intentionally hurt the Linux kernel community and never introduce security vulnerabilities. Our work was conducted with the best of intentions and is all about finding and fixing security vulnerabilities... We are a research group whose members devote their careers to improving the Linux kernel. We have been working on finding and patching vulnerabilities in Linux for the past five years...

This current incident has caused a great deal of anger in the Linux community toward us, the research group, and the University of Minnesota. We apologize unconditionally for what we now recognize was a breach of the shared trust in the open source community and seek forgiveness for our missteps. We seek to rebuild the relationship with the Linux Foundation and the Linux community from a place of humility to create a foundation from which, we hope, we can once again contribute to our shared goal of improving the quality and security of Linux software... We are committed to following best practices for collaborative research by consulting with community leaders and members about the nature of our research projects, and ensuring that our work meets not only the requirements of the Institutional Review Board but also the expectations that the community has articulated to us in the wake of this incident.

While this issue has been painful for us as well, and we are genuinely sorry for the extra work that the Linux kernel community has undertaken, we have learned some important lessons about research with the open source community from this incident. We can and will do better, and we believe we have much to contribute in the future, and will work hard to regain your trust.

Their email also says their work did not introduce vulnerabilities into the Linux code. ("The three incorrect patches were discussed and stopped during exchanges in a Linux message board, and never committed to the code.")

And the email also clarifies that their research was only done in August of 2020, and "All the other 190 patches being reverted and re-evaluated were submitted as part of other projects and as a service to the community; they are not related to the 'hypocrite commits' paper. These 190 patches were in response to real bugs in the code and all correct — as far as we can discern — when we submitted them... Our recent patches in April 2021 are not part of the 'hypocrite commits' paper either."

UPDATE (4/25): Late Saturday night the kernel team's Greg Kroah-Hartman rejected the apology, writing that "the Linux Foundation and the Linux Foundation's Technical Advisory Board submitted a letter on Friday to your University outlining the specific actions which need to happen in order for your group, and your University, to be able to work to regain the trust of the Linux kernel community.

"Until those actions are taken, we do not have anything further to discuss about this issue."
This discussion has been archived. No new comments can be posted.

University of Minnesota Researchers Send Apology to Linux Kernel Mailing List

Comments Filter:
  • Pointless Apology (Score:5, Insightful)

    by Kobun ( 668169 ) on Saturday April 24, 2021 @10:47PM (#61310610)
    Much easier for code reviewers to ban these three for life, than to have to triple check every patch they submit for Underhanded C shenanigans.
    • Double secret probation?
    • by jwhyche ( 6192 ) on Saturday April 24, 2021 @11:02PM (#61310636) Homepage

      Fuck'em

    • Re: (Score:2, Interesting)

      by Anonymous Coward

      File the criminal charges to shut them up if they haven't got the hint that nobody can trust the asshole who cried wolf with critical infrastructure. They could've killed people.

    • by Krishnoid ( 984597 ) on Sunday April 25, 2021 @12:26AM (#61310804) Journal
      Still, they did clearly seek to deceive, so punishment is in order [youtu.be] in any event.
    • by tysonedwards ( 969693 ) on Sunday April 25, 2021 @12:27AM (#61310810)
      How is this any different than adversarial security researchers evaluating the change control process to protect against the next the SolarWinds or Codecov issue? Sure seems like this was a sanctioned task with oversight and zero intent of actually allowing the malicious code from being merged. It showed that significant portions of the system worked, while others are prone to human error and trust exploit. This experiment showed several flaws in the change control and approval process that a true malicious actor could target to get a bug committed that should be taken seriously, not simply ostracizing those who questioned if the system was unflappable before moving on.
      • Re:Pointless Apology (Score:5, Informative)

        by drinkypoo ( 153816 ) <drink@hyperlogos.org> on Sunday April 25, 2021 @08:02AM (#61311462) Homepage Journal

        Sure seems like this was a sanctioned task with oversight

        From the point of view of the university, it was. But this is in fact the actual problem. Because from the view of the people maintaining the kernel merge process, it absolutely was not sanctioned. If they had discussed this with maintainers ahead of time, then they could have avoided the entire flap, and the entirely justifiable ban on commits.

        This experiment showed several flaws in the change control and approval process that a true malicious actor could target to get a bug committed that should be taken seriously

        The code was not approved, and therefore demonstrated the security of the change control and approval process.

      • by HiThere ( 15173 ) <[ten.knilhtrae] [ta] [nsxihselrahc]> on Sunday April 25, 2021 @09:02AM (#61311638)

        They've lied and deceived. How can you trust them? Why should you trust them? What makes you believe this is an honest apology rather than just an attempt to regain submit privileges?

        And when they say the "hope to do better", that could be interpreted in many ways, not all favorable to the Linux community.

        A genuine apology comes with amends.

    • by Anonymous Coward
      The WHOLE UNIVERSITY was banned, as they should be. If there are no consequences to organizations they won't take this shit seriously. If your research involves going submitting Linux kernel patches, then just go to a different university. Pretty much any Big 10 university is of equal quality and there are many universities that are better.
    • by mysidia ( 191772 )

      Much easier for code reviewers to ban these three for life
      Except their patches closed security issues, and the reversion or rejection of their fixes can potentially lead to weaponization of known bugs regressions will reintroduce.

      Banning also does not mean these 3 cannot harm the kernel -- their work is to find security bugs, so they could simply find more, and focus their work on developing exploits for the bugs they find instead of fixes, then keep the exploits secret and sell the details to the highest

    • Re:Pointless Apology (Score:5, Interesting)

      by tlhIngan ( 30335 ) <slashdot@worf.ERDOSnet minus math_god> on Sunday April 25, 2021 @03:04AM (#61310976)

      Yeah, it was just an apology, but nothing more. I saw it as "We're sorry, no harm no foul, all good, ok?".

      If they wanted to apologize, they should've offered something in return - resources of some kind. I'm sure they could've donated a few thousand dollars to the Linux Foundation or find some other means to help out the community. That would show a real apology for the extra time and effort their work was doing.

      And honestly, informed consent is possible - I'm sure they could've talked to say, Linux himself to help oversee the entire process. Linux might not be involved in the day to day coding which means he's no longer overseeing the patches, but at least he'll be in the know and be notified which patches would be the "bad" patches or ones that might need further review later. A maintainer who doesn't deal with the day to day, but still has the power to fix things.

      Even getting in touch with Greg would be possible to alert him to the fact - he could easily arrange for someone else to take care of his duties during an agreed-upon time period (since everyone needs a just-in-case-of-bus) and seeing how things work out.

      If they really cared, they'd solicit reparations from the community - not just a mea culpa. Money, resources, anything else the kernel developers might want or need. Or even just donations to the EFF, EPIC or other group. These guys benefited because of their paper, at the expense of the kernel development community as a whole. An apology isn't enough.

    • by gosso920 ( 6330142 ) on Sunday April 25, 2021 @03:55AM (#61311054)
      Dear Linux community,

      We're sorry we got caught.

      Sincerely, The University of Minnesota

    • by Mitreya ( 579078 )

      Much easier for code reviewers to ban these three for life, than to have to triple check every patch they submit for Underhanded C shenanigans.

      They probably won't try that.
      But also, according to the paper, they used random Gmail accounts to submit the three "hypocrite commits".

    • Let's not forget the graduate student accusing Greg Kroah-Hartman of "preconceived biases" in the original email response. Because if there's one way to get attention off something you did wrong, is to accuse the opponent of racism.

    • Hee hee, let's require them to openly support Stallman and the Free Software Foundation as penance.

  • by Ostracus ( 1354233 ) on Saturday April 24, 2021 @10:52PM (#61310614) Journal

    I'm sorry. [youtu.be]

  • "The Research" (Score:5, Insightful)

    by erexx23 ( 935832 ) on Saturday April 24, 2021 @10:56PM (#61310618)

    and the research says:
    Trust Broken.

    • Re:"The Research" (Score:5, Insightful)

      by rtb61 ( 674572 ) on Saturday April 24, 2021 @11:01PM (#61310630) Homepage

      Ain't no coming back from that, banned for life, every person involved and a ten year ban on the university. They were trying to be dicks, trying to make a name for themselves, get themselves memed.

      They admitted it, so honestly report it to the FBI as attempted fraud, the harm, what would have happened if it had gotten through, the cost of assessing broken software as broken, all ad up to damages, ACROSS A COMPUTER NETWORK, which makes it a criminal act and the Linux Kernel team should make an example of them by reporting the attempted fraud to the FBI and seeking criminal prosecution of the individuals involved.

      They must take action to warn others off, else they will end up flooded with that rubbish. Fraud was attempted and it should be prosecuted.

      • Re: (Score:3, Insightful)

        Comment removed based on user account deletion
      • I've never witnessed a bigger masturbatory fantasy. You really want to start pinning damages on shitty code? Oh like that won't backfire.

        • You really want to start pinning damages on shitty code?

          Yes, please. For example, anyone who writes an SQL injection bug is negligent. There are techniques to avoid that.

        • by Zuriel ( 1760072 )
          Not just shitty code, code where the developer has gone on to write an academic paper titled "I deliberately wrote malicious code!" discussing their commits.
        • by Shinobi ( 19308 )

          Yes. Code, as critical to the function of society as it is now, should be held to the same standards as any other engineering. Kill off all the negligient early releases, and the "Oh, not my problem any more" mentality. Make software engineers be held to engineering standards and ethics, both for safety and for privacy.

          If you object to that, you are part of the negligence culture that infests software development like a cancer.

        • You really want to start pinning damages on shitty code?

          In short, yes.

          However, only code you pay for.

          If you pay for OSS, then whoever you paid should be liable for the quality of the code.

          If you're paying for customizations that should be obvious from your contract, and whoever you paid should only be liable for the quality of the code in the customizations.

          There's no reason why the incompetent should be permitted to create insecure conditions for everyone else without repercussions.

  • by raymorris ( 2726007 ) on Saturday April 24, 2021 @11:07PM (#61310646) Journal

    > These 190 patches were in response to real bugs in the code and all correct â" as far as we can discern

    It looks like they are still intentionally pretending the other problem doesn't exist - most of their patches do nothing at all. They don't fix any bugs. They just waste people's time.

    They've been running an automated tool and send it the output as "needed patches" without *looking* at the output first. For non-programmers, it's kinda like running a grammar checker, which makes suggestions such as changing "it's not untrue that sometimes ..." to "it's true that ...", and then just "sometimes ...". That MIGHT be a better way to phrase it some cases. On the other hand, if the proceeding sentence is "the president said that people always", it may be the author really does want to point out that's not a lie.

    Another example would be that C doesn't have the word "unless", only "if". So sometimes the clearest way to write something is to use a double negative, as a way of saying "unless":

    If (not failed) ...

    An automated tool trying to "simplify" the code by eliminating the double negative can easily be making it *harder* to read. You gotta actually look at what the tool is suggesting might be changed. Sending each one in without looking at it first is just wasting people's time.

  • by slincolne ( 1111555 ) on Saturday April 24, 2021 @11:15PM (#61310662)
    The possibility of damage to their reputation and possible legal action must have them concerned. It's not like the Linux Kernel is hobby project by a university student any more. An apology is cheaper than legal action.
  • by rebill ( 87977 ) on Saturday April 24, 2021 @11:19PM (#61310668) Journal

    Phase II of their security test:

    Public Statement: We are sorry. Please trust that the other things we submitted are safe.
    Private Thought: Let's see if these fools will believe us.

  • IRB (Score:5, Insightful)

    by The Evil Atheist ( 2484676 ) on Saturday April 24, 2021 @11:19PM (#61310670)
    Now the university just needs to apologize for its Institutional Review Board's failure to identify such glaringly unethical research methods and pledge to re-review all its approvals for the past few years, not limited to computer science research. The review board was obviously negligent and may have allowed research to go ahead that shouldn't have. The apology would also need to pledge to have stricter standards and reviews processes.

    Our recent patches in April 2021 are not part of the 'hypocrite commits' paper either.

    No, but that doesn't mean you weren't trying to do further research along similar lines for a newer paper. No one can tell, and in the interest of security, must treat all future attempts at contribution with suspicion.

    • Re:IRB (Score:5, Insightful)

      by The Evil Atheist ( 2484676 ) on Saturday April 24, 2021 @11:35PM (#61310708)
      I would also add that, beside the security issue, it would serve as a good example for other researchers not to try the same thing. Imagine if GregKH didn't ban them - it would send the signal that the Linux developer community would be tolerant of such things and even less ethical researchers will be temped to try other shit. Kernel development would be seriously halted. In fact, the initial response that was less punitive seemed to have encouraged substandard patches such as the one that triggered the ban, so it's better to cut it off at the root than allowing it to grow further.

      As a wise man once said, "Speak softly, and drive a big tank."
    • Now the university just needs to apologize for its Institutional Review Board's failure to identify such glaringly unethical research methods

      Do we even know for a fact that the researchers in question submitted their experimental plans to the IRB? I've worked for a number of universities and research institutes myself, and while IRB review was technically required to do any research involving human subjects, the IRB did not have any proactive powers to police research groups. Rather, the IRB relied on thos

  • Comment removed (Score:4, Interesting)

    by account_deleted ( 4530225 ) on Saturday April 24, 2021 @11:25PM (#61310686)
    Comment removed based on user account deletion
  • People constantly introduce bugs in the kernel without intention and some go unnoticed. It was obvious that it was possible to do it on purpose.
  • An apology should be the first thing you send when you realize someone else seems hurt. Regardless of how justified you feel your actions were. An apology is not an admission of guilt.

    • A weasel worded apology might not be, but I'd say admitting guilt is a large part of an honest apology.
      • admitting guilt is a large part of an honest apology.

        It's also a large part of publishing an academic research paper. :)

      • You two are using two different definitions of "guilt"
        • Can you help further by clarifying the difference? I've thought about it for a bit and can't figure out what you mean. Thanks :)
          • I'm pretty sure he's talking about metaphysical guilt. Guilt on one's soul. And you are talking about more practical guilt. That is, if you drop a bowl of milk and you say it was your fault, I don't think he considers that "admitting guilt".

    • An apology should be the first thing you send when you realize someone else seems hurt. Regardless of how justified you feel your actions were. An apology is not an admission of guilt.

      Absolutely wrong on all counts. Here's a nickel, get yourself a bookmark to a dictionary. And give me back my nickel, because bookmarking is free.

      Apology means "a regretful acknowledgment of an offense or failure". It absolutely is an admission of guilt. You should not send an apology simply because someone else seems hurt. If you want more information and don't want to admit guilt you can send an acknowledgement that they seem hurt, and accompany it with a request for clarification.

      We probably shouldn't ta

  • However, instead of actually making positive kernel changes, someone had an agenda, and they are receiving exactly what they deserve.

    Now, if criminal charges are filed, someone's probably going to go to PMITA prison.

    Which according to current law, is correct.

    So, an apology may be analogous to a confession.

  • by peppepz ( 1311345 ) on Sunday April 25, 2021 @12:24AM (#61310796)
    Now they apologize, but their first reaction had a different tone:

    I respectfully ask you to cease and desist from making wild accusations hat are bordering on slander.

    These patches were sent as part of a new static analyzer that I wrote and it's sensitivity is obviously not great. I sent patches on the hopes to get feedback. We are not experts in the linux kernel and repeatedly making these statements is disgusting to hear.

    Obviously, it is a wrong step but your preconceived biases are so strong that you make allegations without merit nor give us any benefit of doubt.

    I will not be sending any more patches due to the attitude that is not only unwelcome but also intimidating to newbies and non experts.

    Their first reflex was a call for cancellation. I think that only because of the quick and firm response by the kernel community this story wasn't picked up by some news site or medium blog and spun as a "mean geeks have no social skills" story. For now.

  • by misnohmer ( 1636461 ) on Sunday April 25, 2021 @12:25AM (#61310800)

    How is banning the researchers solving the problem they exposed? If they can do it, so can other people, and for sure large state actors. So rather than focus on the actual problem, the public is pacified by the researchers being banned. Problem solved, right? Reminds of when a researcher gave a Blackhat presentation on NUL prefix browser https bug, he demoed it by impersonating PayPal. PayPal's response? "We banned the researcher from PayPal, problem solved".

    • by lessSockMorePuppet ( 6778792 ) on Sunday April 25, 2021 @12:48AM (#61310834) Homepage

      It solves the individual problem of one person who doesn't understand that you don't break other peoples' toys without their permission. If you want to break your own toys, go right ahead.

      The other systemic issues should be addressed separately.

      • There have been multiple slashdot posts on banning these guys, yet nothing on the solution you speak of. So please share a link to the upcoming solution to the systemic issue of malicious insertion of backdoors (yes, inserting vulnerabilities is inserting backdoors). It would make for much more interesting slashdot post, as that is the hard to solve problem at hand here - banning is easy but solves nothing, unless the problem is that you really believe the University of Minnesota was intending to do it agai

        • by Mitreya ( 579078 )

          There have been multiple slashdot posts on banning these guys, yet nothing on the solution you speak of.

          I think the problem is that there is no real solution besides more human review. Solutions such as updating code of conduct or solving understaffing by encouraging more people to join OSS maintaining seem underwhelming:
          https://github.com/QiushiWu/qi... [github.com]

    • How is banning the researchers solving the problem they exposed?.

      Ask yourself this. How is continuing to allow them to submit patches solving the problem? Creating a system where people volunteer to contribute puts ultimate responsibility for those contributions on the creator of that system. The creator of that system will then rely on "good faith" intentions just like any democracy, to help with the daily running of that system. Take away "good faith" then what should be the standard? The assumed privilege enjoyed by the contributor? In fact where does anything less of

  • Are they sorry they did it or sorry they got caught?

    Smells like the latter.

  • "A literal experiment!"

  • "You know I would never intentionally hurt you."
    "Can you forgive me?"

  • by delirious.net ( 595841 ) on Sunday April 25, 2021 @05:22AM (#61311250)
    but you're still banned!
  • Nothing in the "apology" letter about this, other than he's a co-signer of said letter. An adult in a PhD program has this level of professional malfeasance doesn't deserve a degree, at least not for some good time of good-behavior shown.

    • Everyone knowingly involved, or anyone who knows now and continues to defend this behavior, should have their Ph.D.s stripped. Individually, if they recant and perform some atonement, they can keep the existing degree or, if they don't have one, can produce a completely new thesis and start again.

      Whatever this was was not an honorable or ethical attempt at bringing the light of knowledge into the world.

      • On second thought, there's no way to know if anyone involved did legitimate work on their degree, so we need to revoke them all until they can prove they didn't doctor their doctorates (for the PI and so on).

  • ... (we) sincerely apologize for any harm our research group did to the Linux kernel community ...

    ... we knew we could not ask the maintainers of Linux for permission ...

    Easier to ask for forgiveness than permission.

  • who wanted to show how smart they were by putting a back door in the kernel (after all the rubes would never notice), then announcing it and collecting the Congrats and Ego boost. And calling it research!
  • "Sorry we got called out."

Keep up the good work! But please don't ask me to help.

Working...