Follow Slashdot blog updates by subscribing to our blog RSS feed

 



Forgot your password?
typodupeerror
Red Hat Software Intel Hardware

Red Hat Reverts Spectre Patches to Address Boot Issues (bleepingcomputer.com) 78

An anonymous reader quotes BleepingComputer: Red Hat is releasing updates for reverting previous patches for the Spectre vulnerability (Variant 2, aka CVE-2017-5715) after customers complained that some systems were failing to boot. "Red Hat is no longer providing microcode to address Spectre, variant 2, due to instabilities introduced that are causing customer systems to not boot," the company said yesterday. "The latest microcode_ctl and linux-firmware packages are reverting these unstable microprocessor firmware changes to versions that were known to be stable and well tested, released prior to the Spectre/Meltdown embargo lift date on Jan 3rd," Red Had added.

Instead, Red Hat is recommending that each customer contact their OEM hardware provider and inquire about mitigations for CVE-2017-5715 on a per-system basis. Besides Red Hat Enterprise Linux, other RHEL-based distros like CentOS and Scientific Linux are also expected to be affected by Red Hat's decision to revert previous Spectre Variant 2 updates, so these users will also have to contact CPU/OEM vendors.

At least one site "characterized the move as Red Hat washing its hands of the responsibility to provide customers with firmware patches," writes Data Center Knowledge, arguing instead that Red Hat "isn't actually involved in writing the firmware updates. It passes the microcode created by chipmakers to its users 'as a customer convenience.'" "What I would have said if they'd asked us ahead of time is that microcode is something that CPU vendors develop," Jon Masters, chief ARM architect at Red Hat, told Data Center Knowledge in a phone interview Thursday. "It's actually an encrypted, signed binary image, so we don't have the capability, even if we wanted to produce microcode. It's a binary blob that we cannot generate. The only people who can actually generate that are the CPU vendors."
This discussion has been archived. No new comments can be posted.

Red Hat Reverts Spectre Patches to Address Boot Issues

Comments Filter:
  • I didn't knew it had the reboot feature!

  • by klingens ( 147173 ) on Saturday January 20, 2018 @01:10PM (#55967819)

    As written in the summary, the CPU maker is the only entity who can create a fixed microcode, be it for inclusion into firmware (BIOS/UEFI) files or in Linux kernel microcode files. So asking the OEM vendors won't help one bit cause they can simply do nothing at all except use a file create by Intel. If RH is too small to force Intel to create working microcode without boot up bugs, then others aren't big enough either.

    Then next thing is: what will RH do in the future? Never apply the Spectre microcodes due to instability? If so, what happens in the future when other microcode updates are needed, like before? Intel has afaik only a single microcode file for Linux for pretty much all CPUs together. There is no mix and match, no way for Linux to selectively choose what to load. That is why RH had to go back to an older version without the breakage for Spectre. they couldn't just disable the new buggy part of the microcode.
    This file you can see and download here: https://downloadcenter.intel.c... [intel.com]
    Normally it lives in your initrd.
    Sooner or later RH has to include a current microcode file from Intel in RHEL again. Would have been nice if they had clearly communicated this to their customers. Not "wash their hands" but "we will continually work with Intel until this issue is resolved."

    • by ls671 ( 1122017 ) on Saturday January 20, 2018 @02:20PM (#55968153) Homepage

      Actually, here is the approach we have adopted with regards to this problem; Don't apply the patches until this is sorted out, I give it about 3 months before we can make up our minds but any guess is as good as mine.

      Something that critically linked to the hardware as to be tested with all available hardware and configurations which is pretty hard to do. So, as usual, let the users do the testing and the testing cycle isn't completed yet! ;-)

    • red hat didn't cause this bug, they are just an OS. why can't intel put the microcode file up for download on their website? Oh wait they did [intel.com]

    • Under Oracle Linux 7.4, the command "ls /lib/firmware/intel-ucode/ | wc -l" reports 95 files in that location.

      You can run "rpm -qi microcode_ctl" and "rpm -ql microcode_ctl" to get more information about what these are.

    • by joemck ( 809949 )

      This looks like a case of Red Hat telling its customers to go tell their OEMs to go tell Intel they broke stuff with their microcode patch and need to fix it. It's a common strategy to take if you don't think you have enough leverage telling them yourself.

    • by sjames ( 1099 )

      I imagine they will if/when Intel gives them a blob that doesn't make systems crash. I can understand their frustration, Intel hands them a blob of microcode but provides no way to debug or even understand the contents. It fails miserably and RH is left holding the bag with it's customers. They get to deal with the angry calls and the hit to reputation while Intel skates.

      If they punt to the OEMs, then the OEMs get to deal with the angry calls and perhaps re-thinks Intel as a CPU source. More likely, they a

  • by Alain Williams ( 2972 ) <addw@phcomp.co.uk> on Saturday January 20, 2018 @01:19PM (#55967859) Homepage

    I understand why they stopped distributing the microcode. It is something that they cannot test fully, cannot fix and which might brick customers' hardware. If something goes wrong I expect someone to try to sue them. Distributing the microcode bring little benefit (to RedHat) but brings potential risk/cost.

    The onus is really on AMD/Intel/... to test the new microcode on *every* CPU that they have produced and with a large selection of releases & configurations of operating systems (MS Windows, Linux, macOS, ...) with different support chips (RAM, USB, ...). Yes: a lot of work, but they are the ones who screwed up here. They have also known about these problems for some 6 months - so it is a complete disgrace that they have not already produced the microcode updates AND tested them.

    Hopefully in a month or two, when the testing has been done properly: RedHat, Microsoft, ... will push out the microcode updates. We need these distributors, or vendors, to do so because otherwise very few machines will end up patched. Large organisations might contact the CPU vendors, most SME or home users probably don't know what the machines contain and won't think to worry about the update... which would just leave them vulnerable for when someone works out how to produce a real Spectre cracker program. It is ''when'' not ''if''.

    • by Anonymous Coward

      The did not "stop distributing the microcode", they "rolled back to the last known stable release".

      What CPU microcode is available via the microcode_ctl package to mitigate CVE-2017-5715 (variant 2)? [redhat.com]

      The latest microcode_ctl and linux-firmware packages from Red Hat do not include resolutions to the CVE-2017-5715 (variant 2) exploit. Red Hat is no longer providing microcode to address Spectre, variant 2, due to instabilities introduced that are causing customer systems to not boot. The latest microcode_ctl a

    • by Anonymous Coward

      >Which might brick customers' hardware

      Microcode is non-persistent. It is loaded by either the motherboard or the OS at boot time. OS-loaded microcode can't brick your hardware; worst case, reinstall. A bad BIOS flash may be more problematic

      • by arth1 ( 260657 )

        Microcode is non-persistent. It is loaded by either the motherboard or the OS at boot time.

        Most commonly either through a initramfs that loads it directly, or a udev rule that loads it when the devices are set up early in boot.
        But also after boot, by calling micocode_ctl or iucode_tool specifying what microcode update should be loaded, or signaling the kernel by writing 1 to /sys/devices/system/cpu/microcode/reload if there is a /lib/firmware/intel-ucode/ file that matches the family/model/stepping, or by writing the extracted firmware directly into /dev/cpu/microcode

        I prefer to do it with microc

    • The microcode update "should not" brick hardware. The output of the command "rpm -qi microcode_ctl | tail -3" tells you why:

      The microcode update is volatile and needs to be uploaded on each system boot i.e. it doesn't reflash your cpu permanently, reboot and it reverts back to the old microcode.

  • by Marlin Schwanke ( 3574769 ) on Saturday January 20, 2018 @01:34PM (#55967931)

    Seems like there is more damage being done by the Meltdown / Spectre hysteria and attendant patch frenzy than by exploits of the vulnerabilities themselves.

    Friends don't let friends Spectre patch their computers!

    • by HiThere ( 15173 ) <charleshixsnNO@SPAMearthlink.net> on Saturday January 20, 2018 @03:29PM (#55968491)

      You've got a point. The question is, how would you know if you were penetrated by Meltdown? Spectre is something that needs to be dealt with carefully. Meltdown? That's a bit more dangerous.

      My personal suggestion is to start by blocking javascript execution. That's not enough, but it buys you some time. But beyond that, I don't know. I'm presuming that it will get sorted out, so buying yourself some time is valuable.

      • Your personal system won’t be penetrated by Meltdown. Your system will need to be compromised by malware for Meltdown / Spectre vulnerabilities to be exploited. There is literally no point to it if your system is already compromised. Now JavaScript exploits might be possible but all of the major browser vendors have already patched their JavaScript engines. I can see disabling JavaScript on certain secure systems but it would break alot of things users want on popular websites. Note that Amazon s
    • by tlhIngan ( 30335 )

      Seems like there is more damage being done by the Meltdown / Spectre hysteria and attendant patch frenzy than by exploits of the vulnerabilities themselves.

      That's because the most serious vulnerabilities affect virtualized systems more, and given the popularity of the cloud, they are extremely affected because there is no control over what software is run.

      These attacks may seem theoretical, and for a desktop user, they mostly are. Even server users with dedicated machines are generally safe as they own the

      • That's because the most serious vulnerabilities affect virtualized systems more, and given the popularity of the cloud, they are extremely affected because there is no control over what software is run.

        These attacks may seem theoretical, and for a desktop user, they mostly are. Even server users with dedicated machines are generally safe as they own the entire machine and thus control the software on it. Virtualized hosts, not so much, and this ranges from simple VPS hosting companies, to cloud service companies like Amazon and Google. In this case, accessing not-your-memory can be beneficial, like trying to find the encryption keys used by someone sharing your machine. Cloud providers are especially scared because during high peak retail times, encryption keys may be spun to many instances to handle the load, and crafty attackers may simply spin up instances to extract those keys.

        Very good points. I agree that it is the shared environments that have to take the Meltdown / Spectre threats seriously.

  • "each customer contact their OEM hardware provider and inquire about mitigations for CVE-2017-5715 on a per-system basis. "

    Translation: each customer of Intel better think twice next time they consider and trust said company.
  • by thegarbz ( 1787294 ) on Saturday January 20, 2018 @01:41PM (#55967961)

    So Linux provides the ability to update the CPU microcode during boot. Does Windows do something similar? I know that Microsoft has control over their Surface devices and often rolls out UEFI / BIOS updates via Windows updates, but does the OS itself have this functionality too?

    It would seem there's a pretty big exposure if Linux is able to push out updates directly from Intel, but Microsoft says: Apply a BIOS update from your PC OEM. Last BIOS issued to my motherboard was in 2014 and it is listed as beta.

    • by ELCouz ( 1338259 )
      You can do it yourself...use the linux microcode file from Intel website and patch at startup the microcode with the vmware windows microcode update tool.

      I did this this summer when there was a bug regarding threads for the Skylake CPUs until I had the BIOS update available.
  • by GerryGilmore ( 663905 ) on Saturday January 20, 2018 @02:05PM (#55968069)
    ...and I'll say it again: there has been WAY too much hype and hysteria (Hhhhmmm: Hype and Hysteria would be a GREAT band name!) around these issues. A) The vulnerabilities are far too esoteric to be really useful when there are a gazillion very easy, well-worn paths to get at your systems. B) Yes, ultimately, these vulnerabilities need to be addressed, but in a much more reasoned, thoroughly tested fashion as opposed to the current "OMG! I must flash my BIOS, install the latest microcode and update the kernel AT ONCE or all my base are belong to them!!" mindset. Perspective, please.
    • Your lazy and careless attitude is typical of a PC owner who thinks what is true for his little home devices are true for virtualized systems

      • And your stupid post indicates how little you know. VMWare has already issued patches to isolate any infection, protecting virtualized environments. Beyond which, I challenge you (or anyone) to show me a real working infiltration outside a very carefully crafted environment where you can A) gather enough memory data in this abtruse fashion to be useful and then B) make any sense out of this data. But go ahead, crash the delicately balanced ecosystem encompassing everything evolved over the last 40 years or
        • False, you are the ignorant one fearing bad press for vmware, maybe your stock will take a hit? The truth is VMware has reverted several SPECTRE and MELTDOWON patches for causing instability. All intel and AMD issues are NOT currently patched.

        • VMWare has already issued patches to isolate any infection

          The VMWare "patches" are little more than the same patches that everyone else is rolling out, and requires the same microcode update. That same update that VMware has pulled [vmware.com] just like the rest of the industry has.

          But go ahead, crash the delicately balanced ecosystem encompassing everything evolved over the last 40 years or so so that you can be "secure" (while phishers, etc wreak havoc through regular, working methods).

          There's different risk profiles and different attack vectors for different systems. They all should be addressed as needed. Oh and for making that statement: You're a complete moron.

      • Your lazy and careless attitude is typical of a PC owner who thinks what is true for his little home devices are true for virtualized systems

        Anyone who puts his data and his software on _someone else's computer_ didn't really care about its security in the first place. Otherwise he would have invested in a computer of his own, and kept everything in-house.

        • I have no problem sending properly encrypted data to the cloud, and I've had isolated incidents in past workplaces where employees and contractors stole information that was local. In general, cloud storage does pose security challenges, but not insurmountable ones. People who do not understand those challenges, and people who take advantage of them, are the bigger issues IMO.
          • Dude, talk about missing the point. If your data is already encrypted anyway it's no problem if it gets sniffed by Spectre and/or Meltdown, now is it? The discussion was about when your data is potentially available for sniffing, i.e. when it is not encrypted.

  • Intel (Score:5, Insightful)

    by zdzichu ( 100333 ) <{zdzichu} {at} {irc.pl}> on Saturday January 20, 2018 @02:24PM (#55968175) Homepage Journal

    Wow, few paragraphs about Intel's fuckup and the culprit name isn't mentioned even once. It's Intel. Red Hat merely retracted updates developed and provided by Intel.

    • by HiThere ( 15173 )

      Ah, so that's why they said to contact the OEM. The OEM will know which patches (from Intel) are safe to use on their system.

  • so I do not need to worry about Intel's f*ckups anymore: https://www.youtube.com/watch?... [youtube.com] ;-)
    • The R10000, 12 and 14K had speculative execution, you may be vulnerable to spectre-like attacks

  • by manu0601 ( 2221348 ) on Saturday January 20, 2018 @08:48PM (#55969829)
    I wondered about CPU micrcode hacking and I got the answer here: the micrcode update is signed. That suggests modern Intel CPU have a crypto engine to verify the update. I wonder how long will we live before someone leaks private key.

Harrison's Postulate: For every action, there is an equal and opposite criticism.

Working...