Meltdown

Working around Intel Hardware Flaws

Zack Brown — Mon, 30 Apr 2018 12:07:07 +0000

Efforts to work around serious hardware flaws in Intel chips are ongoing. Nadav Amit posted a patch to improve compatibility mode with respect to Intel's Meltdown flaw. Compatibility mode is when the system emulates an older CPU in order to provide a runtime environment that supports an older piece of software that relies on the features of that CPU. The thing to be avoided is to emulate massive security holes created by hardware flaws in that older chip as well.

In this case, Linux is already protected from Meltdown by use of PTI (page table isolation), a patch that went into Linux 4.15 and that was subsequently backported all over the place. However, like the BKL (big kernel lock) in the old days, PTI is a heavy-weight solution, with a big impact on system speed. Any chance to disable it without reintroducing security holes is a chance worth exploring.

Nadav's patch was an attempt to do this. The goal was "to disable PTI selectively as long as x86-32 processes are running and to enable global pages throughout this time."

One problem that Nadav acknowledged was that since so many developers were actively working on anti-Meltdown and anti-Spectre patches, there was plenty of opportunity for one patch to step all over what another was trying to do. As a result, he said, "the patches are marked as an RFC since they (specifically the last one) do not coexist with Dave Hansen's enabling of global pages, and might have conflicts with Joerg's work on 32-bit."

Andrew Cooper remarked, chillingly:

Being 32bit is itself sufficient protection against Meltdown (as long as there is nothing interesting of the kernel's mapped below the 4G boundary). However, a 32bit compatibility process may try to attack with Spectre/SP2 to redirect speculation back into userspace, at which point (if successful) the pipeline will be speculating in 64bit mode, and Meltdown is back on the table. SMEP will block this attack vector, irrespective of other SP2 defenses the kernel may employ, but a fully SP2-defended kernel doesn't require SMEP to be safe in this case.

And Dave, nearby, remarked, "regardless of Meltdown/Spectre, SMEP is valuable. It's valuable to everything, compatibility-mode or not."

SMEP (Supervisor Mode Execution Protection) is a hardware mode, whereby the OS can set a register on compatible CPUs to prevent userspace code from running. Only code that already has root permissions can run when SMEP is activated.

Go to Full Article

diff -u: Intel Design Flaw Fallout

Zack Brown — Mon, 19 Mar 2018 14:41:06 +0000

by Zack Brown

For weeks, the world's been talking about severe Intel design flaws affecting many CPUs and forcing operating systems to look for sometimes costly workarounds.

Linux patches for these issues are in a state of ongoing development. Security is always the first priority, at the expense of any other feature. Next would probably be the general speed of a running system for the average user. After that, the developers might begin piecing together any features that had been pulled as part of the initial security fix.

But while this effort goes on, the kernel developers seem fairly angry at Intel, especially when they feel that Intel is not doing enough to fix the problems in future processors.

In response to one set of patches, for example, Linus Torvalds burst out with, "All of this is pure garbage. Is Intel really planning on making this shit architectural? Has anybody talked to them and told them they are f*cking insane?" He went on, "the IBRS garbage implies that Intel is _not_ planning on doing the right thing for the indirect branch speculation. Honestly, that's completely unacceptable." And then he said:

The whole IBRS_ALL feature to me very clearly says "Intel is not serious about this, we'll have an ugly hack that will be so expensive that we don't want to enable it by default, because that would look bad in benchmarks". So instead they try to push the garbage down to us. And they are doing it entirely wrong.

He went on, even more disturbingly, to say:

The patches do things like add the garbage MSR writes to the kernel entry/exit points. That's insane. That says "we're trying to protect the kernel". We already have retpoline there, with less overhead.

So somebody isn't telling the truth here. Somebody is pushing complete garbage for unclear reasons. Sorry for having to point that out....As it is, the patches are COMPLETE AND UTTER GARBAGE....WHAT THE F*CK IS GOING ON?

At one point, David Woodhouse offered a helpful technical summary of the whole situation for those of us on the edge of our seats:

This is all about Spectre variant 2, where the CPU can be tricked into mispredicting the target of an indirect branch. And I'm specifically looking at what we can do on *current* hardware, where we're limited to the hacks they can manage to add in the microcode.

The new microcode from Intel and AMD adds three new features.

One new feature (IBPB) is a complete barrier for branch prediction. After frobbing this, no branch targets learned earlier are going to be used. It's kind of expensive (order of magnitude ~4000 cycles).

Go to Full Article

Oracle Patches Spectre for Red Hat

Charles Fisher — Thu, 15 Mar 2018 14:52:25 +0000

by Charles Fisher

Red Hat's Spectre remediation currently requires new microcode for a complete fix, which leaves most x86 processors vulnerable as they lack this update. Oracle has released new retpoline kernels that completely remediate Meltdown and Spectre on all compatible CPUs, which I show how to install and test on CentOS here.

The Red Hat community has patiently awaited a retpoline kernel implementation that remediates CVE-2017-5715 (Spectre v2) and closes all Meltdown and Spectre vulnerabilities that have captured headlines this year.

Red Hat's initial fixes rely upon microcode updates for v2 remediation, a decision that leaves the vast majority of AMD64-capable processors in an exploitable state. Intel's new microcode has proven especially problematic; it performs badly and the January 2018 versions were plagued with stability issues that crashed many systems. It is a poor solution to a pressing problem.

Google's retpolines—an ingenious approach to v2—essentially play out the exploit within the Linux kernel in any and all vulnerable code. This method allows the kernel to retain complete control over speculative execution hardware in all architectures upon which it runs. It is faster and more generic than Intel's microcode and seems in every way a superior solution.

Oracle appears to be the first major member of the Red Hat community that has delivered a Linux kernel with retpoline support. The latest version of the Unbreakable Enterprise Kernel preview release 5 (UEKr5) now offers complete remediation for Meltdown and Spectre regardless of CPU microcode status. The UEKr5 is based on the mainline Linux kernel's long-term support branch v4.14 release. In late-breaking news, Oracle has issued a production release of the UEKr4 that also includes retpolines, details below. For corporate users and others with mandated patch schedules over a large number of servers, the UEK now seems to be the only solution for complete Spectre coverage on all CPUs. The UEK brings a number of other advantages over the "Red Hat-Compatible Kernel" (RHCK), but this patch response is likely to drive Oracle deeply into the Red Hat community should they remain the single source.

CentOS Installation

CentOS, Scientific and Oracle Linux are all based off the upstream provider Red Hat. CentOS is a popular variant and is likely the best demonstration for loading the UEK on a peer distro.

It appears that Red Hat itself has been laying groundwork for a retpoline kernel. A January 2018 gcc compiler update implies a successful backport:

Go to Full Article

Meltdown/Spectre Status for Red Hat and Oracle

Charles Fisher — Mon, 05 Feb 2018 20:34:57 +0000

by Charles Fisher

The Red Hat family of operating systems addressed Meltdown and Spectre in its v3.10 kernel quickly, but relied too much upon Intel's flawed microcode and was forced to revert from a complete solution. Oracle implemented alternate approaches more suited to its v4.1 UEK, but both kernels continue to lack full Spectre coverage while they wait for Intel. Conspicuously absent from either Linux branch is Google's retpoline, which offers far greater and more efficient coverage for all CPUs. Auditing this status is a challenge. This article presents the latest tools for vulnerability assessments.

A frenzy of patch activity has surrounded this year's Meltdown and Spectre CPU vulnerability disclosures. Normally quiet microcode packages for Intel chips have seen four updates in the month of January, one of which was finally to roll back flawed code that triggers random reboots. For enterprise-grade hardware, Intel's quality control has left much to be desired.

It is likely premature to deploy new monitoring and compliance tools, and a final solution for this set of vulnerabilities will wait until correct microcode is obtained. Still, it may be important for many organizations to evaluate the patch status of servers running Linux kernels packaged by Oracle and/or Red Hat.

Meltdown patches exist now and should be deployed immediately on vulnerable servers. Remediating all Spectre vulnerabilities requires not only the latest kernels, but also a patched GCC to compile the kernel that is capable of implementing "retpolines", or compatible microcode from your CPU vendor.

RHCK

Red Hat was one of the first Linux distributions to publish guidance on Meltdown and Spectre. It established three files as "kernel tunables" in the /sys/kernel/debug/x86 directory to monitor and control these patches: pti_enabled for Meltdown, ibpb_enabled for Spectre v1 and ibrs_enabled for Spectre v2. Only the root user can access these files.

When these files contain a numerical zero, the patches are not active. If allowed for the CPU, a numerical one may be written to the file to enable the relevant remediation, and a zero may be written later to disable it. This is not always allowed—AMD processors are not vulnerable to Meltdown, and the value in the pti_enabled file is locked to zero and cannot be changed. If the fixes are active and show 1, the performance of the CPU may be reduced. Compatible microcode is required to enable all patches on vulnerable CPUs, which adds new assembler/machine language op codes that erase vulnerable kernel data from CPU pipelines and caches to close the exploit.

Go to Full Article