Reptar

Prefix Decoding

An interesting feature of x86 is that the instruction decoding is generally quite relaxed. If you use a prefix that doesn’t make sense or conflicts with other prefixes nothing much will happen, it will usually just be ignored.

This fact is sometimes useful; compilers can use redundant prefixes to pad a single instruction to a desirable alignment boundary.

Take a look at this snippet, this is exactly the same code as above, just a bunch of useless or redundant prefixes have been added:

            rep lea rdi, [rel dst]
             cs lea rsi, [rel src]
       gs gs gs std
          repnz mov rcx, 32
rep rep rep rep movsb

Perhaps the most interesting prefixes are rex, vex and evex, all of which change how subsequent instructions are decoded.

Let’s take a look at how they work.

The REX prefix

The i386 only had 8 general purpose registers, so you could specify which register you want to use in just 3 bits (because 2^3 is 8).

The way that instructions were encoded took advantage of this fact, and reserved just enough bits to specify any of those registers.

Simple 2-byte instructions that use modr/m might be encoded like this, for example mov eax, ebx.

This is an 8-bit opcode, 2 bit addressing mode (labeled m), and 3 bits each for the source (s) and destination (d).

Well, this is a problem, because x86-64 added 8 additional general purpose registers. We now have sixteen possible registers..that’s 2^4, so we’re going to need another bit! 😆

The solution to this is the rex prefix, which gives us some spare bits that the next instruction can borrow.

When we’re talking about rex, we usually write it like this:

rex.rxb

rex is a single-byte prefix, the first four bits are mandatory and the remaining four bits called b, x, r and w are all optional. If you see rex.rb that means only the r and b bits are set, all the others are unset.

These optional bits give us room to encode more general purpose registers in the following instruction.

The rex prefix can lend the next instruction extra bits to use for operands, so now we can encode all 16 possible general purpose registers!

Now we’re fine until someone adds another register! 😂

Encoding Rules

So now we know that rex increases the available space for encoding operands, and that useless or redundant prefixes are usually ignored on x86. So… what should this instruction do?

rex.rxb rep movsb

The movsb instruction doesn’t have any operands - they’re all implicit - so any rex bits are meaningless, right?

If you guessed that the processor will just silently ignore the rex prefix, you would be correct!

Well… except on machines that support a new feature called fast short repeat move! We discovered that a bug with redundant rex prefixes could interact with this feature in an unexpected way and introduce a serious vulnerability, oops 🙂

Fast Short Repeat Move

FSRM is a new feature introduced in Ice Lake that fixes some of the shortcomings of ERMS. Hopefully that clears up any confusion. 😆

Just kidding, let’s quickly look at ERMS.

The hard part of moving strings around efficiently is getting all the buffers aligned so you can use the widest possible stores available. You could do this in software, but if we do it in microcode then the processor can just transparently make your existing code faster for you.

This requires some expensive setup, but once that’s done you get vastly improved throughput. This feature is known as enhanced repeat move/store, ERMS.

If you have a processor with ERMS support, simple rep movsb operations can sometimes perform comparably with more complicated hand-tuned vector move operations.

However, there is a problem with ERMS. That initial setup is so expensive that it just isn’t worth it for very short strings. This is what FSRM is designed to solve, it handles the case of only moving 128 bytes or less and makes that faster too!

I’m not aware of any documentation that explains exactly how FSRM works, but you can check if you have a processor that supports it by looking at the flags line in /proc/cpuinfo:

flags       : fpu vme de pse tsc msr pae mce cx8 [...] fsrm

Some of the processors that have this feature include:

• Ice Lake
• Rocket Lake
• Tiger Lake
• Raptor Lake
• Alder Lake
• Sapphire Rapids

Note: This list may not be comprehensive, please see Intel advisory INTEL-SA-00950 for a complete list.

Reproduce

We’re publishing all of our research today to our security research repository. If you want to reproduce the vulnerability you can use our icebreak tool, I’ve also made a local mirror available here.

$ ./icebreak -h
usage: ./icebreak [OPTIONS]
    -c N,M      Run repro threads on core N and M.
    -d N        Sleep N usecs between repro attempts.
    -H N        Spawn a hammer thread on core N.
icebreak: you must at least specify a core pair with -c! (see -h for help)

The testcase enters what should be an infinite loop, and unaffected systems should see no output at all. On affected systems, a . is printed on each successful reproduction.

$ ./icebreak -c 0,4
starting repro on cores 0 and 4
.........................................................................
.........................................................................
.........................................................................
.........................................................................
.........................................................................

In general, if the cores are SMT siblings then you may observe random branches and if they’re SMP siblings from the same package then you may observe machine checks.

If you do not specify two different cores, then you might need to use a hammer thread to trigger a reproduction.

Analysis

We know something strange is happening, but how microcode works in modern systems is a closely guarded secret. We can only theorize about the root cause based on observations.

Questions

I’m sure some readers will have questions about what is possible in this unexpected “glitch” state. Well, so do we!

We know that we can corrupt the system state badly enough to cause machine check errors, and we’ve also observed threads interfere with execution of processes scheduled on SMT siblings.

However, we simply don’t know if we can control the corruption precisely enough to achieve privilege escalation. I suspect that it is possible, but we don’t have any way to debug μop execution!

If you’re interested in studying this, then we would love to get your input!

Credit

This bug was independently discovered by multiple research teams within Google, including the silifuzz team and Google Information Security Engineering. The bug was analyzed by Tavis Ormandy, Josh Eads, Eduardo Vela Nava, Alexandra Sandulescu and Daniel Moghimi.

Workaround

If you can’t update for some reason, you could disable fast strings via the IA32_MISC_ENABLE model specific register.

This will cause a significant performance penalty, and should not be used unless absolutely necessary.