Skip to content

Project 5: ISA Support for Security

Due: Friday, March 13th, 11:59 PM

Note: You may use no more than two late days on this project.

Goal

Our goal is to explore how additional hardware support can simplify and improve the efficiency of software security mechanisms. Specifically, we will build a new SFI runtime in which modules use hardware instructions for address masking and to guarantee atomic execution of basic blocks.

Background

This project builds on Project 2 and Project 4, plus our discussion of the RISC-V CFI reading and HFI reading.

The Environment

For the first part of this project, we will not be using our RISC-V environment. Instead, we will be using your host computer development environment to build a modified version of Qemu; in later parts of the project, we will boot a RISC-V VM using the modified Qemu.

Depending on what operating system your computer runs, your build instructions will vary. The Qemu wiki has instructions for installing the build prerequisites and for building Qemu on Linux, Mac, and Windows – but, for Windows, it may be easier to build Qemu inside WSL than in Windows proper.

Download the Qemu 8.2.10 source tarball. You can verify that its SHA-256 hash is

2610efc216d52903f73ee7601cdf50a0006b9cea200785174c5f3037c25d3577  qemu-8.2.10.tar.bz2

Unpack the tarball. Create a build subdirectory in the qemu-8.2.10 directory, and cd into that subdirectory and run

$ ../configure --target-list=riscv64-softmmu --enable-debug   \
       --enable-slirp --disable-gtk --disable-sdl --disable-oss \
       --disable-pa --disable-xen

If configuring the build succeeds, you can run make, still inside the build directory. After a while, this should create a qemu-system-riscv64 binary in the build directory.

If you have trouble downloading, configuring, or building Qemu, let us know on Piazza.

Part A

Our goal in Part A is to implement hardware support for forward-edge control flow tracking and hardware instructions for masking registers to SFI code and data regions.

We have already done a lot of the work to enable this, including an lpad landing pad instruction (using the same encoding as in Project 2). Your job is to implement a jalr.lp instruction (for “jalr to landing pad”) and sfi.cmask and sfi.dmask masking instructions.

While Qemu is a large project, you should not need to look at any file outside of target/riscv, and you will modify only two of the files in this directory: insn32.decode and insn_trans/trans_rvi.c.inc.

Briefly, Qemu is a just-in-time compiler that translates guest CPU code basic blocks on the fly to host CPU native code snippets, and caches the translated native code snippets for reuse. Each guest CPU implementation (“target”) provides code to translate guest CPU instructions to a Qemu-specific intermediate representation called TCG; the generated TCG is translated again to native code by a host CPU specific backend.

Restated using this jargon, your task in Part A is to define three new instructions and to write the functions that translate those instructions to a sequence of TCG operations.

For more information, you may want to refer to the Qemu TCG developer docs and to the Airbus Security Lab’s Qemu internals blog series.

Adding project starter code to Qemu

Download our Project 5 Qemu starter patch, proj5-qemu.patch and apply it to the Qemu source by running, from the main qemu-8.2.10 directory (not the build subdirectory),

$ patch -p1 --verbose < path/to/proj5-qemu.patch

There should be no warnings in the patch command output; all changes should apply cleanly. Back in the build subdirectory, rerun make. After a much faster build than last time, you should have again have a working qemu-system-riscv64 binary.

Implementing a jalr-to-landing-pad instruction

Our starter patch adds the machinery needed for Qemu to support forward-edge control-flow tracking, following the draft patchset by Deepak Gupta implementing (an earlier revision of) the RISC-V CFI draft specification.

Of particular interest to you, we have modified struct CPUArchState (defined in cpu.h; other code refers to it by the typedef CPURISCVState), which tracks the guest CPU’s ISA-visible state, to add a new field, cpu_expect_lp. This field should take on the value CPU_LP_EXPECTED when the next instruction must be a landing pad, and should have the value CPU_LP_NOT_EXPECTED otherwise. (These two macros are defined in cpu_bits.h.)

The file insn32.decode defines the encoding of (uncompressed) RISC-V instructions, using a Qemu-specific format called decodetree. We have modified the definition of the slti instruction to carve out the encoding 0x00002013 for our landing pad instruction, lpad:

-slti     ............     ..... 010 ..... 0010011 @i
+# *** Proj 5 ***
+{
+  lpad     000000000000     00000 010 00000 0010011
+  slti     ............     ..... 010 ..... 0010011 @i
+}

(The curly braces allow lpad to take precedence over slti zero, 0 for 0x00002013; without them, the Qemu build would complain about instruction conflicts.) Note that the lpad decodetree definition has no . characters; this means that the lpad instruction does not take arguments. (We won’t bother with arity matching.) By contrast, slti does take arguments; the . characters say which bits are interpreted as arguments, and the @i says to interpret slti as an I-format instruction.

Over in insn_trans/trans_rvi.c.inc, we supplied a matching trans_lpad instruction called when Qemu encounters an lpad instruction. This function changes disassembler context state (to keep the assembler from generating TCG for this block that raises an illegal instruction fault; see translate.c if you’re interested); it also emits a TCG that, when executed after being translated to native code, will write CPU_LP_NOT_EXPECTED to the cpu_expect_lp field of CPURISCVState.

Now it’s your turn. Define a new I-format instruction in insn32.decode called jalr_lp with opcode 0b1100111 (same as jalr) and funct3 field 0b001. In insn_trans/trans_rvi.c.inc, define a trans_jalr_lp function that emits TCG that sets cpu_expect_lp to CPU_LP_EXPECTED and also emits the TCG to perform a jalr. (The gen_jalr function in may be useful.)

Note that Qemu will expect trans_jalr_lp to take a pointer to an arg_jalr_lp struct that has rs1, rd, and imm fields.

Testing your new instruction

You could use the qemu-system-riscv64 you just built to run the course VM, but we don’t recommend doing that, because bugs in our modified Qemu implementation could cause VM crashes that lead to filesystem corruption. We have posted a modified version of the course VM starter zip file as vm-proj5.zip. Download and unzip this file inside your Qemu build directory. Use the vm/runvm-proj5.sh file to run the VM. Compared to the usual runvm.sh script, this script makes two changes: we run the qemu-system-riscv binary from the parent directory rather than the system search path, and we listen to SSH on port 8292 rather than 8291, so you can have both VMs running at once.

You can test your jalr.lp instruction by writing simple assembly programs and making sure that jalr.lp behaves like jalr when the first instruction at the jump target is an lpad, but causes an illegal instruction exception when it isn’t.

Because we have not modified the assembler to recognize our new instructions by name, you’ll need to use the .insn directive to request them in an assembly program (or in inline assembly in a C program). You already know how to generate an lpad. To generate a jalr.lp instruction and give it arguments, you can use the more sophisticated .insn i opcode, funct3, rd, simm12(rs1) directive; see the GNU assembler’s manual sections on RISC-V directives and RISC-V instruction formats.

Implementing code and data masking instructions

We have defined four new 64-bit fields in struct CPUArchState: sfi_code_andmask, sfi_code_ormask, sfi_data_andmask, and sfi_data_ormask. These CPU state fields take the place of the dedicated registers we used to hold these values in Project 4.

Your task will be to define two new instructions, sfi.cmask rd, imm(rs1) and sfi.dmask rd, imm(rs1) that mask the value rs1+imm with the appropriate and mask and or mask and store the result in rd. SFI modules will use sfi.dmask before any load or store instructions, and sfi.cmask before any indirect jumps.

A userspace program can assign a value from a general-purpose register to sfi_code_andmask, sfi_code_ormask, sfi_data_andmask, or sfi_data_ormask using the csrw csr, rs1 (pseudo)instruction, giving CSR index 0x861, 0x862, 0x863, and 0x864, respectively. (These constants are defined in cpu_bits.h, and the machinery for updating the CPU state fields is in csr.c.)

Now it’s your turn. Define two new I-format instructions in insn32.decode called sfi_cmask and sfi_dmask. Both should have opcode 0b0001011; sfi_cmask should have funct3 0b000 whereas sfi_dmask should have funct3 0b001.

Next, in insn_trans/trans_rvi.c.inc, define the corresponding trans_sfi_cmask and trans_sfi_dmask functions.

The trans_sfi_cmask function should emit TCG that computes rs1+imm, performs the bitwise and of the result with the current value of sfi_code_andmask (loaded from CPURISCVState), performs the bitwise or of the result with the current value of sfi_code_ormask (loaded from CPURISCVState), and, finally, stores the resulting value in rd.

The trans_sfi_dmask function should emit analogous TCG, but sfi_data_andmask and sfi_data_ormask.

You’ll need to write more TCG than for jalr.lp. You may want to refer to the implementation of get_address in translate.c for inspiration, and to consult the TCG operation reference.

Part B

In Part B, you will modify the CParser/libFirm compiler to produce SFI modules that use the new instructions you implemented in Part A. For the SFI system we build in Part C, we will do away with the assembly file rewriting by Perl script; the compiler changes you make in Part B are all we need.

The environment

For Part B (and Part C), you can either keep using the throwaway VM you set up for testing Part A, or you can go back to your course VM, but modify its run script to use the qemu-system-riscv64 binary that includes your Part A implementation instead of the Qemu install you used for previous projects.

In the project 5 VM, download and unpack proj5.tar.gz; the proj5/cparser subdirectory here is another copy of CParser from Project 2. Note: you do not want any of the modifications you made for Project 2; that’s why we’re unpacking a separate copy. You may, of course, consult the changes you made in Project 2 to remind yourself about libFirm internals. Our changes this time will be less ambitious; you will modify files only in the libfirm/ir/be/riscv directory.

Follow the instructions from Project 2 to build a cparser compiler binary in the cparser/build/debug directory.

Marking every basic block with a landing pad

Whereas in Project 2 we used landing pads for forward-edge control-flow integrity, in this project we will use them to guarantee a weaker property: all computed jumps are to the beginning of a basic block, meaning that an SFI module always executes a basic block from beginning to end. This in turn means that we get atomic execution of a basic block, and we also rule out jumping into the middle of an instruction. We can allow compressed instructions and don’t have to bother with packing instructions into 32-byte bundles.

Change the riscv_gen_block function in riscv_emitter.c to emit a landing pad as the first instruction of a basic block. A one-line change should do it!

Remember that, because our assembler doesn’t know about our new instructions, we need to use the .insn directive.

Replacing computed jumps with a mask and a checked jalr

Next, replace every jalr rs1 with the two-instruction sequence

    sfi.cmask rs1, 0(rs1)
    jalr.lp ra, 0(rs1)

and every ret with the two-instruction sequence

    sfi.cmask ra, 0(ra)
    jalr.lp zero, 0(ra)

(Recall that ret is shorthand for jalr zero, 0(ra), and jalr rs1 is shorthand for jalr ra, 0(rs1).)

You will definitely need to modify riscv_spec.pl; you can either keep your changes confined entirely there or create new emit_riscv_jalr and emit_riscv_ret functions in riscv_emitter.c.

Oh, in addition to the changes above, you’ll also want to patch the emit properties of jal and jalr nodes to emit a landing pad immediately after the jal or jalr.lp instruction. For some reason, libFirm doesn’t think a call ends a basic block, which means that a function return will cause an exception when it tries to jump (using jalr.lp) to the instruction after the call.

Adding masks to loads and stores

Finally, change the definition of the loadOp and storeOp templates to also emit, before the actual load or store instruction, the instruction

    sfi.dmask rs1, 0(rs1)

(Note that we want a dmask for loads and stores, whereas we used a cmask for jumps.)

Oh, in addition to the changes to loadOp and storeOp, you’ll also want to add an sfi.dmask of the stack pointer to emit_be_MemPerm in riscv_emitter.c; in a bit of questionable design, it directly emits stack loads and stores rather than lowering to separate nodes for each operation. The fib function in modules/demo.c is compiled to code complicated enough to create a MemPerm node; if your validator rejects it for having unmasked loads and stores, emit_be_MemPerm is why.

Part C

In Part C, you will create an SFI runtime, based on the one we used in Project 4, to validate modules that follow the rules described in Part B above.

The environment

The proj5/sfi directory includes an SFI runtime similar to the one you studied in Project 4. The Makefile assumes that cparser and sfi are sibling directories; if you’ve arrange your filesystem differently, edit the MODULE_CC variable to reflect the correct path to your Part B cparser binary.

The Makefile is adapted from Project 4; run make to build the demo-runner and target binaries, and run make modules/foo.sfi to turn compile an SFI module from source file modules/foo.c.

Unlike the Makefile from Project 4, this one uses the cparser compiler binary to turn modules/foo.c into modules/foo.s. The demo module should exercise all the features of Part B, so you can use it to check that your cparser changes are right.

Oh, we’ve augmented our disassembler, in riscv-disas.[ch], to recognize and disassemble our new instructions! (You’re welcome.) If you want to see what it thinks of a module, change the #if 0 in the create_instmap function in sfi.c.

Hardware-assisted SFI

We’ve changed the SFI runtime to use the new instructions you implemented in Part A. We use lpad instructions at the beginning of each atomic block to ensure that instructions within a block are always executed together. That allows us to add arbitrary checks before a potentially unsafe instruction, without needing fixed-length bundles and without using dedicated registers like in Project 4.

There are no longer any reserved registers, and no special registers have to be used for indirect control flow or memory access. We can even allow compressed instructions.

The new trampoline reads the address of sfi_return from a special metadata page allocated next to the module text segment. You can see the implementation in modules/trampoline.h and the check in the check_trampoline function of sfi.c. The sfi_call and sfi_return context-switch routines have changed, notably to use csrw instructions to tell the processor (implemented by Qemu) what code and data masks to use.

The security checks applied to modules have changed the most. In addition to check_trampoline, already mentioned, we use opcode_allowed and no_floating_point to forbid system calls, indirect branches other than jalr.lp, and (still grumpy over Part A of Project 4) absolutely any floating-point instructions.

To get SFI security, the most important properties we need to ensure are that all branches, direct or indirect, are to the start of a basic block (and therefore to a landing pad) in the module code region; and that all loads and stores are to the data region. That’s where you come in!

Checking direct branch targets

The check_entrypoints_to_lpads function, provided for you, makes sure that each function entrypoint defined in the .sfi file points to an lpad instruction. There is nothing you need to change, but you may want to note how it uses the addr_holds_lpad utility function—it’ll come in handy.

The direct_branch_to_lpad function is called on each instruction in the module (through the check_each_insn higher-order function, which behaves mostly the same way it did in Project 4). It should return 0 if an instruction is okay, -1 if it isn’t. We have already written a switch statement for you with (we think) every direct branch instruction, but the code to handle a direct branch is missing.

Fill in the missing code, in place of the line that reads

      err_quit("Proj 5: COMPLETE!");

As noted above, you’ll want to make sure that the branch target is in range and that it points to a landing pad. Because we are checking only direct branches, we can figure out the target address from the address of the branch instruction (addr) and the various fields of the disassembled branch instruction (available for you in the dec structure).

Checking indirect branches use code-masked addresses

The cmask_before_indirect_branch function is also called on each instruction, and needs to return 0 if the instruction is okay, or -1 if it is not okay. There is only one kind of indirect branch instruction allowed, our new jalr.lp, so the switch statement is very short this time.

Your job, as before, is to replace the line that reads

      err_quit("Proj 5: COMPLETE!");

with the missing logic.

As you know from having implemented it in Part A, the jalr.lp function will ensure that the target of the jump is an lpad instruction, so that part is taken care of. (By contrast, with regular jump instructions, we’d have needed to make sure the module preceded them with a code snippet like the one you wrote for Project 2. This is part of what hardware support simplifies for us.)

We still need to make sure that the target of the indirect jump is within the SFI module code region. (It’s possible that there are lpad instructions elsewhere in the program, and we don’t want to let the module jump to those.) Luckily, there’s a hardware instruction that can mask a register to ensure it’s in the code region: sfi.cmask.

Here’s what we need to check. We need to figure out what register holds the jump target address. (It’s always the register in the rs1 instruction encoding field.) Then we need to find the last time that register was modified within this basic block, by scanning the instructions between the start of the block (the lpad) and the current instruction. And we need to make sure that this instruction was an sfi.cmask instruction, writing to the register we care about.

Fill in the missing code.

Oh, you will definitely want to make use of the find_register_reaching_defn helper function, which implements a simple version of what compiler theorists call a reaching definition analysis.

Checking loads and stores use data-masked addresses

The dmask_before_load_store function is also called on each instruction, and needs to return 0 if the instruction is okay, or -1 if it is not okay. There are lots and lots of load and store instructions, so the switch statement is long. Luckily, the register that holds the (base) address is encoded in the rs1 field for every load and store instruction, so we can handle all of them the same way.

We still need to make sure that the memory address is within the SFI module data region. Luckily, there’s a hardware instruction that can mask a register to ensure it’s in the data region: sfi.dmask.

More specifically, we need to find the last time that the address register was modified within this basic block, and we need to make sure that the instruction that modified it was an sfi.dmask instruction, writing to the register we care about.

Fill in the missing code, again replacing the line

      err_quit("Proj 5: COMPLETE!");

It will be quite similar to the code you wrote for indirect branches above.

(What about the immediate offset? RISC-V has very small immediate spans; if the base register is within the data region, then, even accounting for the immediate, the access will either also be to the data region or it will be to the guard pages that surround it.)

Bonus: Audit our code for vulnerabilities

With the missing checks filled in, we think the resulting SFI system should be secure. But we haven’t proved it secure using a system like VeriWasm, so it’s possible there are problems we’ve missed.

If you find one, let us know, for some extra credit! Check with us if you’re not sure what attacks are in scope.

Logistics

You will submit using Gradescope. You should submit a zip file of your solution, without directory structure.

More details are TBD.

Grading

TBD.

Scoring is based on functionality in our testing, though solutions that violate the spirit of the assignment may be docked points.