Reading and Schedule

Below are the tentative schedule and reading list of this course.

Date	Reading	Lead
1/7	Course Introduction and the History of Virtualization	Yiying Slides
1/9	Background and Virtualization Overview Comet Book Chapter on Virtual Machine Monitors Additional Readings Formal Requirements for Virtualizable Third Generation Architectures (Comm ACM 1974) Disco: Running Commodity Operating Systems on Scalable Multiprocessors (TOCS'97) Scale and Performance in the Denali Isolation Kernel	Yiying Slides
1/14	Virtualizing CPU A Comparison of Software and Hardware Techniques for x86 Virtualization (ASPLOS'06) Questions Why is x86 un-virtualizable with trap-and-emulate? Give one example. How are jump instructions translated? With hardware virtualization extensions (e.g., Intel VT), do we still need binary translation? Why or why not? Additional Readings The Evolution of an x86 Virtual Machine Monitor Software Techniques for Avoiding Hardware Virtualization Exits Embra: Fast and Flexible Machine Simulation Fast Dynamic Binary Translation for the Kernel Enabling Intel Virtualization Technology Features and Benefits	Yiying Slides
1/16	Virtualizing Memory Memory Resource Management in VMware ESX Server (OSDI'02) Questions What is the double paging problem and what caused it? Would a malicious guest OS (or a buggy one) be able to access memory that it has swapped out during ballooning? Why/why not? What is the benefit of keeping a "hint" entry for each scanned (but unshared) page (as compared to not maintaining anything for the page) Additional Readings Difference Engine: Harnessing Memory Redundancy in Virtual Machines Performance Evaluation of Intel EPT Hardware Assist	Yiying Slides
1/21	Virtualizing I/O Virtualizing I/O Devices on VMware Workstation's Hosted Virtual Machine Monitor (ATC'01) Questions What is the down side of virtualizing I/O with hosted architecture (VMware Workstation)? At least how many context switches (including VM/VMM switches and VMM/Host switches) do a guest OS network send and a receive involve? Name two optimization techniques that could reduce the number of world switches during network operations. Additional Readings virtio: Towards a De-Facto Standard For Virtual I/O Devices vIC: Interrupt Coalescing for Virtual Machine Storage Device IO ELI: Bare-Metal Performance for I/O Virtualization	Yiying Slides
1/23	Container Basics Understanding and Hardening Linux Containers (mainly Ch 2 to Ch 5; you can ignore many of the details in these chapters. Read Ch 1 for more background on virtualization. Read other chapters if you are interested in security.) Questions What types of isolations does Linux containers achieve? Can one Linux container affect the performance of another Linux container on the same machine (i.e., performance isolation)? Why or why not? Why do you think containers are less "secure" than virtual machines? Additional Readings LXC/LXD Docker Understanding Security Implications of Using Containers in the Cloud Container Security: Issues, Challenges, and the Road Ahead Slacker: Fast Distribution with Lazy Docker Containers	Yiying Slides
1/28	Kubernetes and gVisor Kubernetes and gVisor, Quiz 1 Questions What is a Kubernetes Pod? How do you think it is useful in container orchestration? What does Kubernetes use etcd for? Why is having a consistent, atomic key-value store important for Kubernetes' control plane? Vulnerabilities in the Linux kernel makes it unsafe for containers to call Linux system calls. How does gVisor solve this problem? Additional Readings Borg, Omega, and Kubernetes (Google) The True Cost of Containing: A gVisor Case Study Container Isolation at Scale (Introducing gVisor) - Dawn Chen & Zhengyu He, Google Nabla Containers	Yiying Slides
1/30	Serverless Computing Overview Cloud Programming Simplified: A Berkeley View on Serverless Computing Questions Current datacenters use container as the host to run serverless functions. Do you think that is a good way? Why and why not? Today's serverless functions are stateless. How do you think different functions can share data and communicate? Can you think of any security threats of serverless computing? Bonus points if you can outline a real threat/attack. Additional Readings Amazon Lambda Google Cloud Functions Azure Functions Serverless Computing: Current Trends and Open Problems Occupy the Cloud: Distributed Computing for the 99% (PyWren) Serverless Computing: One Step Forward, Two Steps Back	Yiying Slides Lihao Slides
2/4	Serverless Computing Advanced Pocket: Elastic Ephemeral Storage for Serverless Analytics (OSDI'18) , Debate on Serverless Questions Why isn't using existing in-memory key-value stores such as Redis and Memcached a good option for storing ephemeral data in serverless computing? How does Pocket balance storage load? Do you think Pocket solve all the problems of managing states in serverless computing? If not, what do you think are the remaining problems? Additional Readings SAND: Towards High-Performance Serverless Computing Encoding, Fast and Slow: Low-Latency Video Processing Using Thousands of Tiny Threads A Case for Serverless Machine Learning Archipelago: A Scalable Low-Latency Serverless Platform Cloudburst: Stateful Functions-as-a-Service	Yiying Slides
2/6	Library OS Unikernels: Library Operating Systems for the Cloud (ASPLOS'13) Questions Name one benefit and one drawback of compiling a single-image VM. Unikernel runs in a single address space. Give one example of how this design helps improve performance. Comparing gVisor and Unikernels, which one do you think is more secure and which is more lightweight? Additional Readings Unikernels as Processes Rethinking the Library OS from the Top-Down Mirage OS Nabla Containers ClickOS and the Art of Network Function Virtualization Libra: a library operating system for a JVM in a virtualized execution environment Exokernel: an operating system architecture for application-level resource management	Yiying Slides
2/11	Para-Virtualization Xen and the Art of Virtualization (SOSP'03) Questions Why can Xen allow guest OS system call handlers to be accessed directly (without any ring-0 Xen involvement) but not guest page fault handler? What's the benefit of using asynchronous event notifications from Xen to a VM? What goals of Xen are not valid or less valid in today's cloud environments? Additional Readings Understanding Full Virtualization, Paravirtualization, and Hardware Assist Safe Hardware Access with the Xen Virtual Machine Monitor Optimizing Network Virtualization in Xen Measuring CPU Overhead for I/O Processing in the Xen Virtual Machine Monitor Breaking Up is Hard to Do: Security and Functionality in a Commodity Hypervisor (SOSP'11)	Xinyuan, Tianrui, Xiaohan Slides
2/13	Light VM My VM is Lighter (and Safer) than your Container (SOSP'17) , Quiz 2 Questions What are the benefits of LightVM (Tinyx) over Unikernels? Why does device creation dominate VM creation when there are fewer concurrent VMs, while XenStore interactions dominate VM create when there are more concurrent VMs? What is the case that requires guest OS modification to run with LightVM? Additional Readings OSv - Optimizing the OS for VMs Kata Containers Jitsu: Just-In-Time Summoning of Unikernels	Yiying Slides
2/18	KVM and Dune KVM, Dune: Safe User-level Access to Privileged CPU Features (OSDI'12) * It is OK if you only understand the very high-level ideas of Dune Questions Why does Linux needs to notify KVM when it preempts a process that has guest states or when it flushes the PTEs of KVM guests? How is a Dune process different from a regular VM? Additional Readings KVM Documentation	Yiying Slides
2/20	VM Migration and Replication Live Migration of Virtual Machines (NSDI'05) Questions How does the system proposed in this paper determine when to end the pre-copy phase and perform a stop-and-copy? What do you think are the pros and cons of managed migration and self migration? Can you think of at least one new challenge and one new opportunity when designing a system for live container migration? Additional Readings The Design and Implementation of Zap: A System for Migrating Computing Environments (OSDI'02) XvMotion: Unified Virtual Machine Migration over Long Distance (ATC'14) VMware vMotion Remus: High Availability via Asynchronous Virtual Machine Replication (NSDI'08) RemusDB: Transparent High Availability for Database Systems	Yiying Slides
2/25	Security When Virtual is Harder than Real: Security Challenges in Virtual Machine Based Computing Environments (HotOS'05) Secure Container Isolation: Problem Statement & Solution Space Questions Can you think of some drawback of enforcing security mechanisms at the hypervisor level (compared to at the guest OS or above)? Choose one type in the "Attack Surfaces" section of the Google doc (the last section), and describe how sandboxing can help defend that type of attacks. Name one vulnerability of containers that cannot be addressed by sandboxing. Additional Readings When Virtual Is Better Than Real (HotOS'01) Secure Pods: Sandboxing workloads in Kubernetes TrustVisor: Efficient TCB Reduction and Attestation SecVisor: A Tiny Hypervisor to Provide Lifetime Kernel Code Integrity for Commodity OSes (SOSP'07) Breaking Up is Hard to Do: Security and Functionality in a Commodity Hypervisor (SOSP'11) InkTag: Secure Applications on an Untrusted Operating System (ASPLOS'13) Overshadow: A Virtualization-Based Approach to Retrofitting Protection in Commodity Operating Systems VirtuOS: An Operating System with Kernel Virtualization SCONE: Secure Linux Containers with Intel SGX Understanding Security Implications of Using Containers in the Cloud (ASPLOS'08) Container Security: Issues, Challenges, and the Road Ahead	Ian Slides Yiying Slides
2/27	Virtualizing non-CPU Processors (Accelerators) A Full GPU Virtualization Solution with Mediated Pass-Through (ATC'14) Sharing, Protection and Compatibility for Reconfigurable Fabric with AmorphOS (OSDI'18) * It is OK to only understand the very high-level ideas of AmorphOS. Questions Why does gVirt uses a fixed and large (16ms) time slice to schedule virtual GPUs? Name at least one place where virtualizing GPU memory is different from virtualizing CPU memory. What makes it hard to space multiplex FPGA? (bonus point for an additional question of "what makes it hard to time multiple (dynamically schedule) FPGA?" Additional Readings Accelerating & Optimizing HPC/ML on vSphere Leveraging NVIDIA GPU (2019/02 talk) GPUvm: Why Not Virtualizing GPUs at the Hypervisor? (ATC'14) PTask: Operating System Abstractions To Manage GPUs as Compute Devices (SOSP'11)	Yiying Slides Yizhou Shan Slides
3/3	Amazon Nitro Amazon Nitro (esp. the video talk on that page) Questions With Amazon Nitro, virtualization functions are mostly offloaded to hardware. Do we still need a hypervisor (or an OS)? Can everything just run in user space and interact with Nitro cards directly? Can you think of a drawback of offloading tasks to hardware (i.e., Nitro's approach)? Additional Readings	Vansh, Kunlin Slides
3/5	Amazon Firecracker Firecracker: Lightweight Virtualization for Serverless Applications (NSDI'20) Questions What is the benefit of Firecracker over gVisor in terms of the specific goals Amazon has for their cloud production environments? What mechanism(s) allow Firecracker to run thousands of MicroVMs on the same machine (with 10x-20x oversubscription rate)? Why do you think Firecracker (when deployed to power AWS Lambda) run one process (one slot) in one MicroVM? Additional Readings Amazon Firecracker Git repo Nabla Containers Kata Containers	Yiying Slides
3/10	Course Summary Hints for Computer System Design -- Butler Lampson Questions Read the "Hints for Computer System Design" paper and summarize what you have learned over the course. Feel free to write about anything else you want to comment on the course.	Yiying Slides
3/12	Project Presentations