Reading and Schedule

Date Reading Lead
1/8 Course Introduction and the History of Virtualization
 Yiying
Slides
1/10 Virtualization Overview and Virtualizing CPU
Comet Book Chapter on Virtual Machine Monitors and A Comparison of Software and Hardware Techniques for x86 Virtualization (ASPLOS'06)
Questions

  1. Why is x86 un-virtualizable with trap-and-emulate? Give one example.
  2. How are jump instructions translated?
  3. With hardware virtualization extensions (e.g., Intel VT), do we still need binary translation? Why or why not?

Additional Readings

  1. Formal Requirements for Virtualizable Third Generation Architectures (Comm ACM 1974)
  2. Disco: Running Commodity Operating Systems on Scalable Multiprocessors (TOCS'97)
  3. Scale and Performance in the Denali Isolation Kernel
  4. Software Techniques for Avoiding Hardware Virtualization Exits
  5. Embra: Fast and Flexible Machine Simulation
  6. Fast Dynamic Binary Translation for the Kernel
  7. Enabling Intel Virtualization Technology Features and Benefits

 Yiying
Slides
1/17 Virtualization CPU ctd. and Virtualizing Memory
Performance Evaluation of Intel EPT Hardware Assist and
Memory Resource Management in VMware ESX Server (OSDI'02)
Questions

  1. List at least one pro and one con for software MMU
  2. List at least one pro and one con for hardware MMU
  3. What is the double paging problem and what caused it?
  4. What is the benefit of keeping a "hint" entry for each scanned (but unshared) page (as compared to not maintaining anything for the page)

Additional Readings

  1. Difference Engine: Harnessing Memory Redundancy in Virtual Machines

 Yiying
Slides
1/22 Virtualizing I/O
First three sections of virtio: Towards a De-Facto Standard For Virtual I/O Devices and
first three sections of High Performance Network Virtualization with SR-IOV and
Network Virtualization Overview
Questions

  1. Is virtio a full virtualization or a paravirtualization technique? What's its main benefit?
  2. List at least one limitation of SR-IOV
  3. What are the similarities and differences between network virtualization and traditional server virtualization?

Additional Readings

  1. vIC: Interrupt Coalescing for Virtual Machine Storage Device IO
  2. ELI: Bare-Metal Performance for I/O Virtualization
  3. Virtualizing I/O Devices on VMware Workstation's Hosted Virtual Machine Monitor (ATC'01)
  4. Network Virtualization in Multi-tenant Datacenters (NSDI'14)
  5. The Design and Implementation of Open vSwitch (NSDI'15)

 Yiying
Slides
1/24 Cloud Computing
Above the Clouds: A Berkeley View of Cloud Computing
Quiz 1
Questions

  1. Why do you think cloud computing has been a huge success and gained majority of IT market?
  2. What challenges mentioned in the Bekerley cloud paper do you think still remain today?
  3. If you could change one thing about the cloud with a magic wand, what would you change?

Additional Readings

Yiying
Slides
1/29 Container Basics
Understanding and Hardening Linux Containers (mainly Ch 2 to Ch 5; you can ignore many of the details in these chapters. Read Ch 1 for more background on virtualization. Read other chapters if you are interested in security.)
Questions

  1. What types of isolations does Linux containers achieve?
  2. Can one Linux container affect the performance of another Linux container on the same machine (i.e., performance isolation)? Why or why not?
  3. Why do you think containers are less "secure" than virtual machines?

Additional Readings

  1. LXC/LXD
  2. Docker
  3. Understanding Security Implications of Using Containers in the Cloud
  4. Container Security: Issues, Challenges, and the Road Ahead
  5. Slacker: Fast Distribution with Lazy Docker Containers

 Yiying
Slides
1/31 Kubernetes and gVisor
Kubernetes and gVisor
Questions

  1. What is a Kubernetes Pod? How do you think it is useful in container orchestration?
  2. What does Kubernetes use etcd for? Why is having a consistent, atomic key-value store important for Kubernetes' control plane?
  3. Vulnerabilities in the Linux kernel makes it unsafe for containers to call Linux system calls. How does gVisor solve this problem?

Additional Readings

  1. Borg, Omega, and Kubernetes (Google)
  2. The True Cost of Containing: A gVisor Case Study
  3. Container Isolation at Scale (Introducing gVisor) - Dawn Chen & Zhengyu He, Google
  4. Nabla Containers

 Yiying
Slides
2/5 Serverless Computing Basics
Pages 3 to 8 of Cloud Programming Simplified: A Berkeley View on Serverless Computing and briefly about Serverless in the Wild: Characterizing and Optimizing the Serverless Workload at a Large Cloud Provider (ATC'20)
Questions

  1. Today's serverless functions are stateless. How do you think different functions can share data and communicate?
  2. Can you think of any security threats of serverless computing? Bonus points if you can outline a real threat/attack.
  3. Can you think of any other ways to reduce or avoid cold start for serverless computing (other than what the ATC'20 paper talks about).

Additional Readings

  1. Amazon Lambda
  2. Google Cloud Functions
  3. Azure Functions
  4. Serverless Computing: Current Trends and Open Problems
  5. Serverless Workflows with Durable Functions and Netherite
  6. Serverless Computing: One Step Forward, Two Steps Back
  7. ORION and the Three Rights: Sizing, Bundling, and Prewarming for Serverless DAGs
  8. Occupy the Cloud: Distributed Computing for the 99% (PyWren)
  9. Encoding, Fast and Slow: Low-Latency Video Processing Using Thousands of Tiny Threads
  10. SAND: Towards High-Performance Serverless Computing
  11. Encoding, Fast and Slow: Low-Latency Video Processing Using Thousands of Tiny Threads
  12. A Case for Serverless Machine Learning
  13. Archipelago: A Scalable Low-Latency Serverless Platform
  14. Cloudburst: Stateful Functions-as-a-Service

 Yiying
Slides
2/7 Resource Disaggregation
LegoOS: A Disseminated, Distributed OS for Hardware Resource Disaggregation (OSDI'18)
Questions

  1. List two to three pros of resource disaggregation
  2. List two to three cons of resource disaggregation
  3. What is the benefit of using virtual regions in LegoOS?

Additional Readings

  1. Clio: A Hardware-Software Co-Designed Disaggregated Memory System
  2. Mira: A Program-Behavior-Guided Far Memory System
  3. Make It Real: An End-to-End Implementation of A Physically Disaggregated Data Center
  4. Hermit: Low-Latency, High-Throughput, and Transparent Remote Memory via Feedback-Directed Asynchrony
  5. Canvas: Isolated and Adaptive Swapping for Multi-Applications on Remote Memory
  6. Disaggregating Persistent Memory and Controlling Them Remotely: An Exploration of Passive Disaggregated Key-Value Stores

 Yiying
Slides
2/12 Next-Generation Serverless Computing
Resource-Centric Serverless Computing
Questions

  1. What are some of the benefits of using Resource Graphs? Be brief.
  2. Briefly describe how Zenix performs adaptive materialization.
  3. What applications do you think fit FaaS and resource-based serverless? List at least one for each.

Additional Readings

Yiying
Slides
2/14 New Cloud Infrastructure
Amazon Nitro (esp. the video talk on that page)
Quiz 2
Questions

  1. With Amazon Nitro, virtualization functions are mostly offloaded to hardware. Do we still need a hypervisor (or an OS)? Can everything just run in user space and interact with Nitro cards directly?
  2. Can you think of a drawback of offloading tasks to hardware (i.e., Nitro's approach)?

Additional Readings

  1. Intel Unveils Infrastructure Processing Unit

 Yiying
Slides
2/21 KVM and QEMU
kvm: the Linux Virtual Machine Monitor,
and QEMU, a Fast and Portable Dynamic Translator (It's OK to not fully understand Section 2)
Questions

  1. What is the implication of KVM forwarding I/O requests to the user space?
  2. What is the benefit of QEMU first translating the source instructions (guest) into micro-operations implemented in C and their compiled object files and then translating the object files into the target instructions (host)?
  3. Can you think of some good use cases for QEMU+KVM?

Additional Readings

  1. KVM Documentation

 Yiying
Slides
2/26 Security
When Virtual is Harder than Real: Security Challenges in Virtual Machine Based Computing Environments (HotOS'05)
and Hey, You, Get Off of My Cloud: Exploring Information Leakage in Third-Party Compute Clouds (CCS'09)
Questions

  1. Can you think of some drawback of enforcing security mechanisms at the hypervisor level (compared to at the guest OS or above)?
  2. When a zone and/or an instance type are used more frequently (i.e., having higher loads from more tenants), do you think the co-location attack would be come easier or harder? Why?
  3. Do you think a similar co-location attack exist with serverless computing (i.e., one function attacking another function on the same physical machine)? Does serverless computing make such attacks harder or easier and why?

Additional Readings

  1. Secure Container Isolation: Problem Statement & Solution Space
  2. When Virtual Is Better Than Real (HotOS'01)
  3. Secure Pods: Sandboxing workloads in Kubernetes
  4. TrustVisor: Efficient TCB Reduction and Attestation
  5. SecVisor: A Tiny Hypervisor to Provide Lifetime Kernel Code Integrity for Commodity OSes (SOSP'07)
  6. Breaking Up is Hard to Do: Security and Functionality in a Commodity Hypervisor (SOSP'11)
  7. InkTag: Secure Applications on an Untrusted Operating System (ASPLOS'13)
  8. Overshadow: A Virtualization-Based Approach to Retrofitting Protection in Commodity Operating Systems
  9. VirtuOS: An Operating System with Kernel Virtualization
  10. SCONE: Secure Linux Containers with Intel SGX
  11. Understanding Security Implications of Using Containers in the Cloud (ASPLOS'08)
  12. Container Security: Issues, Challenges, and the Road Ahead

 Yiying
Slides
2/28 Next-Generation Cloud
User-Defined Cloud (HotOS'21) and From Cloud Computing to Sky Computing (HotOS'21)
Questions

  1. Other than the examples given in the UDC paper, can you think of another cloud usage case that could benefit from UDC? and how exactly it can benefit from UDC?
  2. What do you think is the biggest obstacle for cloud users to adopt sky computing? for cloud providers to adopt it?

Additional Readings

Jiaying Yang and Shiv Mulchandani
UDC Slides, Sky Slides
3/4 Systems for (Scalable) Machine Learning - Training
PipeDream: Generalized Pipeline Parallelism for DNN Training (SOSP'19) and Alpa: Automating Inter- and Intra-Operator Parallelism for Distributed Deep Learning (OSDI'22)
Questions

  1. By making the pipeline more smooth (less pipeline bubbles), what tradeoff does PipeDream make? i.e., in what aspect is GPipe better than PipeDream?
  2. What type of parallelism do you think is most widely adopted in practice? Why?
  3. Try to come up with an alternative approach to automatically figure out a parallelism plan (instead of Alpa's way). Is it better/worse than Alpa?

Additional Readings

  1. GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism
  2. Scaling Distributed Machine Learning with the Parameter Server

 Yiying
Slides
3/6 Systems for (Scalable) Machine Learning - Serving and Agents
Efficient Memory Management for Large Language Model Serving with PagedAttention and LLM Powered Autonomous Agents
Questions

  1. List at least two reasons why GPU memory for KV cache is wasted without PagedAttention.
  2. What are the tradeoffs of using larger/smaller block sizes in PagedAttention?
  3. Can you come up with a practical case that needs multiple agents operated as a taskflow (or a DAG) to solve a task?

Additional Readings

  1. FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
  2. FasterTransformer
  3. DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale
  4. Orca: A Distributed Serving System for Transformer-Based Generative Models (OSDI'22)
  5. AlpaServe: Statistical Multiplexing with Model Parallelism for Deep Learning Serving (OSDI'23)
  6. Ray Serve: Scalable and Programmable Serving
  7. FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU
  8. DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Models
  9. Ring Attention with Blockwise Transformers for Near-Infinite Context
  10. Large Language Model based Multi-Agents: A Survey of Progress and Challenges
  11. APIServe: Efficient API Support for Large-Language Model Inferencing
  12. Efficiently Programming Large Language Models using SGLang
  13. LangChain
  14. ReAct: Synergizing Reasoning and Acting in Language Models
  15. Gorilla: Large Language Model Connected with Massive APIs
  16. Guidance: A guidance language for controlling large language models

 Yiying and Bei Pei
Slides
3/11 Course Summary
Hints for Computer System Design - Butler Lampson
Quiz 3
Questions

Read the "Hints for Computer System Design" paper and summarize what you have learned over the course. Feel free to write about anything else you want to comment on the course.

Yiying
Slides
3/13 Project Presentations