Nachos Virtual Memory Project, More Filesystems

March 7 2007

Lecture Review

  1. RPC, NFS
  2. Rio, Journaling, Softupdates

Project 1

I'll talk about some of the more common problems we saw when grading project 1:

  1. Forgetting to disable interrupts, or unnecessarily disabling interrupts
  2. Linked lists
  3. Multiple senders and receivers, multiple whales
  4. Deallocating memory

Project 3

In project 3, you will implement demand paged virtual memory. Make sure you understand how demand paging works before starting this project. In particular, make sure you understand what causes page faults, what the kernel needs to do when page faults occur, and how the valid and dirty bits are used.

Project 3 consists of two main parts:

  1. Lazy Loading: In project 2, you set up every page of the user's address space before starting the user process. To do this, you loaded all the data from the executable into the address space, and zeroed out the user's stack.

    For the first part of project 3, you delay these setup operations for as long as possible. So, for example, if the user program never uses the second page of their text segment, you never load the second page of their text segment into memory.

    To do this, all page table entries are initially marked as invalid, so the machine will raise PageFaultExceptions when it tries to access a page that hasn't been set up yet. You need to catch these exceptions in the Nachos exception handler, figure out which virtual address caused the page fault (check BadVAddrReg), set up that page appropriately, mark the page table entry as valid, and tell the user program to retry their memory operation.

  2. Page Replacement: The previous part dynamically brings in pages as they are needed. But what happens when there are no more free pages of physical memory? You need to kick another page out of physical memory, and that's what happens in this part. If the physical page you want to page out is clean, this is easy: you just invalidate the page table entry that points to the physical page, and the page is yours. But if the page is dirty, you need to write the contents of the physical page to the swap file before you can reuse the physical page. Of course, if this paged-out dirty physical page is needed later, it must be read from the swap file.

    Note that the dirty bit only indicates whether a page needs to be written to the swap file. The dirty bit does not indicate whether a page needs to be read from the swap file. Specifically: if we want to page out a dirty page, we must write that dirty page to the swap file. Suppose we later bring that page back into memory (from the swap file). At this point we mark the page as not dirty, because, at this point, the copy of the page in the swap file is the same as the copy of the page in memory. If we need to kick the page out again, and it is still not dirty, we don't need to re-write the page into the swapfile.

That's pretty much it. :) The same warnings apply to this project as the last project: The two main parts of this project are highly dependent on each other, and you will be tempted to code up the whole thing before testing it. This is not recommended.

Starting Points

Project 3 builds directly on project 2. You will be fiddling with page tables a lot in project 3, so make sure you understand how they work in Nachos.

There is very little new code you need to understand for project 3. You will be making most of your changes in AddrSpace and the exception handler. You will need to write dirty pages to a swap file, which means you will need to do disk I/O in Nachos. Take a look at nachos/filesys/openfile.h, which should explain everything.

RAID

RAID takes a bunch of disks and uses them to create one big virtual disk. Depending on the specific technique used, this can result in improved reliability, performance, or both.

The different kinds of RAID are named with numbered "levels". Very important: higher RAID levels are not necessarily better. The different RAID levels split data across multiple disks in different ways, and they each have advantages and disadvantages. This means that RAID 5 is not always better than RAID 0. Whether it is better or not depends entirely on how the disks will be used: how much do we care about performance? How much disk space are we willing to sacrifice for reliability?

That said, here are four popular RAID configurations, and what they do:

RAID 0
Stripe data across all the disks in the array. If we have two disks, for example, we would put all the even numbered blocks on one disk, and all the odd numbered blocks on the other disk.
RAID 1
Mirror data across all disks in the array. Every disk in the array has a copy of every data block.
RAID 4
Requires at least three disks. If we have three disks, we stripe data across the first two (like RAID 0), and the third disk contains the bitwise XOR of the first two disks. This third disk is called the parity disk.
RAID 5
RAID 5 is an extension of RAID 4. RAID 4 creates a bottleneck at the parity disk, because every write needs to update the parity disk, even if the writes are going to completely different data blocks. RAID 5 works around this by not using a dedicated parity disk - instead, the parity blocks are spread across all disks.

For example, we can put the first and second data blocks on the first two disks, and the parity for the first two blocks on the third disk, but the third and fourth data blocks can be placed on the last two disks, and the parity for the third and fourth blocks would be placed on the first disk.

Questions

  1. How does RAID 4 and 5 recover data using the parity blocks?
  2. How does read and write performance compare across different RAID levels?
  3. Why do we need to worry about metadata consistency?
  4. What are the differences between hard links and symbolic links?
  5. [from Michael] Many Unix systems allow multiple hard links to files, but do not allow hard links to be created to directories. Why do you think this is?