Validating against a cycle-accurate C++ simulator

By now you are familiar with testbenches and their usages. For this part of Lab 2, you will be using the output of a C++ cycle-accurate fetch unit simulator to verify that your design is correct.

The simulator takes a set of inputs and automatically generates a log file that shows the correct input-output pairs, for comparison against your RTL. It also has a debug file (.dbg) that helps you figure out what evens should be going on in your fetch unit. In order to facilitate the comparison, we have a SystemVerilog testbench that interfaces to your fetch unit and generates the same format input/output file so you can textually diff them to find errors.

To reduce the effort for the lab, we do not require you to actually run the simulator, we provide the input and output files for your convenience. At the end of this document, we provide a link to the svn repo for the simulator, if you are curious. You can easily create your own tests with it.

In this section we provide you 5 testbenches (*.tfw) and their expected outputs (*.chk). Your task is to understand what each of these testbenches is doing, run them and compare the results with reference outputs. If there are differences, you will want to debug them. This way, you know that your fetch unit is ready for use in Lab 3.

Downloading new testbenches

We are providing new testcases and the expected outputs from those testbenches. This will help you make sure that your design is working for simple cases. These files are in the svn repository. To checkout the files do:

svn checkout http://parallel.ucsd.edu/fetch-unit-tests/trunk/advanced

This will checkout the 'advanced' directory. You can find the testbenchs (*.tfw) files in this directory. You will also see (*.in) files which will be used in the testbench, so keep them in the same directory as testbench. You will also see (*.chk) files which will show you the expected input and output.

These files have been created using a C++ simulator/test generator. Given a test vector file the program will generate a testbench and the expected output file.

Using the files

Follow the instructions from this page for doing behavioral or timing simulation. In the part where you need to specify a .tfw testbench file, select one of the ones generated by the tool. You will need to right click on the .tfw file and select "Properties..." and select "SystemVerilog". Note, that each .tfw file includes a .in file with the same prefix; that needs to be findable by the .tfw file (hence why we said to put everything in your simulation/modelsim dir). When you run the testbench, you want to run it for enough time for the testbench to finish all of the parts of the vector (e.g. 1500 ns or more). You can look at the ModelSim "Transcript" window to see if it says it has begin/finished. Note that when you run ModelSim, you should not select the "Package" with the name of the test, but rather the module named test, which should show (on the adjacent column) the path of the test bench file you are trying to run.

Analyze outputs

The testbench outputs a file, vector_<xx>.out, which contains the input/output pairs of the simulation. This file should match vector_<xx>.chk very closely, as described below. The vector_<xx>.dbg file can be used to understand what is going on in the C++ simulation. The .chk and .out logs have the inputs (i.e. _i's) presented to the fetch stage on the left side, and the outputs (i.e. _o's) on the right side. The inputs are captured right before the clock pos edge, and the outputs are captured right before the next clock pos edge. The order expressed in the chk file is:

restart_i restart_addr_i load_store_valid_i store_en_i load_store_addr_i store_data_i dequeue_i ###### load_data_valid_o load_data_o instruction_valid_o instruction_addr_o instruction_data_o

To check for differences, you can inspect the .chk and .out files side-by-side. You can most likely ignore differences that occur before restart is asserted, since the design is in a unknown state. A cycle or two after restart is asserted, your design should match the C++ simulator (*.chk) exactly. You can use the .dbg file to understand what it is doing. It's conceivable that you have a legitimate difference, or that the C++ simulator is wrong, or that the inputs are invalid, but if it is, you should be able to explain exactly why this is. But first, you may want to look for bugs in your own design!

Optional: Tips for Diffing the Files

To look at the differences more carefully, I recommend using the emacs ediff mode, which will visually show you the differeces:

emacs vector_xx.cc.chk vector_xx.cc.out

ESC-x ediff-buffers < enter >

< enter > (to accept currently visible buffer vector_xx_cc.out)

< enter > (to accept currently visible buffer vector_xx_cc.chk)

click on the little Ediff window that pops up to make sure it is in the foreground

| (side by side mode)

* (refine region)

! (update)

(finds next difference, keep on repeating this to cycle through the differences; p goes backwards)

You can also use 'vimdiff' or 'tkdiff' if you are familiar with them. Also, Notepad++ for windows has a "compare" plug-in feature that works well.

Optional: Generate your own test vectors with a fetch unit simulator

If you want to generate your own testbench files and run them then you can access the above mentioned C++ simulator and use it to generate new testbenches. To get the simulator:

svn checkout http://parallel.ucsd.edu/fetch-unit-simulator

After you checkout, you can follow the instructions given in the README file to generate your own testcase. THIS SECTION IS OPTIONAL.