SimPoint FAQ: How to use simulation points.

Q: I have simulation points, now how do I use them?

A: Once a set of simulation points and their respective weights have been collected, they can be used to quickly simulate parts of a program’s execution to represent the entire execution.   To do this you must first simulate the execution interval for each simulation point, collecting the desired statistics.  To efficiently do this we recommend the following methodology:

1. Sort the simulation points in chronological order

2. Fast forward to the start of each simulation point. This can be done by using the simulation point interval number OR the Start PC with Execution Count.  We recommend using the Start PC and Execution Count.

 Simulation Point Interval Number - Note: Simulation points now start count at 0, so 0 means the first interval of execution = no fast-forwarding. This was changed from SimPoint 2.0!  To get the start of a simulation point,  multiply it by the interval size: e.g. simulation point 15 at 100 million instruction interval size == fast-forward 1500 million i.e. 15*100M from the start of execution. Each simulation point is relative to the start of execution, not the last simulation point given. Start PC with Execution Count -   The preferred method of using the simulation points is to use the instruction PC that marks the start of a simulation point. You fast forward until the specified instruction PC is encountered for the number of times specified by the PC Execution Count. When this occurs, you start simulation for the length of the interval size. This helps remove non-deterministic behavior (due to results returned for some operating system calls) for the simulation of a program-input pair when simulated on different machines.

3. Simulate at desired detail for the size of interval (e.g. for 100 million instruction interval simulation points you need to simulate each interval for 100 million instructions)

4. Repeat steps 2 and 3, fast forwarding from one point to the next combined with detailed simulation until all simulation intervals have been collected

In this manner a set of simulation points can be efficiently gathered, and the next step is combining them to get meaningful results.

Q: OK, I simulated them, now what?

A: The final step in using SimPoint is to combine the simulation points’ simulation results to estimate the full execution.  To combine the simulation points, each point first needs to be weighted by its corresponding weight in the .weights file.  Each weight in the .weights file contains a weight for every simulation point.   The weight represents the percent of overall executed instructions each simulation point represents.  The weight for a simulation point is the total instructions executed by all of the intervals in that simulation point's cluster divided by the total number of instructions executed for the program/input pair.

e.g.  Lets assume we have 3 simulation points with the corresponding weights:  Weights: [.25, .25, .5]

The simulation points are used with the weights to compute the weighted average for a given metric.  For example, if we collected performance statistics in terms of CPI for 3 simulation points (CPIsp1, CPIsp2, CPIsp3) with the weights shown above, then the combined average of these points is:

CPI = 0.25xCPI1 + 0.25xCPI2 + 0.5xCPI3

The computed CPI will be the estimate for the full execution, and this technique can be used with any metric*.

*Pitfall:  If the metric used has instructions as part of the unit (e.g.  CPI = cycles per instruction) then the instructions portion of the metric needs to be in the denominator.  For CPI this is fine, but if instead we wanted to use IPC (instructions per cycle == instructions/cycle) then we would not be able to use the weighted average technique on the raw simulated values.  We would first need to convert the data so that the instructions are in the denominator, compute the weighted average on that data, and than convert it back to the original format.  In terms of IPC, this would mean first converting all the simulated samples to CPI before computing the weighted average, and then converting back to IPC.  The reason for this is that the simulated samples are collected for intervals based on a constant number of instructions (e.g. 100 million instructions).  Adding fractions (e.g. cycles/instructions) requires that the denominators are all the same.  For IPC this would not be the case because the varying parameter, cycles, would be in the denominator.

Q:  How accurate is the estimated result from the simulation points relative to the full simulation result?

A: To quantify the accuracy of SimPoint, we compute the relative error of the SimPoint estimated results (as described on this page) in comparison to the full simulation result.  To do this we take the absolute difference of the two values, and divide by the full simulation result.

E.g. SimPoint CPI=2.1,  full simulation CPI = 2.0

Relative error = |2.1– 2| / 2 = 5% error.

We provide a detailed error evaluation of the tool on the full SPEC 2000 CPU suite in error analysis.

Home ] Up ]