High-end distributed computing and communication systems require a new generation of techniques for predicting the performance of distributed applications. Performance prediction must reflect hardware and software heterogeneity, and adapt to the dynamic variation in the performance deliverable to an application by system resources. A key requirement for any performance prediction technology that will be effective for modern high-end systems is the ability to target performance for a specific time frame. Since the system varies dynamically due to contention for shared resources, application performance can fluctuate dramatically over time.

We are developing a new methodology -- Performance Prediction Engineering -- to address the problem of modeling time-sensitive, dynamic and heterogeneous performance information and using it to predict performance of distributed applications in a metacomputing environment. This methodology involves the design and development of

• Structural Models/Performance Grammars Performance predictions made by our system are generated from compositional models (called "structural models" or "performance grammars"). Structural models consist of components which represent the performance activities (e.g. computation or communication activities) of the application. Each component can be instantiated using time-dependent dynamic parameters at the level of "accuracy" appropriate for its use.
• Quality of Information ("QoIn") Measures Each prediction generated by a PPE can be associated with a set of quantifiable attributes which characterizes its qualitative value. Such attributes might include the "lifetime" of the prediction, its "accuracy", the "overhead" of computing the prediction, etc. QoIn attributes for component models of structural models can be combined to produce QoIn attributes for predictions of the overarching structural model. The quality of performance information critically impacts the effectiveness of performance estimations for scheduling. QoIn measures and values provide a way of quantifying qualitative information so that it can be used to improve application schedules and ultimately application performance.
• Performance Forecasting The basis of all PPE predictions is a set of resource or application component performance forecasts. Dynamic and time-dependent information is critical for accurate modeling. We use the Network Weather Service (NWS) -- a facility which monitors, reports, and forecasts dynamic load and availability of system resources -- to provide performance forecasts based on statistical analysis of continually-updated resource performance histories to parameterize PPE application performance predictions. In this project, we extend the NWS to support PPE by calculating QoIn measures associated with performance predictions, and by enhancing the performance sensors themselves to better support PPE performance predictions.

The confluence of these techniques results in a new definition of performance prediction that is dynamic and time-sensitive, and that carries a quantifiable measure of quality. Exposure of the complex but quantifiable relationship between application requirements and deliverable resource performance will lead to a an increase in the performance achievable by applications on distributed heterogeneous systems.