Profiling libraries

The MPI profiling interface provides a convenient way for you to add performance analysis tools to any MPI implementation. We demonstrate this mechanism in MPICH, and give you a running start, by supplying three profiling libraries with the MPICH distribution. MPE users may build and use these libraries with any MPI implementation.

Accumulation of time spent in MPI routines

The first profiling library is simple. The profiling version of each MPI_Xxx routine calls PMPI_Wtime (which delivers a time stamp) before and after each call to the corresponding PMPI_Xxx routine. Times are accumulated in each process and written out, one file per process, in the profiling version of MPI_Finalize. The files are then available for use in either a global or process-by-process report. This version does not take into account nested calls, which occur when MPI_Bcast, for instance, is implemented in terms of MPI_Send and MPI_Recv.

Logfile creation

The second profiling library generates logfiles, which are files of timestamped events. During execution, calls to MPE_Log_event are made to store events of certain types in memory, and these memory buffers are collected and merged in parallel during MPI_Finalize. During execution, MPI_Pcontrol can be used to suspend and restart logging operations. (By default, logging is on. Invoking MPI_Pcontrol(0) turns it off; MPI_Pcontrol(1) turns it back on again.) The calls to MPE_Log_event are made automatically for each MPI call. You can analyze the logfile produced at the end with a variety of tools; these are described in Sections upshot and nupshot and Jumpshot .

Customizing logfiles

In addition to using the predefined MPE logging libraries, you can insert your own calls to the logging routines to define and log states. States may be nested, allowing you to define a state describing a user routine that contains several MPI calls, and display both the user defined state and the MPI operations contained within it. The routines MPE_Describe_state and MPE_Log_event are used to describe user-defined states. The routine MPE_Log_get_event_number may be used to get unique event numbers (this is important if you are writing a library that uses the MPE logging routines).

MPE_Describe_state( 1001, 1002, "Amult", "bluegreen" ); 
... 
MyAmult( Matrix m, Vector v ) 
{ 
    /* Log the start event along with the size of the matrix */ 
    MPE_Log_event( 1001, m->n, (char *)0 ); 
    ... amult code, including MPI calls ... 
    MPE_Log_event( 1002, 0, (char *)0 ); 
} 

If you are not using the MPE logging library, you will also need to use MPE_Init_log and MPE_Finish_log.

Real-time animation

The third library does a simple form of real-time program animation. The MPE graphics library contains routines that allow a set of processes to share an X display that is not associated with any one specific process. Our prototype uses this capability to draw arrows that represent message traffic as the program runs.