Using Berkeley UPC with TotalView
As of version 2.2, Berkeley UPC programs can be debugged by the "TotalView"
debugger produced by Rogue Wave Software. Support for
UPC-level source examination, stepping through UPC code, and examination of
shared variable's values are all supported.
If you do not have TotalView, you can also use a regular C
debugger and get partial debugging support. See
Attaching a regular C debugger to
Berkeley UPC programs for details.
Requirements for debugging Berkeley UPC with TotalView
- TotalView version 7.0.1 or
greater.
- Support is currently only provided for Berkeley UPC installations
running on x86 architectures, using either MPICH MPI or
Quadrics "elan" for the network.
(Other MPI implementations may work if they have integrated TotalView
support into their 'mpirun' command, but this has not been as well tested).
- The back-end C compiler used by Berkeley UPC must be GNU GCC (other
C compilers may work, but have not been tested).
- You must enable TotalView support when building the Berkeley runtime
layer. To enable TotalView support, include this option in your invocation
of the Berkeley UPC runtime configure to activate the TotalView conf:
--with-multiconf=+dbg_tv
(if your configure line already includes a --with-multiconf clause, then
append ",+dbg_tv" to the existing value).
Compiling programs for TotalView
You must compile with the '-tv' flag if you will be debugging your UPC
program with TotalView:
upcc -tv foo.upc
The '-tv' flag implicitly turns on '-g', and also ensures that
the executable is instrumented for TotalView support.
TotalView does not currently support pthreaded Berkeley UPC executables: you
cannot compile with both '-tv' and '-pthreads'.
Running Berkeley UPC programs under TotalView
Unfortunately, 'upcrun' and TotalView do not yet work together. You
must instead use the underlying network spawner for your network type:
Depending on your TotalView installation, you may need to have certain
environment variables and/or other settings (such as the TotalView license
manager) working correctly. In general, whatever settings you would need to use
TotalView with MPI programs will also be needed for debugging UPC programs. On
the author's system, for instance, the TVDSVRLAUNCHCMD environment variable must
be set to "ssh". Consult your system administrator and/or your TotalView
documentation for details.
Setting up TotalView for Berkeley UPC
When you launch a debugging session using one of the above commands, TotalView
will open one or more windows. In the main, "root" window, select File >
Preferences, and go to the Action Points tab. Change each of the
When ... , stop entries to stop the "Thread" rather than the
"Process". (This picture shows the default settings where all are set to
"Process".)
Close the Preferences Window by hitting the OK button.
You'll only need to do this the first time you use TotalView as they'll be saved
in your TotalView preferences file.
Setting breakpoints and running your program
For MPI programs, you will see your UPC program's code displayed in the main
TotalView window. You may set any breakpoints as you wish in the usual fashion
(for instance by left-clicking on the line number), and then hit "Go" to start
the program. If you see a dialog asking you if you want to "Stop the parallel
job now?", you may select "no", and your program should run until it hits your
first breakpoint.
For elan programs, the code for the 'prun' command itself will be
shown initially, rather than your UPC code. Hit "Go" to launch 'prun',
then attach to the parallel job, and you should see your UPC code, at which time
you can select breakpoints and run your UPC code.
TotalView's support for UPC constructs
TotalView supports a number of UPC constructs. For instance, it knows which
functions in the stack trace are UPC, and which are part of Berkeley UPC's
runtime infrastructure. For example, in the picture below, the "Stack Trace" in
the upper left shows that 'print_shared()' and 'user_main()'
are UPC code, while the functions marked with 'C' are from the Berkeley runtime
initialization logic.
Note: Berkeley UPC renames the 'main()' function in UPC code to
'user_main()'. Other function and variable names in your program
will keep their exact name within TotalView.
TotalView also has support for understanding 'shared' variables. If you
click on (or otherwise "dive" into) a shared variable in your UPC program, you
will see that TotalView correctly displays it. Arrays' values are shown, and
pointers to shared data show all the logical components of the pointer-to-shared
(address, UPC thread, and phase), as well as the value of the data pointed to.
Here, for instance, is the display from clicking on
'shared int bar[4*THREADS]' (this is with THREADS=3, and
array values set to bar[0]=0, bar[1]=1, etc.):
Note how the correct type for the array is shown, as well as its address,
values, and the affinity of each value (in the "Node" field). (Note: to see the
"Node" value, right-click within the header area (the area containing Field and
Value in the previous figure) and select "Node").
You can switch which UPC thread is displayed in the main TotalView window by
hitting the P+ and P- buttons in the bottom right of the window.
"Rank 0" will correspond to UPC thread 0, "Rank 1" to thread 1, etc.
Also, if you are examining a variable that potentially has different values
on different UPC threads, you can see all its values by selecting
View > Laminate > Process from the menu:
More information
This tutorial has only covered UPC-related features in TotalView. For more
information, see your TotalView documentation.