Imagine the case where a node has computed a vector of data that must be added to a linear algebra object, e.g. all or part of a column of a matrix. There are a number of possibilities:
Clearly, considerable complexity results from managing the required communication. Add to this the distinct possibility that many or all nodes may be simultaneously generating vectors of data to be entered in linear algebra objects and the code that generates the problems to be passed to PLAPACK or other parallel linear algebra packages can become unmanageable.
- The given node may own the target entries in the linear algebra object. In this case, a local operation suffices.
- Another node owns the target entries in the linear algebra object. Now, communication is required before the data can be added to the linear algebra object. Moreover, if conventional MPI communication calls are used, the target node must be aware that the data is about to arrive.
- Many nodes own some of the the target entries in the linear algebra object. Now a number of communications are required, and all the target nodes must be aware of the data that is about to arrive.
Once an application has entered the PLAPACK API-active state, a call to the former of the following routines will achieve the above described operation. The latter call will retrieving information from global objects. place HR here
place HR here We first discuss PLA_API_axpy_vector_to_global. On the calling node, the local vector is given by its size, in size, the address where it starts in memory, in local_vector, and the stride in memory between entries in the vector, in local_stride. The target object is given by obj, which can be of any object type compatible with a vector (unit global length or width). The displacement, in displ, gives the starting index in the target object where the data in the local vector is to be placed. The scaling parameter, in alpha, allows a multiple of the local data to be added. The operation performed is
where y represents the target (global) object
and x the local vector.
(Notice that the call is like the ``axpy'' BLAS call, described
in Chapter 
.)
The notation y( i:j ) is used to indicate the sub-vector
of y starting at entry i and ending at entry j .
The data type of the data at the address alpha
is assumed to be the same as the data type of the local vector and
the target object. Notice that the call is only performed
on the node that owns the local vector. All nodes can simultaneously
perform such calls to enter local vectors in global objects.
The operation performed by PLA_API_axpy_global_to_vector reverses the above. Now y equals the local vector and x the global, so that the operation performed becomes
Again, only the node that owns the local vector makes the call and all nodes can make similar calls simultaneously.
It is important to realize that the above calls only initiate the operation. The data cannot be assumed to have arrived in the target until the object is closed, or a call to PLA_Obj_API_sync, to be discussed later in this chapter, is performed for the given global object.
Often, a locally computed vector contributes to a given object, but not as a contiguous sub-vector. For this PLAPACK provides a call that contributes multiple sub-vectors. PLAPACK also provides the mirror operation, gathering a number of sub-vectors from a global object into a local vector. place HR here
place HR here Notice that the code segment
int nsub, *sizes, *displs;
double *y, alpha;
PLA_Obj obj = NULL;
<...>
PLA_API_multi_axpy_vector_to_global( nsub, sizes, &alpha, y, 1, obj, displs )
<...>
is equivalent to the loop
int nsub, *sizes, *displs, i;
double *y, alpha;
PLA_Obj obj = NULL;
<...>
local_displ = 0;
for ( i=0; i<nsub; i++ ) {
PLA_API_axpy_vector_to_global( sizes[i], &alpha, &y[ local_displ ], 1,
obj, displs[i] );
local_displ += sizes[i];
}
<...>