Parallel point evaluator (PyOPUS subsystem name: PPE)
Evaluates mutually independent points in parallel. Can be used for parallel implementations of Monte-Carlo analysis, and parametric sweeps. The set of points for evaluation is specified by means of point generator objects.
A generic point generator class
Point generators are callable. The calling convention is object(n) where n is the number of points to generate. The return value is a tuple of two arrays (indices, chunk). indices is a 1-dimensional array with the indices of the n generated points while chunk is a 2-dimensional array with first index specifying the point and the second index specifying the point’s components.
The generator returns None for chunk when no more points can be generated.
The generator keeps track of point indices so every generated point
receives a unique index strting from 0. The indices of the points in the
first chunk are . Points in the second chunk have indices
A point generator for generating uniformly distributed random points from a rectangle specified by lower and upper bounds on point’s components.
xlo and xhi are the 1-dimensional arrays of lower and upper bounds with matching lengths. The lengths of xlo and xhi define the dimension of space from which the points are taken.
A point generator for generating points with normally distributed components.
xmean and xstddev are the 1-dimensional arrays of mean values and standard deviations of point’s components. The length of stddev must be the same as the length of mean.
If a truncation factor is specified with truncate the points that lie
outside are regenerated until they
lie within that interval. truncate must be a 1-dimensional array with
length 1 (applied to all point components) or its length must be the same
as the length of mean and stddev.
A point generator that generates the poits specified by array pointSet. The first index in the array is the point index while the second index is the point component index. The points are returned in the same order as they appear in pointSet.
When the set is exhausted the generator returns None. the reset() method resets the generator to its initial state and starts over generating points at the first point in pointSet.
Parallel point evaluator class
Evaluates function on a set of points generated by a list of generators. If the generators list has more than one member then every generated point is obtained by concatenating the components produces by individual generators in the same order as they appear in the generators list.
maxiter specifies the number of points to evaluate. If some generator produces None when it is called (indicating it ran out of points) the algorithm is stopped regardless of maxiter.
fstop (if it is not None) stops the algorithm when a point is found where the value of function is below fstop. It can be used to implement Monte-Carlo optimization that stops when a goal (fstop) is reached.
If a virtual machine object is passed as the vm argument the evaluations are spread out across the virtual machine and performed in parallel.
If the evaluation of a single point takes too little time the parallel algorithm can be very inefficient (the communication delay dominates over computation time). Setting chunkSize (the number of points that are evaluated by one task) to a larger value improves the efficiency.
The results can be collected by a plugin (see for instance the pyopus.evaluator.cost.CostCollector class). The order in which the results are obtained is not nccessarilly the order in which they are roduced by the generators. Therefore every point receives a unique index when it is generated and this index is available in the index member of the ParallelPointEvakuator object when a point is registered (plugins are called).
The algorithm is capable of handling notification messages received when a slave fails or a new host joins the virtual machine. In the first case the work that was performed by the failed slave is reassigned. In the second case new slaves are spawned. The latter is performed by the handler in the EventDrivenMS class. Work is assigned to new slaves when the algorithm detects that they are idle.
See the Optimizer and the EventDrivenMS classes for more information.
Generates a chunk of n points with the generators listen in the generators list.
Returns a tuple (indices, points) where indices is a 1-dimensional array of indices corresponding to *points. The first index in the points array is the index of a point within the array and the second index is the index of the point’s component.
Handles a MsgIdle message.
Distributes work to the slaves by sending them MsgEvaluateChunk messages.
Handles a MsgResult message and takes care of the evaluation result.
Marks the slave that sent the message as idle so that new work can be assigned to it by the MsgIdle message handler.
Handles a MsgStartup message received by master/slave when the event loop is entered.
Performs some initializations on the master and prevents the plugins in the slave from printing messages.
Handles a notification message reporting that a slave has failed (is down).
Stores the work that was performed at the time the slave failed so it can later be reassigned to some other slave.