4.7. pyopus.optimizer.ppe — Parallel point evaluator

Inheritance diagram of pyopus.optimizer.ppe

Parallel point evaluator (PyOPUS subsystem name: PPE)

Evaluates mutually independent points in parallel. Can be used for parallel implementations of Monte-Carlo analysis, and parametric sweeps. The set of points for evaluation is specified by means of point generator objects.

class pyopus.optimizer.ppe.PointGenerator

A generic point generator class

Point generators are callable. The calling convention is object(n) where n is the number of points to generate. The return value is a tuple of two arrays (indices, chunk). indices is a 1-dimensional array with the indices of the n generated points while chunk is a 2-dimensional array with first index specifying the point and the second index specifying the point’s components.

The generator returns None for chunk when no more points can be generated.

The generator keeps track of point indices so every generated point receives a unique index strting from 0. The indices of the points in the first chunk are 0..n-1. Points in the second chunk have indices n..2n-1

reset()
Resets the generator to its initial state. This means that the point index becomes 0.
class pyopus.optimizer.ppe.GenUniform(xlo, xhi)

A point generator for generating uniformly distributed random points from a rectangle specified by lower and upper bounds on point’s components.

xlo and xhi are the 1-dimensional arrays of lower and upper bounds with matching lengths. The lengths of xlo and xhi define the dimension of space from which the points are taken.

class pyopus.optimizer.ppe.GenNormal(xmean, xstdev, xtruncate=None)

A point generator for generating points with normally distributed components.

xmean and xstddev are the 1-dimensional arrays of mean values and standard deviations of point’s components. The length of stddev must be the same as the length of mean.

If a truncation factor is specified with truncate the points that lie outside mean \pm truncate \cdot stddev are regenerated until they lie within that interval. truncate must be a 1-dimensional array with length 1 (applied to all point components) or its length must be the same as the length of mean and stddev.

class pyopus.optimizer.ppe.GenPointSet(pointSet)

A point generator that generates the poits specified by array pointSet. The first index in the array is the point index while the second index is the point component index. The points are returned in the same order as they appear in pointSet.

When the set is exhausted the generator returns None. the reset() method resets the generator to its initial state and starts over generating points at the first point in pointSet.

class pyopus.optimizer.ppe.MsgEvaluateChunk(indices, xchunk)
A message sent by a master to a slave requesting the evaluation of the points given by chunk. The indices of the points are given by indices.
class pyopus.optimizer.ppe.MsgResult(xchunk, fchunk, annotations=None)
A message sent by a slave reporting the cost function values fchunk calculated for the points in xchunk. annotations is a list of annotations corresponding to the points in the chunk.
class pyopus.optimizer.ppe.ParallelPointEvaluator(function, generators, debug=0, fstop=None, maxiter=None, chunkSize=1, vm=None, maxSlaves=None, minSlaves=0, loadRatio=1)

Parallel point evaluator class

Evaluates function on a set of points generated by a list of generators. If the generators list has more than one member then every generated point is obtained by concatenating the components produces by individual generators in the same order as they appear in the generators list.

maxiter specifies the number of points to evaluate. If some generator produces None when it is called (indicating it ran out of points) the algorithm is stopped regardless of maxiter.

fstop (if it is not None) stops the algorithm when a point is found where the value of function is below fstop. It can be used to implement Monte-Carlo optimization that stops when a goal (fstop) is reached.

If a virtual machine object is passed as the vm argument the evaluations are spread out across the virtual machine and performed in parallel.

If the evaluation of a single point takes too little time the parallel algorithm can be very inefficient (the communication delay dominates over computation time). Setting chunkSize (the number of points that are evaluated by one task) to a larger value improves the efficiency.

The results can be collected by a plugin (see for instance the pyopus.evaluator.cost.CostCollector class). The order in which the results are obtained is not nccessarilly the order in which they are roduced by the generators. Therefore every point receives a unique index when it is generated and this index is available in the index member of the ParallelPointEvakuator object when a point is registered (plugins are called).

The algorithm is capable of handling notification messages received when a slave fails or a new host joins the virtual machine. In the first case the work that was performed by the failed slave is reassigned. In the second case new slaves are spawned. The latter is performed by the handler in the EventDrivenMS class. Work is assigned to new slaves when the algorithm detects that they are idle.

See the Optimizer and the EventDrivenMS classes for more information.

fillHandlerTable()
Fills the handler table of the EventDrivenMS object with the handlers that take care of the parallel point evaluator’s messages.
generateChunk(n=1)

Generates a chunk of n points with the generators listen in the generators list.

Returns a tuple (indices, points) where indices is a 1-dimensional array of indices corresponding to *points. The first index in the points array is the index of a point within the array and the second index is the index of the point’s component.

handleEvaluateChunk(source, message)
Handles MsgEvaluateChunk messages received by slaves. Evaluates the received points and sends back a MsgResult message.
handleIdle(source, message)

Handles a MsgIdle message.

Distributes work to the slaves by sending them MsgEvaluateChunk messages.

handleResult(source, message)

Handles a MsgResult message and takes care of the evaluation result.

Marks the slave that sent the message as idle so that new work can be assigned to it by the MsgIdle message handler.

handleSlaveStarted(source, message)
Handles the MsgSlaveStarted message received by the master from a freshly started slave. Does nothing special, just calls the inherited handler from the EventDrivenMS class.
handleStartup(source, message)

Handles a MsgStartup message received by master/slave when the event loop is entered.

Performs some initializations on the master and prevents the plugins in the slave from printing messages.

handleTaskExit(source, message)

Handles a notification message reporting that a slave has failed (is down).

Stores the work that was performed at the time the slave failed so it can later be reassigned to some other slave.

run()
Runs the optimization algorithm.

Previous topic

4.6. pyopus.optimizer.sanm — Unconstrained successive approximation Nelder-Mead simplex optimizer

Next topic

4.8. pyopus.optimizer.de — Box constrained differential evolution optimizer

This Page