2.3. pyopus.evaluator.cost — Cost function management

Inheritance diagram of pyopus.evaluator.cost

Cost function management module (PyOPUS subsystem name: CE)

Normalization is the process where a performance measure is scaled in such way that values not satifying the goal result in normalized values smaller than 0. If the performance measure value exceeds the goal the normalized value is greater than 0.

Cost shaping is the process where the normalized performance is shaped. Usually positive values (corresponding to performance not satifying the goal) are shaped differently than negative values (corresponding to performance satifying the goal). Cost shaping results in a shaped cost contribution for every corner.

Corner reduction is the process where shaped cost contributions of individual corners are incorporated into the cost function. There are several ways how to achieve this. For instance one could incorporate only the contribution of the corner in which worst peroformance is observed, or on the other hand one could incorporate the mean contribution of all corners.

The main data structure is the cost function description which is a list of cost function component descriptions. Every cost function component description is a dictionary with the following members:

  • measure - the name of the performance meeasure on which the cost function component is based.
  • goal - a string that evaluates to an object representing the normalization of the performance measureof.
  • shape - a string that evaluates to an object representing the cost shaping of normalized performance measure. Defaults to CSlinear2(1.0, 0.0).
  • reduce - a string that evaluates to an object representing the corner reduction of shaped cost cotributions. Defaults to CCworst().

The cost function is a function of the input parameters. The cost function input description is a dictionary of parameter descriptions with parameter name as key. Every parameter description is a dictionary with the following members:

  • init - the initial value of the parameter
  • lo - the upper bound on the parameter
  • hi - the lower bound on the parameter
  • step - the step in which the parameter is incremented

A parameter dictionary is a dictionary with parameter name as key holding the values of parameters.

The ordering of parameters is a list of parameter names that defines the order in which parameter values apper in a parameter vector.

A parameter vector is a list of parameter values where the values are ordered according to a given ordering of input parameters.

pyopus.evaluator.cost.formatParameters(param, inputOrder=None, nParamName=15, nNumber=15, nSig=6)
Returns string representation of a parameter dictionary param where the ordering of parameters is specified by inputOrder. The width of parameter name and parameter value formatting is given by nParamName and nNumber, while nSig specifies the number of significant digits.
pyopus.evaluator.cost.parameterDictionary(inputOrder, parameterVector)
Returns a parameter dictionary corresponding to parameterVector where the ordering of parameters is given by inputOrder. parameterVector can be a list of values or an array.
pyopus.evaluator.cost.parameterVector(inputOrder, paramDict)
Return a list of parameter values in order inputOrder corresponding to paramDict parameter dictionary.
pyopus.evaluator.cost.parameterSetup(inputOrder, costInput)
Returns tuple (initVec, loVec, hiVec, stepVec) specifying the lists of initial, low, high, and stepvalues for every parameter. Parameter order is given by the inputOrder while the values are taken from the cost function input description given by costInput.
class pyopus.evaluator.cost.MNbase(goal, norm=None, failure=10000.0)

Basic normalization class. Objects of this class are callable. The calling convention of the object is object(value) where value is a scalar or an array of performance measure values. When called with a scalar the return value is a scalar. When called with an array the return value is an array of the same shape where every component is treated as if it was a scalar.

The return value is greater than 0 if the passed value fails to satisfy the goal. It is less than 0 if the passed value exceeds the goal. Exceeding the goal by norm means that the return value is -1.0. Failing to satify the goal by norm results in a return value of 1.0. If the value passed at call is None, the return value is equal to failure.

If norm is not given the default normalization is used which is equal to goal/10 or 1.0 if goal is equal to 0.0. If goal is a vector, norm must either be a vector of the same size or a scalar is which case it applies to all components of the goal.

report(name, nName=12, nGoal=12, nSigGoal=3)

Formats the goal as a string of the form

name normalization_symbol goal where name

is the name of the performance measure. The normalization_symbol depends on the type of normalization (derived class).

nName and nGoal specify the width of performance measure name and goal formatting. nSigGoal is the number of significant digits of the goal in the formatted string.

worst(values, total=False)

Returns the worst performance value across all corners (the one with the largest normalized value). The values across corners are given in the values array where first array index is the corner index.

If the array has more than 1 dimension the worst value is sought along the first dimension of the array. This means that if value is of shape (n, m1, m2, ...) then the return value is of shape (m1, m2, ...). The return value is an array of performance measure values.

If total is True the worst value is sought acros the whole array and the return value is a scalar worst performance value.

worstCornerIndex(values, corners, total=False)

Returns the index corresponding to the corner where the performance measure takes its worst value (the one with the largest normalized value). The values across corners are given in the values array where first array index is the corner index.

If the array has more than 1 dimension the worst value is sought along the first dimension of the array. This means that if value is of shape (n, m1, m2, ...) then the return value is of shape (m1, m2, ...). The return value as an array of corner indices.

The corner indices corresponding to the first dimension of values are given by corners.

If total is True the worst value is sought across the whole array and the return value is a scalar worst corner index.

class pyopus.evaluator.cost.MNabove(goal, norm=None, failure=10000.0)

Performance normalization class requiring the performance to to be above the given goal. See MNbase for more information.

report(name, nName=12, nGoal=12, nSigGoal=3)

Format the goal as a string. The output is a string of the form

name > goal

See MNbase.report() method for more information.

worst(values, total=False)
Find the worst value. See MNbase.worst() method for more information.
worstCornerIndex(values, corners, total=False)
Find the worst corner index. See MNbase.worstCornerIndex() method for more information.
class pyopus.evaluator.cost.MNbelow(goal, norm=None, failure=10000.0)

Performance normalization class requiring the performance to to be below the given goal. See MNbase for more information.

report(name, nName=12, nGoal=12, nSigGoal=3)

Format the goal as a string. The output is a string of the form

name < goal

See MNbase.report() method for more information.

worst(values, total=False)
Find the worst value. See MNbase.worst() method for more information.
worstCornerIndex(values, corners, total=False)
Find the worst corner index. See MNbase.worstCornerIndex() method for more information.
class pyopus.evaluator.cost.CSlinear2(w=1.0, tw=0.0)

Two-segment linear cost shaping. Normalized performances above 0 (failing to satify the goal) are multiplied by w, while the ones below 0 (satisfying the goal) are multiplied by tw. This cost shaping has a discontinuous first derivative at 0.

Objects of this class are callable. The calling comvention is object(value) where value is an array of normalized performance measures.

class pyopus.evaluator.cost.CCbase

Basic corner reduction class. Objects of this class are callable. The calling convention of the object is object(shapedMeasure) where shapedMeasure is an array of shaped cost contributions. The return value is a scalar.

flagFailure()
Return a string that represents the flag for marking performance measures for which the process of evaluation failed (their value is None).
flagSuccess(fulfilled)
Return a string that represents a flag for marking performance measures that satify the goal (fulfilled is True) or fail to satisfy the goal (fulfilled is False).
class pyopus.evaluator.cost.CCexcluded

Corner reduction class for excluding the performance measure from the cost function. Objects of this class are callable and return 0. See CCbase for more information.

flagFailure()

Return a string that represents the flag for marking performance measures for which the process of evaluation failed (their value is None). Returns 'x'.

See CCbase.flagFailure() method for more information.

flagSuccess(fulfilled)

Return a string that represents a flag for marking performance measures. A successfully satisfied goal is marked by ' ' while a failure is marked by '.'.

See CCbase.flagSuccess() method for more information.

class pyopus.evaluator.cost.CCworst

Corner reduction class for including only the worst performance measure across all corners. Objects of this class are callable and return the larget cost contribution.

See CCbase for more information.

flagFailure()

Return a string that represents the flag for marking performance measures for which the process of evaluation failed (their value is None). Returns 'X'.

See CCbase.flagFailure() method for more information.

flagSuccess(fulfilled)

Return a string that represents a flag for marking performance measures. A successfully satisfied goal is marked by ' ' while a failure is marked by 'o'.

See CCbase.flagSuccess() method for more information.

class pyopus.evaluator.cost.CCmean

Corner reduction class for including only the mean cost contribution across all corners. Objects of this class are callable and return the mean of cost contribution passed at call.

See CCbase for more information.

flagFailure()

Return a string that represents the flag for marking performance measures for which the process of evaluation failed (their value is None). Returns 'X'.

See CCbase.flagFailure() method for more information.

flagSuccess(fulfilled)

Return a string that represents a flag for marking performance measures. A successfully satisfied goal is marked by ' ' while a failure is marked by 'o'.

See CCbase.flagSuccess() method for more information.

class pyopus.evaluator.cost.CostEvaluator(perfEval, inputOrder, costDefinition, debug=0)

Cost evaluator class. Objects of this class are callable. The calling convention is object(paramVector) where paramvector is an array of input parameter values. The ordering of input parameters is given at object construction. The return value is the value of the cost function.

perfEval is an object of the PerformanceEvaluator class which is used for evaluating the performance measures of the system. inputOrder is the ordering of system’s input parameters. costDefinition is the cost function description.

If debug is set to a value greater than 0, debug messages are generated at the standard output.

Objects of this class store the details of the last evaluated cost function value in the results member which is a list (one member for every cost function component) of dictionaries with the following members:

  • worst - the worst value of corresponding performance mesure across corners where the performance measure was computed. This is the return value of the normalization object’s MNbase.worst() method when called with with total set to True. None if performance measure evaluation fails in at least one corner
  • worst_vector - a vector with the worst values of the performance measure. If the performance measure is a scalar this is also a scalar. If it is an array of shape (m1, m2, ...) then this is an array of the same shape. This is the return value of the normalization object’s MNbase.worst() method with total set to False. None if performance measure evaluation fails in at least one corner.
  • worst_corner - the index of the corner in which the worst value of performance measure occurs. If the performance measure is an array of shape (m1, m2, ..) this is still a scalar which refers to the corner index of the worst performance measure across all components of the performance measure in all corners. This is the return value of the normalization object’s MNbase.worstCornerIndex() method with total set to True. If the performance evaluation fails in at least one corner this is the index of one of the corners where the failure occurred.
  • worst_corner_vector - a vector of corner indices where the worst value of the performance measure is found. If the performance measure is a scalar this vector has only one component. If the performance measure is an array of shape (m1, m2, ...) this is an array with the same shape. This is the return value of the normalization object’s MNbase.worstCornerIndex() method with total set to False. If the evaluation of a performance measure fails in at least one corner this vector holds the indices of corners in which the failure occured.
  • contribution - the value of the contribution to the cost function. This is always a number, even if the evaluation of some performance measures fails (see failure argument to the constructor of normalization objects - e.g. MNbase).
  • fulfilled - True if the corresponding performance measure is successfully evaluated in all of its corresponding corners and all resulting values satisfy the corresponding goal. False otherwise.

Corner indices refer to corners as they are defined by the corners argument a list of corner descriptions) to the constructor of perfEval object (see PerformanceEvaluator class).

The paramVector member holds the input parameter values passed at the last call to this object.

allBelowOrAtZero()

Returns True if all cost contributions obtained with the last call to this object are not greater than zero. Assuming that the following holds:

  • normalization produces positive values for satisfied goals and negative values for unsatisfied goals
  • normalization returns a positive value in case of a failure to evaluate a performance measure (failed is greater than 0)
  • cost function shaping is nondecreasing and is greater than zero for positive normalized performance measures

the return value is True, if all performance measures that appear in cost function components not using the CCexcluded corner reduction satisfy their goals.

allFulfilled()
Returns True if the performance measures corresponding to all cost function components that were evaluated with the last call to this object were successfully evaluated and fulfill their corresponding goals. All cost function components are taken into account, even those with the CCexcluded corner reduction.
formatParameters(x=None, nParamName=15, nNumber=15, nSig=6)
Formats a string corresponding to the parameters passed at the last call to this object. Generates one line for every parameter. If x is specified it is used instead of the stored parameter vector. nParamName and nNumber specify the width of the formatting for the parameter name and its value. nSig specifies the number of significant digits.
formatResults(nTargetSpec=29, nMeasureName=12, nNumber=12, nSig=3, nCornerName=6)

Formats a string representing the results obtained with the last call to this object. Only the worst performance across corners along with the corresponding cost function component values is reported. Generates one line for every cost function component.

nTargetSpec specifies the formatting width for the target specification (specified by the corresponding normalization object) of which nMeasureName is used for the name of the performance measure. nNumber and nSig specify the width of the formatting and the number of the significant digits for the cost function contribution. nCornerName specifies the width of the formatting for the worst corner name.

getAnnotator()
Returns an object of the CostAnnotator class which can be used as a plugin for iterative algorithms. The plugin takes care of cost function details (results member) propagation from the machine where the evaluation of the cost function takes place to the machine where the evaluation was requested (usually the master).
getCollector(chunkSize=10)

Returns an object of the CostCollector class which can be used as a plugin for iterative algorithms. The plugin gathers input parameter and cost function values across iterations of the algorithm.

chunkSize is the chunk size used when allocating space for stored values (10 means allocation takes place every 10 iterations).

getReporter(reportParameters=True)
Returns an object of the ReportCostCorners class which can be used as a plugin for iterative algorithms. Every time an iterative algorithm calls this CostEvaluator object the reporter is invoked and prints the details of the cost function components.
getStopWhenAllSatisfied()
Returns an object of the StopWhenAllSatisfied class which can be used as a plugin for iterative algorithms. The plugin signals the iterative algorithm to stop when all cost contributions obtained with the last call to this CostEvaluator object are smaller than zero (when the allBelowOrAtZero() method returns True).
class pyopus.evaluator.cost.CostAnnotator(costEvaluator)

A subclass of the Plugin iterative algorithm plugin class. This is a callable object whose job is to

  • produce an annotation (details of the cost function value) stored in the costEvaluator object (when invoked with None for annotation)
  • update the costEvaluator object with the given annotation (when invoked with an annotation that is not None)

Annotation is a copy of the results member of costEvaluator.

Annotators are used for propagating the details of the cost function from the machine where the evaluation takes place to the machine where the evaluation was requested (usually the master).

class pyopus.evaluator.cost.CostCollector(chunkSize=100)

A subclass of the Plugin iterative algorithm plugin class. This is a callable object invoked at every iteration of the algorithm. It collects the input parameter vector (n components) and the cost function value.

Let niter denote the number of stored iterations. The input parameter values are stored in the xval member which is an array of shape (niter, n) while the cost function values are stored in the fval member (array with shape (niter)).

Some iterative algorithms do not evaluate iterations sequentially. Such algorithms denote the iteration number with the index member. If the index member is not present in the iterative algorithm object the internal iteration counter of the CostCollector is used.

The first index in the xval and fval arrays is the iteration index. If iterations are not performed sequentially these two arrays may contain gaps where no valid input parameter or cost function value is found. The gaps are denoted by the valid array (of shape (niter)) where zeros denote a gap.

xval, fval, and valid arrays are resized in chunks of size chunkSize.

finalize()

Removes the space beyond the recorded iteration with highest iteration number. This space was reserved for the last chunk, but the highest recorded iteration may not be the one recorded at the end of the chunk.

Must be called after the iterative algorithm is stopped.

reset()
Clears the xval, fval, and valid members.
class pyopus.evaluator.cost.ReportCostCorners(costEvaluator, reportParameters=True)

A subclass of the Reporter iterative algorithm plugin class that reports the details of the last evaluated cost function value obtained by the costEvaluator object. Uses the CostEvaluator.reportParameters() and CostEvaluator.formatResults() methods of costEvaluator for obtaining the output that is printed at first iteration and every time the cost function value decreases.

If reportParameters is True the report includes the corresponding input parameter values.

The ReportCostCorners takes care of annotations (i.e. once it is installed in an iterative algorithm no further CostAnnotator is needed).

Previous topic

2.2. pyopus.evaluator.performance — System performance evaluation

Next topic

3. pyopus.parallel — Parallel computation support

This Page