3.2. pyopus.parallel.pvm — A virtual machine based on the PVM library

Inheritance diagram of pyopus.parallel.pvm

A virtual machine based on the PVM library (PyOPUS subsystem name: PVM)

Attempts to import pyopus.parallel.pypvm. If import fails the PVM library is not available on the platform and an arror is raised.

class pyopus.parallel.pvm.PVMHostID(dtid, name=None)

A PVM host ID class based on the HostID class.

In the PVM library host IDs are integers. Valid host IDs are nonnegative.

The actual host ID can be accessed as the dtid member.

The host name is stored in the name member provided that it is successfully obtained from the pvmStatus static member of the PVM class. If this fails name is used or Unknown if name is None.

See the HostID class for more information.

bad()
A static member function. Called with PVMHostID.bad(). Returns an invalid host ID.
valid()
Returns True if this PVMHostID object is valid.
class pyopus.parallel.pvm.PVMTaskID(tid)

A PVM task ID class based on the TaskID class.

In the PVM library task IDs are integers. Valid task IDs are nonnegative.

The actual task ID can be accessed as the tid member.

See the TaskID class for more information.

bad()
A static member function. Called with PVMTaskID.bad(). Returns an invalid task ID.
valid()
Returns True if this PVMTaskID object is valid.
class pyopus.parallel.pvm.PVM(debug=0, importUser=False)

A virtual machine class based on the PVM library.

One task is the master and does all the spawning. Others are just spawned workers.

Assumes homogeneous clusters in terms of operating system. This means that LINUX32 and LINUX64 are homogeneous while LINUX32 and WINDOWS32 are not.

See the TaskID class for more information on the constructor.

debug levels above 1 enable task output forwarding to the spawner (see the catchout() PVM library function).

alive()

Returns True if the virtual machine is alive. Uses the mytid() PVM library function for the check (it fails if the VM is down).

Can’t use the more appropriate config() function because it leaks memory due to a bug in the PVM library). This method can be called many times so memory leaks are a bad thing.

checkForIncoming()
Returns True if there is a message waiting to be received.
dtid

Static member that appears once per every Python process using the PVM library.

Represents the PVM host ID (integer) of the task that spawned this task. Can be obtained as PVM.dtid.

formatSpawnerConfig()
Formats the configuration information gathered by a spawner task as a string. Works only if called by a spawner task.
freeSlots()

Returns the number of free slots for tasks in the virtual machine.

The information on the slots is gathered by the updateSpawnerInfo() method.

hostID()
Returns the PVMHostID object corresponding to the host on which the caller task runs.
hosts()

Returns the list of PVMHostID objects representing the nosts in the virtual machine. The information on the hosts is gathered by the updateSpawnerInfo() method.

Works only for hosts that are spawners.

kill(taskID)
Kills a task represented by the taskID PVMTaskID object and updates the pvmStatus static member.
killAll()

Kills all tasks listed in the pvmStatus static member and updates the pvmStatus static member. Does not kill the calling task.

Works only on spawner.

orderHosts(hostList)

Returns a list of PVMHostID objects representing the members of hostList list of PVMHostID objects sorted by free slots (most free slots first).

The information on used slots comes from the pvmStatus static member. If a host is marked as not responsive, it is ommitted from the list.

parentTaskID()
Returns the PVMTaskID object corresponding to the task that spawned the caller task.
ptid

Static member that appears once per every Python process using the PVM library.

Represents the PVM parent task ID (integer) of the task that spawned this task. Can be obtained as PVM.tid.

pvmStatus

Static member that appears once per every Python process using the PVM library.

Represents the status of the PVM virtual machine. Updated by the updateSpawnerInfo() method. Available only to spawners. See updateSpawnerInfo() for more information.

Can be obtained as PVM.pvmStatus.

receiveMessage(timeout=-1.0)

Receives a message (a Python object) and returns a tuple (senderTaskId, message)

The sender of the message can be identified through the senderTaskId object of class PVMTaskID.

If timeout (seconds) is negative the function waits (blocks) until some message arrives. If timeout>0 seconds pass without receiving a message, an empty tuple is returned. Zero timeout performs a nonblocking receive which returns an empty tuple if no message is received.

In case of an error the return value is None.

Handles transparently all

  • new hosts notification messages
  • failed host notification messages
  • task exit notification messages
  • spawned function return value messages

Discards all other low-level PVM messages that were not sent with the sendMessage() method.

sendMessage(destination, message)
Sends message (a Python object) to a task with PVMTaskID destination. Returns True on success.
slots()

Returns the number of slots for tasks in a virtual machine.

The information on the slots is gathered by the updateSpawnerInfo() method.

spawnFunction(function, args=(), kwargs={}, count=None, targetList=None, sendBack=False)

Spawns a count instances of a Python function on remote hosts and passes args and kwargs to the function. Spawning a function actually means to start a Python interpreter, import the function, and call it with args and kwargs.

targetList specifies the hosts on which the tasks are started.

If sendBack is True the status and the return value of the spawned function are sent back to the spawner with a pyopus.parallel.base.MsgTaskResult message when the spawned function returns.

Invokes the runFunctionWithArgs() function on the remote host for starting the spawned function.

If spawning succeeds, updates the tasks and hosts structures of the pvmStatus member.

Makes sure the spawned tasks are spread as uniformaly as possible across the hosts.

Returns a list of TaskID objects representing the spawned tasks.

See the spawnFunction() method of the VirtualMachine class for more information.

spawnerBarrier(timeout=-1.0)

Tells the PVM library to notify this task (the spawner) to receive information on new hosts and host failures.

timeout in seconds is used where applicable. Negative values stand for infinite timeout.

taskID()
Returns the PVMTaskID object corresponding to the calling task.
tid

Static member that appears once per every Python process useing the PVM library.

Represents the PVM task ID (integer). Can be obtained as PVM.tid.

updateCoreCount(timeout=-1.0)

Updates the CPU core count in the pvmStatus static member.

The update is performed by spawning the reportHostInfo() function on all hosts listed in the pvmStatus static member. The function detects the number of CPU cores and sends back a message with the result. Communication (send back) is performed on the lowest level so receiveMessage() knows nothing of it. The response messages are collected with a given timeout in seconds. Negative timeout means infinite timeout.

If some hosts do not respond within timeout seconds, they are marked as unresponsive.

updateSpawnerInfo(timeout=-1.0)

Updates the internal information used by a spawner task (static members tid, ptid, dtid, and pvmStatus). Applies timeout in seconds where needed. If timeout is negative blocks until all data is collected.

pvmStatus is a dictionary with three members:

  • hosts - information about the hosts in the virtual machine
  • tasks - information about the tasks in the virtual machine
  • narch - the number of architectures in the virtual machine

hosts is a dictionary with dtid (integer host ID) for key and members which are dictionaries with the following members:

  • speed - integer giving the relative speed of the host
  • arch - the architecture of the host (string)
  • name - the name of the host (string)
  • task_count - the number of tasks running on the host
  • ncpu - the number of CPU cores in the host
  • responsive - a boolean flag indicating that the host is alive

tasks is a dictionary with tid (integer task ID) for key and members which are dictionaries with the following members:

  • parent - the tid (integer task ID) of the parent task
  • dtid - the dtid (integer host ID) on which the task is running
  • flags - an integer representing the flags set for the task
  • name - the name of the task
  • pid - the process identifier of the task

hosts and tasks are obtained with the PVM library functions config() and tasks(). Because config() leaks memory (PVM library bug), this method should be called only occasionally.

The ncpu and the responsive dictionary members of the hosts dictionary are obtained with the updateCoreCount() method. Hosts marked as unresponsive are not used for spawning new tasks.

Returns True on success.

updateWorkerInfo()
Updates the internal information used by a worker task (static members tid, ptid, and dtid).
waitForVM(timeout=-1.0, beatsPerTimeout=50)

Waits for the virtual machien to come up (becoem alive). Returns True if it does.

See the waitForVM() method of the VirtualMachine class for more information.

Previous topic

3.1. pyopus.parallel.base — Base classes for virtual machines

Next topic

3.3. pyopus.parallel.pypvm — The pypvm PVM library wrapper

This Page