3.3. pyopus.parallel.pypvm — The pypvm PVM library wrapper
The pypvm PVM library wrapper
Written by W. Michael Petullo and Greg Baker.
http://pypvm.sourceforge.net
Functions pkbytestr() and upkbytestr() added by Arpad Buermen.
Currently supported under Linux only.
-
pyopus.parallel.pypvm.pk(obj)
- Packs a Python object obj into the default pvm buffer by picking it and
calling the pkbytestr() function.
-
pyopus.parallel.pypvm.upk()
- Unpacks a Python object obj from the default pvm buffer by first calling
the upkbytestr() function and then unpickles the Python object which
is returned to the caller.
-
pyopus.parallel.pypvm.addhosts()
pypvm.addhosts(hostlist) adds the computers named in [hostlist] to the
configuration of computers making up the virtual machine. The names
should have the same syntax as lines of a pvmd hostfile (see man page
for pvmd3): A hostname followed by options of the form xx=y.
pypvm.addhosts returns a list; this list will (hopefully) just contain
zeros. However, if any of them are negative numbers, they are indicativethat the addhost operation failed on that corresponding
machine. The return results are equal to members of of the pypvm.results
dictionary.
- pypvm.results['BadParam'] => bad hostname syntax.
- pypvm.results['NoHost'] => no such host.
- pypvm.results['CantStart'] => failed to start pvmd on host.
- pypvm.results['DupHost'] => host already configured.
- pypvm.results['BadVersion'] => pvmd protocol versions don’t match.
- pypvm.results['OutOfRes'] => PVM has run out of system resources.
-
pyopus.parallel.pypvm.archcode()
The routine pypvm.archcode() returns an integer given an architecture
name. The code returned identifies machines with compatible binary
data formats. For example, SUN4 and RS6K have the same code, while
ALPHA has a different one (because a few datatypes have different
sizes). This lets you know when you can get away with using
pypvm.data.raw instead of the default encoding to pass messages between
tasks on two machines.
Naturally, you shouldn’t assume the values returned by pvm_archcode
are etched in stone; the numbers have no intrinsic meaning except that
if two different arch names map to the same value then they’re
compatible.
-
pyopus.parallel.pypvm.barrier()
pypvm.barrier(group, count)
The routine pypvm.barrier blocks the calling process until count members
of the group have called pypvm.barrier.
-
pyopus.parallel.pypvm.bcast()
pypvm.bcast(group, msgtag)
Broadcasts the data in the active message buffer to the processes in group
-
pyopus.parallel.pypvm.bufinfo()
pypvm.bufinfo(bufid) returns a tuple of information about a message buffer -
(bytes, msgtag, tid) where
- bytes is an integer returning the length in bytes of the entire message.
- msgtag is an integer returning the message label. Useful when the message
was received with a wildcard msgtag
- tid is an integer returning the source of the message. Useful when the
message was received with a wildcard tid.
-
pyopus.parallel.pypvm.catchout()
pypvm.catchout(file) causes the calling task (the parent) to catch output
from tasks spawned after the call to pypvm.catchout(). Characters printed
on stdout or stderr in children tasks are collected by the pvmds and sent
in control messages to the parent task, which tags each line and appends
it to the specified file. Output from grandchildren (spawned by children)
tasks is also collected, provided the children don’t reset PvmOutputTid.
If file is not specified, or is None, output collection is turned off.
If pypvm.exit() is called while output collection is in effect, it will block
until all tasks sending it output have exited, in order to print all their
output. To avoid this, turn collection off first.
-
pyopus.parallel.pypvm.config()
pypvm.config() returns a three element tuple
- the number of hosts in the PVM cluster
- the number of architectures in the cluster
- a list of dictionaries giving information about each machine in the cluster.
Each dictionary has keys
Note that the speed field may have no particularly relationship
with reality, as it just reporting the number stored in the
startup hostsfile for the master pvmd. Read the pvmd3 man
pages for more information.
The information provided by pypvm.config is also available through
pypvm.hostinfo and pypvm.narch.
-
pyopus.parallel.pypvm.delete()
- pypvm.delete(name, index) deletes entry <name, index> from the database ;
see pvm_insert(3PVM) for a description of this database.
-
pyopus.parallel.pypvm.delhosts()
pypvm.delhosts(hostlist) deletes the computers pointed to in hostlist
from the existing configuration of computers making up the virtual
machine. All PVM processes and the pvmd running on these computers
are killed as the computer is deleted.
The returned array can checked to determine which host caused the error,
and compared against the elements in the pypvm.results dictionary.
-
pyopus.parallel.pypvm.exit()
- The routine pypvm.exit() tells the local pvmd that this process is leaving PVM.
This routine does not kill the process, which can continue to perform tasks just
like any other serial process.
-
pyopus.parallel.pypvm.export()
- pypvm.export(name) is provided for convenience in editing the environment
variable PVM_EXPORT, while maintaining the colon-separated list syntax it
requires.
-
pyopus.parallel.pypvm.freebuf()
- The routine pypvm.freebuf(bufid) frees
the memory associated with the message buffer identified by bufid.
Message buffers are created by pypvm.mkbuf, pypvm.initsend, and pypvm.recv.
-
pyopus.parallel.pypvm.freecontext()
- pypvm.freecontext(ctx) frees ctx so that it may be reused. Contexts
are a system resource that will be exhausted if not recycled.
-
pyopus.parallel.pypvm.getcontext()
- pypvm.getcontext() returns the current context of the requesting task.
-
pyopus.parallel.pypvm.getinst()
- pypvm.getinst(group, tid) returns the instance number in group of a
PVM process identified by tid
-
pyopus.parallel.pypvm.getopt()
pypvm.getopt(what) returns the value of the specified option in PVM.
For a discussion of options and values, see the pvm_setopt man page.
The permissible values of what are in the pypvm.opt dictionary:
- pypvm.opt['Route'] ... 1 ... Message routing policy
- pypvm.opt['DebugMask'] ... 2 ... Libpvm debug mask
- pypvm.opt['AutoErr'] ... 3 ... Auto error reporting
- pypvm.opt['OutputTid'] ... 4 ... Stdout destination for children
- pypvm.opt['OutputCode'] ... 5 ... Output message tag
- pypvm.opt['TraceTid'] ... 6 ... Trace data destination for children
- pypvm.opt['TraceCode'] ... 7 ... Trace message tag
- pypvm.opt['FragSize'] ... 8 ... Message fragment size
- pypvm.opt['ResvTids'] ... 9 ... Allow messages to reserved tags and TIDs
- pypvm.opt['SelfOutputTid'] ... 10 ... Stdout destination
- pypvm.opt['SelfOutputCode'] ... 11 ... Output message tag
- pypvm.opt['SelfTraceTid'] ... 12 ... Trace data destination
- pypvm.opt['SelfTraceCode'] ... 13 ... Trace message tag
- pypvm.opt['ShowTids'] ... 14 ... pvm_catchout prints task ids with output
- pypvm.opt['PollType'] ... 15 ... Message wait policy (shared memory)
- pypvm.opt['PollTime'] ... 16 ... Message spinwait duration
- Example:
- pypvm.getopt(pypvm.opt[‘FragSize’])
-
pyopus.parallel.pypvm.getrbuf()
- pypvm.getrbuf() returns the message buffer identifier for the active receive buffer.
-
pyopus.parallel.pypvm.getsbuf()
- pypvm.getsbuf() returns the message buffer identifier for the active send buffer.
-
pyopus.parallel.pypvm.gettid()
- pypvm.gettid(group, instance) returns the tid of the process identified by
group and instance.
-
pyopus.parallel.pypvm.gsize()
- pypvm.gsize(group) returns the number of members presently in group.
-
pyopus.parallel.pypvm.halt()
- pypvm.halt() shuts down the entire PVM system.
-
pyopus.parallel.pypvm.hostinfo()
Returns a list (one tuple for each host in the virtual machine) - each tuple
is (task id of pvm daemon, hostname, architecture, speed).
This is a subset of the information that can be retrieved with pypvm.config()
-
pyopus.parallel.pypvm.initsend()
- pypvm.initsend(encoding) / pypvm.initsend() clears the default send buffer and
specify message encoding to be encoding/default. Valid encoding values are
in pypvm.data and can be either pypvm.data['default'], pypvm.data['raw'] or
pypvm.data['inplace']. I don’t recommend using inplace with this version of PyPVM.
-
pyopus.parallel.pypvm.insert()
- pypvm.insert(name, index, data) stores data in the simple database maintained by
the master pvmd , which can be used to store values such as tids and make them accessible
anywhere within a virtual machine. This is useful when building an application
such as the group server, which must advertise its task id so clients can register
send messages to register. data (an integer) gets stoed as the entry <name, index>.
-
pyopus.parallel.pypvm.joingroup()
- pypvm.joingroup(group) enrolls the calling process in group
-
pyopus.parallel.pypvm.kill()
- pypvm.kill(tid) sends a terminate (SIGTERM) signal to the PVM process tid.
-
pyopus.parallel.pypvm.lookup()
pypvm.lookup(name, index) retrieves data stored in the location given by
<name,*index*>. If index is -1, the data stored at the first
existing index in the named class is returned.
See pvm_insert(3PVM) (or pypvm.insert.__doc__) for a description of this database.
-
pyopus.parallel.pypvm.lvgroup()
- pypvm.lvgroup(group) unenrolls the calling process from a named group.
-
pyopus.parallel.pypvm.mcast()
- pypvm.mcast(tids, msgtag) multicasts a message stored in the active send
buffer to the tasks specified in the tids list. The message is not
sent to the caller even if listed in tids. The content of the
message can be distinguished by msgtag.
-
pyopus.parallel.pypvm.mkbuf()
- pypvm.mkbuf(encoding) creates a new message buffer. Valid encoding values are
in pypvm.data and can be either default, raw or inplace. I don’t recommend
using inplace with this version of PyPVM. The encoding argument is optional.
-
pyopus.parallel.pypvm.mstat()
pypvm.mstat(host) returns the status of a host in the virtual machine.
value |
MEANING |
PvmOk |
host is OK |
PvmNoHost |
host is not in virtual machine |
PvmHostFail |
host is unreachable (and thus possibly failed) |
-
pyopus.parallel.pypvm.mytid()
- pypvm.mytid() returns the tid of the calling process.
-
pyopus.parallel.pypvm.narch()
- pypvm.narch() returns a count of the number of different data formats
being used (narch = number of architectures). This is a subset of
the information returned by pypvm.config()
-
pyopus.parallel.pypvm.newcontext()
- pypvm.newcontext() returns a newly allocated context. However, this new
context is not yet active. See pypvm.setcontext.
-
pyopus.parallel.pypvm.notify()
pypvm.notify(what, msgtag, tids, count) requests PVM to notify the caller on
detecting certain events. One or more notify messages (see below) are
sent by PVM back to the calling task. The messages have tag msgtag
supplied to notify.
what is the type of event to trigger the notification.
Presently one of:
- pypvm.notifyDict[‘TaskExit’] => Task exits or is killed
- pypvm.notifyDict[‘HostDelete’] => Host is deleted or crashes
- pypvm.notifyDict[‘HostAdd’] => New host is added
msgtag Message tag to be used in notification.
tids For TaskExit and HostDelete, list of task of pvmd TIDs
to be notified about. It is not used when what is HostAdd.
count for HostAdd, it determines how many messages will be sent.
It is not used for TaskExit or HostDelete.
The notification messages have the following format:
- pypvm.notifyDict[‘TaskExit’]
One notify message for each TID requested. The
message body contains a single TID of exited task.
- pypvm.notifyDict[‘HostDelete’]
One notify message for each TID requested. The message body
contains a single pvmd-TID of exited pvmd.
- pypvm.notifyDict[‘HostAdd’]
count notify messages are sent, one each time the local pvmd’s host
table is updated. The message body contains an integer length
followed by a list of pvmd-TIDs of new pvmds. The counter of HostAdd
messages yet to be sent is replaced by successive calls to pvm_notify.
Specifying a count of -1 turns on PvmHostAdd messages until a future
notify; a count of zero disables them. count defaults to 0.
TIDs in the notify messages are packed as integers. The calling task
is responsible for receiving messages with the specified tag and
taking appropriate action. Future versions of PVM may expand the list
of available notification events.
-
pyopus.parallel.pypvm.nrecv()
pypvm.nrecv(tid, msgtag) is a non-blocking receive.
The routine pypvm.nrecv checks to see if a message with
label msgtag has arrived from tid and also clears the
current receive buffer if any, If a matching message has
arrived pypvm.nrecv immediately places the message in a new
active receive buffer, and returns the buffer identifier
in bufid.
If the requested message has not arrived, then pypvm.nrecv
immediately returns with a 0 in bufid.
A -1 or a missing in msgtag or tid matches anything.
-
pyopus.parallel.pypvm.pack_by_type()
- pypvm.pack_by_type(arg) packs the active message buffer with the
contents of arg. arg will probably be a list, although it can be
a single value. At the moment, only ints, floats and strings can be
packed (they get packed with pvm_pkint, pvm_pkdbl and pvm_pkstr
respectively - the programmer has no control over this... future
versions will see improvements)
-
pyopus.parallel.pypvm.parent()
- pypvm.parent() returns the tid of the process that spawned the calling process.
-
pyopus.parallel.pypvm.perror()
- pvm_perror(msg) returns the error message of the last PVM call. The user can
(optionally) use msg to add additional information to the error message,
for example, its location. I can’t imagine this function being used all that
often.
-
pyopus.parallel.pypvm.pkbyte()
The family of functions pypvm.pk*(list, stride) take a list
and sends them as an array of whatever type is given in the argument
(byte, short, int, long, float, or double). stride defaults to
1, and is the separation to use.
One day in the future (not now) these functions will have one-argument
forms so that if the argument given is not a list, that it will pack
it as if it were a one-element list.
-
pyopus.parallel.pypvm.pkbytestr()
- pypvm.pkbytestr(str) packs the string str.
The string can contain null characters, because it
gets packed as a sequence of bytes.
-
pyopus.parallel.pypvm.pkdouble()
The family of functions pypvm.pk*(list, stride) take a list
and sends them as an array of whatever type is given in the argument
(byte, short, int, long, float, or double). stride defaults to
1, and is the separation to use.
One day in the future (not now) these functions will have one-argument
forms so that if the argument given is not a list, that it will pack
it as if it were a one-element list.
-
pyopus.parallel.pypvm.pkfloat()
The family of functions pypvm.pk*(list, stride) take a list
and sends them as an array of whatever type is given in the argument
(byte, short, int, long, float, or double). stride defaults to
1, and is the separation to use.
One day in the future (not now) these functions will have one-argument
forms so that if the argument given is not a list, that it will pack
it as if it were a one-element list.
-
pyopus.parallel.pypvm.pkint()
The family of functions pypvm.pk*(list, stride) take a list
and sends them as an array of whatever type is given in the argument
(byte, short, int, long, float, or double). stride defaults to
1, and is the separation to use.
One day in the future (not now) these functions will have one-argument
forms so that if the argument given is not a list, that it will pack
it as if it were a one-element list.
-
pyopus.parallel.pypvm.pklong()
The family of functions pypvm.pk*(list, stride) take a list
and sends them as an array of whatever type is given in the argument
(byte, short, int, long, float, or double). stride defaults to
1, and is the separation to use.
One day in the future (not now) these functions will have one-argument
forms so that if the argument given is not a list, that it will pack
it as if it were a one-element list.
-
pyopus.parallel.pypvm.pkshort()
The family of functions pypvm.pk*(list, stride) take a list
and sends them as an array of whatever type is given in the argument
(byte, short, int, long, float, or double). stride defaults to
1, and is the separation to use.
One day in the future (not now) these functions will have one-argument
forms so that if the argument given is not a list, that it will pack
it as if it were a one-element list.
-
pyopus.parallel.pypvm.pkstr()
- pypvm.pkstr(str) packs the string [str]
-
pyopus.parallel.pypvm.probe()
- pypvm.probe(tid, msgtag) checks to see if a message with label msgtag has
arrived from tid. If a matching message has arrived pypvm.probe
returns a buffer identifier. This bufid can be used in a pypvm.bufinfo
call to determine information about the message such as its source
and length.
-
pyopus.parallel.pypvm.pstat()
- pypvm.pstat(tid) returns the status of the process identified by tid.
Also note that pypvm.notify() can be used to notify the caller that a task
has failed.
-
pyopus.parallel.pypvm.recv()
- pypvm.recv(tid, msgtag) blocks the process until a message with label
msgtag has arrived from tid. pypvm.recv() then places the message
in a new active receive buffer, which also clears the current receive buffer.
tid and msgtag can be -1, meaning “match anything”, which is the default
if either is not mentioned.
-
pyopus.parallel.pypvm.send()
- pypvm.send(tid, msgtag) sends a message stored in the active send buffer
to the PVM process identified by tid. msgtag is used to label the
content of the message.
-
pyopus.parallel.pypvm.sendsig()
- pypvm.sendsig(tid, signum) sends the signal number signum to the PVM
process identified by tid.
-
pyopus.parallel.pypvm.setcontext()
- pypvm.setcontext(newctx) changes the current context from old_ctx (which is
returned to newctx.
-
pyopus.parallel.pypvm.setopt()
pypvm.setopt(what, value) is a general purpose function used
to set miscellaneous options in the PVM library. The previous
value is returned.
what is an integer defining what to set. One of:
- pypvm.opt['Route'] ... 1 ... Message routing policy
- pypvm.opt['DebugMask'] ... 2 ... Libpvm debug mask
- pypvm.opt['AutoErr'] ... 3 ... Auto error reporting
- pypvm.opt['OutputTid'] ... 4 ... Stdout destination for children
- pypvm.opt['OutputCode'] ... 5 ... Output message tag
- pypvm.opt['TraceTid'] ... 6 ... Trace data destination for children
- pypvm.opt['TraceCode'] ... 7 ... Trace message tag
- pypvm.opt['FragSize'] ... 8 ... Message fragment size
- pypvm.opt['ResvTids'] ... 9 ... Allow messages to reserved tags and TIDs
- pypvm.opt['SelfOutputTid'] ... 10 ... Stdout destination
- pypvm.opt['SelfOutputCode'] ... 11 ... Output message tag
- pypvm.opt['SelfTraceTid'] ... 12 ... Trace data destination
- pypvm.opt['SelfTraceCode'] ... 13 ... Trace message tag
- pypvm.opt['ShowTids'] ... 14 ... pvm_catchout prints task ids with output
- pypvm.opt['PollType'] ... 15 ... Message wait policy (shared memory)
- pypvm.opt['PollTime'] ... 16 ... Message spinwait duration
val Integer specifying new setting of option. Usually this is just any
old number, but there are some predefined values for pypvm.opt['Route']:
- pypvm.opt['DontRoute'] ... 1 ... Don’t request or grant connections
- pypvm.opt['AllowDirect'] ... 2 ... (Default) Don’t request but allow
- pypvm.opt['RouteDirect'] ... 3 ... Request and allow connections
-
pyopus.parallel.pypvm.setrbuf()
- pypvm.setrbuf(bufid) switches the active receive buffer to bufid and returns
the previous buffer id.
-
pyopus.parallel.pypvm.setsbuf()
- pypvm.setsbuf(bufid) switches the active send buffer to bufid and returns
the previous buffer id.
-
pyopus.parallel.pypvm.spawn()
- pypvm.spawn(task, argv, flag, where, ntasks) starts ntask copies of the
executable named task. The tids are returned in a list.
-
pyopus.parallel.pypvm.tasks()
Returns a list (one tuple for each task) of all tasks running in the
virtual machine. Each tuple is (task id, parent task id, task id of
the pvm daemon on the tasks’ machine, flags it was called with,
the name of the task, and the process id of the task).
If called with a tid as an argument, it will just return that tasks
info; if called with the tid of a pvm daemon, it will return all
the tasks on that host.
-
pyopus.parallel.pypvm.tidtohost()
- pypvm.tidtohost(tid) returns the tid of the daemon process on the host running
the process identified by tid is located.
-
pyopus.parallel.pypvm.trecv()
pypvm.trecv(timeout, msgtag, tid) blocks the process until a message with
label msgtag has arrived from tid. pypvm.trecv then places the
message in a new active receive buffer, also clearing the current
receive buffer. If no matching message arrives within the specified
waiting time timeout (in seconds), pypvm.trecv returns without a
message.
As usual tid and msgtag can be -1, which is the default if either
are missing.
pypvm.trecv returns the bufid of the new active receive buffer.
Note that the order of arguments here is different to the pvm_trecv call,
which puts the arguments in the opposite order.
-
pyopus.parallel.pypvm.unexport()
- pypvm.unexport(name) is provided for convenience in editing the environment
variable PVM_EXPORT, while maintaining the colon-separated list syntax it
requires.
-
pyopus.parallel.pypvm.upkbyte()
- Each of the pypvm.upk*(nitem, stride) routines unpacks an array of the
given data type from the active receive buffer. The arguments for each
of the routines are a pointer to the array to be unpacked into,
nitem which is the total number of items to unpack, and the (optional)
stride which is the stride to use when unpacking.
-
pyopus.parallel.pypvm.upkbytestr()
- pypvm.upkbytestr() unpacks a byte string from the active receive buffer
NULL characters are allowed in the string.
-
pyopus.parallel.pypvm.upkdouble()
- Each of the pypvm.upk*(nitem, stride) routines unpacks an array of the
given data type from the active receive buffer. The arguments for each
of the routines are a pointer to the array to be unpacked into,
nitem which is the total number of items to unpack, and the (optional)
stride which is the stride to use when unpacking.
-
pyopus.parallel.pypvm.upkfloat()
- Each of the pypvm.upk*(nitem, stride) routines unpacks an array of the
given data type from the active receive buffer. The arguments for each
of the routines are a pointer to the array to be unpacked into,
nitem which is the total number of items to unpack, and the (optional)
stride which is the stride to use when unpacking.
-
pyopus.parallel.pypvm.upkint()
- Each of the pypvm.upk*(nitem, stride) routines unpacks an array of the
given data type from the active receive buffer. The arguments for each
of the routines are a pointer to the array to be unpacked into,
nitem which is the total number of items to unpack, and the (optional)
stride which is the stride to use when unpacking.
-
pyopus.parallel.pypvm.upklong()
- Each of the pypvm.upk*(nitem, stride) routines unpacks an array of the
given data type from the active receive buffer. The arguments for each
of the routines are a pointer to the array to be unpacked into,
nitem which is the total number of items to unpack, and the (optional)
stride which is the stride to use when unpacking.
-
pyopus.parallel.pypvm.upkshort()
- Each of the pypvm.upk*(nitem, stride) routines unpacks an array of the
given data type from the active receive buffer. The arguments for each
of the routines are a pointer to the array to be unpacked into,
nitem which is the total number of items to unpack, and the (optional)
stride which is the stride to use when unpacking.
-
pyopus.parallel.pypvm.upkstr()
- pypvm.upkstr() unpacks a string from the active receive buffer
-
pyopus.parallel.pypvm.version()
- Returns the version string of the PVM this was built against.