Package mdp :: Package parallel
[hide private]
[frames] | no frames]

Package parallel


This is the MDP package for parallel processing.

It is designed to work with nodes for which a large part of the computation
is embaressingly parallel (like in PCANode). The hinet package is also fully
supported, i.e., there are parallel versions of all hinet nodes.

This package consists of two decoupled parts. The first part consists of
parallel versions of the familiar MDP structures (nodes and flows). At the top
there is the ParallelFlow, which generates tasks that are processed in 
parallel (this can be done automatically in the train or execute methods).

The second part consists of the schedulers. They take tasks and process them in 
a more or less parallel way (e.g. in multiple processes). So they are designed
to deal with the more technical aspects of the parallelization, but do not
have to know anything about flows or nodes.

Classes [hide private]
  FlowExecuteCallable
Implements data execution through the whole flow.
  FlowTrainCallable
Implements a single training phase in a flow for a data block.
  ListResultContainer
Basic result container using simply a list.
  NoTaskException
Exception for problems with the task creation.
  NodeResultContainer
Container for parallel nodes.
  OrderedResultContainer
Default result container with automatic restoring of the result order.
  ParallelCheckpointFlow
Parallel version of CheckpointFlow.
  ParallelExtensionNode
Base class for parallel trainable MDP nodes.
  ParallelFlow
A parallel flow provides the methods for parallel training / execution.
  ParallelFlowException
Standard exception for problems with ParallelFlow.
  ProcessScheduler
Scheduler that distributes the task to multiple processes.
  ResultContainer
Abstract base class for result containers.
  Scheduler
Base class and trivial implementation for schedulers.
  SleepSqrTestCallable
Callable for testing.
  SqrTestCallable
Callable for testing.
  TaskCallable
Abstract base class for callables.
  TaskCallableWrapper
Wrapper to provide a fork method for simple callables like a function.
  TrainingPhaseNotParallelException
Exception for parallel nodes that do not support fork() in some phases.