Package mdp :: Package parallel :: Class ParallelCheckpointFlow
[hide private]
[frames] | no frames]

Class ParallelCheckpointFlow


Parallel version of CheckpointFlow.

Note that train phases are always closed, so e.g. CheckpointSaveFunction
should not expect open train phases. This is necessary since otherwise
stop_training() would be called remotely.

Instance Methods [hide private]
 
__init__(self, flow, verbose=False, **kwargs)
Initialize the internal variables.
 
setup_parallel_training(self, data_iterables, checkpoints, train_callable_class=<class 'mdp.parallel.parallelflows.FlowTrainCallable'>, **kwargs)
Checkpoint version of parallel training.
 
train(self, data_iterables, checkpoints, scheduler=None, train_callable_class=<class 'mdp.parallel.parallelflows.FlowTrainCallable'>, overwrite_result_container=True, **kwargs)
Train all trainable nodes in the flow.
 
use_results(self, results)
Checkpoint version of use_results.

Inherited from object: __delattr__, __getattribute__, __hash__, __new__, __reduce__, __reduce_ex__, __setattr__

    Inherited from ParallelFlow
 
_create_execute_task(self)
Create and return a single execution task.
 
_create_train_task(self)
Create and return a single training task without callable.
 
_next_train_phase(self)
Find the next phase or node for parallel training.
 
execute(self, iterable, nodenr=None, scheduler=None, execute_callable_class=None, overwrite_result_container=True)
Train all trainable nodes in the flow.
 
get_task(self)
Return a task either for either training or execution.
 
setup_parallel_execution(self, iterable, nodenr=None, execute_callable_class=<class 'mdp.parallel.parallelflows.FlowExecuteCallable'>)
Prepare the flow for handing out tasks to do the execution.
    Inherited from CheckpointFlow
 
_train_check_checkpoints(self, checkpoints)
    Inherited from Flow
 
__add__(self, other)
 
__call__(self, iterable, nodenr=None)
Calling an instance is equivalent to call its 'execute' method.
 
__contains__(self, item)
 
__delitem__(self, key)
 
__getitem__(self, key)
 
__iter__(self)
 
__len__(self)
 
__repr__(self)
repr(x)
 
__setitem__(self, key, value)
 
__str__(self)
str(x)
 
_check_dimension_consistency(self, out, inp)
Raise ValueError when both dimensions are set and different.
 
_check_nodes_consistency(self, flow=None)
Check the dimension consistency of a list of nodes.
 
_check_value_type_isnode(self, value)
 
_close_last_node(self)
 
_execute_seq(self, x, nodenr=None)
 
_inverse_seq(self, x)
 
_propagate_exception(self, except_, nodenr)
 
_stop_training_hook(self)
Hook method that is called before stop_training is called.
 
_train_check_iterables(self, data_iterables)
Return the data iterables after some checks and sanitizing.
 
_train_node(self, data_iterable, nodenr)
Train a single node in the flow.
 
append(flow, node)
append node to flow end
 
copy(self, protocol=-1)
Return a deep copy of the flow.
 
extend(flow, iterable)
extend flow by appending...
 
insert(flow, index, node)
insert node before index
 
inverse(self, iterable)
Process the data through all nodes in the flow backwards (starting from the last node up to the first node) by calling the inverse function of each node.
node

pop(flow, index=...)
remove and return node at index...
 
save(self, filename, protocol=-1)
Save a pickled serialization of the flow to 'filename'.
 
set_crash_recovery(self, state=True)
Set crash recovery capabilities.
Properties [hide private]

Inherited from object: __class__

    Inherited from ParallelFlow
  is_parallel_executing
Return True if parallel execution is underway.
  is_parallel_training
Return True if parallel training is underway.
  task_available
Return True if tasks are available, otherwise False.
Method Details [hide private]

__init__(self, flow, verbose=False, **kwargs)
(Constructor)

 
Initialize the internal variables.

Overrides: object.__init__

setup_parallel_training(self, data_iterables, checkpoints, train_callable_class=<class 'mdp.parallel.parallelflows.FlowTrainCallable'>, **kwargs)

 
Checkpoint version of parallel training.

Overrides: ParallelFlow.setup_parallel_training

train(self, data_iterables, checkpoints, scheduler=None, train_callable_class=<class 'mdp.parallel.parallelflows.FlowTrainCallable'>, overwrite_result_container=True, **kwargs)

 
Train all trainable nodes in the flow.

Same as the train method in ParallelFlow, but with additional support
of checkpoint functions as in CheckpointFlow.

Overrides: Flow.train

use_results(self, results)

 
Checkpoint version of use_results.

Calls the checkpoint functions when necessary.

Overrides: ParallelFlow.use_results