Package mdp :: Package nodes :: Class RBMWithLabelsNode
[hide private]
[frames] | no frames]

Class RBMWithLabelsNode


Restricted Boltzmann Machine with softmax labels. An RBM is an
undirected probabilistic network with binary variables. In this
case, the node is partitioned into a set of observed ('visible')
variables, a set of hidden ('latent') variables, and a set of
label variables (also observed), only one of which is active at
any time. The node is able to learn associations between the
visible variables and the labels.

By default, the 'execute' function returns the *probability* of
one of the hiden variables being equal to 1 given the input.

Use the 'sample_v' function to sample from the observed variables
(visible and labels) given a setting of the hidden variables, and
'sample_h' to do the opposite. The 'energy' function can be used
to compute the energy of a given setting of all variables.

The network is trained by Contrastive Divergence, as described in
Hinton, G. E. (2002). Training products of experts by minimizing
contrastive divergence. Neural Computation, 14(8):1711-1800

Internal variables of interest:
self.w -- generative weights between hidden and observed variables
self.bv -- bias vector of the observed variables
self.bh -- bias vector of the hidden variables

For more information on RBMs with labels, see

Geoffrey E. Hinton (2007) Boltzmann machine. Scholarpedia, 2(5):1668

Hinton, G. E, Osindero, S., and Teh, Y. W. (2006). A fast learning
algorithm for deep belief nets. Neural Computation, 18:1527-1554. 

Nested Classes [hide private]
    Inherited from Node
  __metaclass__
This Metaclass is meant to overwrite doc strings of methods like execute, stop_training, inverse with the ones defined in the corresponding private methods _execute, _stop_training, _inverse, etc...
Instance Methods [hide private]
 
__init__(self, hidden_dim, labels_dim, visible_dim=None, dtype=None)
Arguments:
 
_get_supported_dtypes(self)
Return the list of dtypes supported by this node.
 
_sample_v(self, h, sample_l=False, concatenate=True)
 
_set_input_dim(self, n)
 
energy(self, v, h, l)
Compute the energy of the RBM given observed variables state 'v' and 'l', and hidden variables state 'h'.
 
execute(self, v, l, return_probs=True)
If 'return_probs' is True, returns the probability of the hidden variables h[n,i] being 1 given the observations v[n,:] and l[n,:].
 
is_invertible(self)
Return True if the node can be inverted, False otherwise.
 
sample_h(self, v, l)
Sample the hidden variables given observations v and labels l.
 
sample_v(self, h)
Sample the observed variables given hidden variable state h.
 
train(self, v, l, n_updates=1, epsilon=0.1, decay=0.0, momentum=0.0, verbose=False)
Update the internal structures according to the visible data 'v' and the labels 'l'.

Inherited from object: __delattr__, __getattribute__, __hash__, __new__, __reduce__, __reduce_ex__, __setattr__

    Inherited from RBMNode
 
_energy(self, v, h)
 
_execute(self, v, return_probs=True)
If 'return_probs' is True, returns the probability of the hidden variables h[n,i] being 1 given the observations v[n,:].
 
_init_weights(self)
 
_pre_inversion_checks(self, y)
This method contains all pre-inversion checks.
 
_sample_h(self, v)
 
_stop_training(self)
 
_train(self, v, n_updates=1, epsilon=0.1, decay=0.0, momentum=0.0, verbose=False)
Update the internal structures according to the input data 'v'.
    Inherited from Node
 
__add__(self, other)
 
__call__(self, x, *args, **kargs)
Calling an instance of Node is equivalent to call its 'execute' method.
 
__repr__(self)
repr(x)
 
__str__(self)
str(x)
 
_check_input(self, x)
 
_check_output(self, y)
 
_check_train_args(self, x, *args, **kwargs)
 
_get_train_seq(self)
 
_if_training_stop_training(self)
 
_inverse(self, x)
 
_pre_execution_checks(self, x)
This method contains all pre-execution checks.
 
_refcast(self, x)
Helper function to cast arrays to the internal dtype.
 
_set_dtype(self, t)
 
_set_output_dim(self, n)
 
copy(self, protocol=-1)
Return a deep copy of the node.
 
get_current_train_phase(self)
Return the index of the current training phase.
 
get_dtype(self)
Return dtype.
 
get_input_dim(self)
Return input dimensions.
 
get_output_dim(self)
Return output dimensions.
 
get_remaining_train_phase(self)
Return the number of training phases still to accomplish.
 
get_supported_dtypes(self)
Return dtypes supported by the node as a list of numpy.dtype objects.
 
inverse(self, y, *args, **kargs)
Invert 'y'.
 
is_trainable(self)
Return True if the node can be trained, False otherwise.
 
is_training(self)
Return True if the node is in the training phase, False otherwise.
 
save(self, filename, protocol=-1)
Save a pickled serialization of the node to 'filename'.
 
set_dtype(self, t)
Set internal structures' dtype.
 
set_input_dim(self, n)
Set input dimensions.
 
set_output_dim(self, n)
Set output dimensions.
 
stop_training(self, *args, **kwargs)
Stop the training phase.
Properties [hide private]

Inherited from object: __class__

    Inherited from Node
  _train_seq
List of tuples: [(training-phase1, stop-training-phase1), (training-phase2, stop_training-phase2), ...
  dtype
dtype
  input_dim
Input dimensions
  output_dim
Output dimensions
  supported_dtypes
Supported dtypes
Method Details [hide private]

__init__(self, hidden_dim, labels_dim, visible_dim=None, dtype=None)
(Constructor)

 

Arguments:

hidden_dim -- number of hidden variables
visible_dim -- number of observed variables

Overrides: object.__init__
(inherited documentation)

_get_supported_dtypes(self)

 
Return the list of dtypes supported by this node.

Overrides: Node._get_supported_dtypes
(inherited documentation)

_sample_v(self, h, sample_l=False, concatenate=True)

 
Overrides: RBMNode._sample_v

_set_input_dim(self, n)

 
Overrides: Node._set_input_dim

energy(self, v, h, l)

 
Compute the energy of the RBM given observed variables state 'v'
and 'l', and hidden variables state 'h'.

Overrides: RBMNode.energy

execute(self, v, l, return_probs=True)

 
If 'return_probs' is True, returns the probability of the
hidden variables h[n,i] being 1 given the observations v[n,:]
and l[n,:].  If 'return_probs' is False, return a sample from
that probability.

Overrides: Node.execute

is_invertible(self)

 
Return True if the node can be inverted, False otherwise.

Overrides: Node.is_invertible
(inherited documentation)

sample_h(self, v, l)

 
Sample the hidden variables given observations v and labels l.

Returns a tuple (prob_h, h), where prob_h[n,i] is the
probability that variable 'i' is one given the observations
v[n,:] and the labels l[n,:],and h[n,i] is a sample from
the posterior probability.

Overrides: RBMNode.sample_h

sample_v(self, h)

 
Sample the observed variables given hidden variable state h.

Returns a tuple (prob_v, probs_l, v, l), where prob_v[n,i] is
the probability that the visible variable 'i' is one given the
hidden variables h[n,:], and v[n,i] is a sample from that
conditional probability. prob_l and l have similar
interpretations for the label variables. Note that the labels
are activated using a softmax function, so that only one label
can be active at any time.

Overrides: RBMNode.sample_v

train(self, v, l, n_updates=1, epsilon=0.1, decay=0.0, momentum=0.0, verbose=False)

 
Update the internal structures according to the visible data 'v'
and the labels 'l'.
The training is performed using Contrastive Divergence (CD).

v -- a binary matrix having different variables on different columns
     and observations on the rows
l -- a binary matrix having different variables on different columns
     and observations on the rows. Only one value per row should be 1.
n_updates -- number of CD iterations. Default value: 1
epsilon -- learning rate. Default value: 0.1
decay -- weight decay term. Default value: 0.
momentum -- momentum term. Default value: 0.

Overrides: Node.train