Home | Trees | Indices | Help |
|
---|
|
Restricted Boltzmann Machine with softmax labels. An RBM is an undirected probabilistic network with binary variables. In this case, the node is partitioned into a set of observed ('visible') variables, a set of hidden ('latent') variables, and a set of label variables (also observed), only one of which is active at any time. The node is able to learn associations between the visible variables and the labels. By default, the 'execute' function returns the *probability* of one of the hiden variables being equal to 1 given the input. Use the 'sample_v' function to sample from the observed variables (visible and labels) given a setting of the hidden variables, and 'sample_h' to do the opposite. The 'energy' function can be used to compute the energy of a given setting of all variables. The network is trained by Contrastive Divergence, as described in Hinton, G. E. (2002). Training products of experts by minimizing contrastive divergence. Neural Computation, 14(8):1711-1800 Internal variables of interest: self.w -- generative weights between hidden and observed variables self.bv -- bias vector of the observed variables self.bh -- bias vector of the hidden variables For more information on RBMs with labels, see Geoffrey E. Hinton (2007) Boltzmann machine. Scholarpedia, 2(5):1668 Hinton, G. E, Osindero, S., and Teh, Y. W. (2006). A fast learning algorithm for deep belief nets. Neural Computation, 18:1527-1554.
|
|||
Inherited from Node | |||
---|---|---|---|
__metaclass__ This Metaclass is meant to overwrite doc strings of methods like execute, stop_training, inverse with the ones defined in the corresponding private methods _execute, _stop_training, _inverse, etc... |
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
Inherited from |
|||
Inherited from RBMNode | |||
---|---|---|---|
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
Inherited from Node | |||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|
|||
Inherited from |
|||
Inherited from Node | |||
---|---|---|---|
_train_seq List of tuples: [(training-phase1, stop-training-phase1), (training-phase2, stop_training-phase2), ... |
|||
dtype dtype |
|||
input_dim Input dimensions |
|||
output_dim Output dimensions |
|||
supported_dtypes Supported dtypes |
|
Arguments: hidden_dim -- number of hidden variables visible_dim -- number of observed variables
|
Return the list of dtypes supported by this node.
|
|
|
Compute the energy of the RBM given observed variables state 'v' and 'l', and hidden variables state 'h'.
|
If 'return_probs' is True, returns the probability of the hidden variables h[n,i] being 1 given the observations v[n,:] and l[n,:]. If 'return_probs' is False, return a sample from that probability.
|
Return True if the node can be inverted, False otherwise.
|
Sample the hidden variables given observations v and labels l. Returns a tuple (prob_h, h), where prob_h[n,i] is the probability that variable 'i' is one given the observations v[n,:] and the labels l[n,:],and h[n,i] is a sample from the posterior probability.
|
Sample the observed variables given hidden variable state h. Returns a tuple (prob_v, probs_l, v, l), where prob_v[n,i] is the probability that the visible variable 'i' is one given the hidden variables h[n,:], and v[n,i] is a sample from that conditional probability. prob_l and l have similar interpretations for the label variables. Note that the labels are activated using a softmax function, so that only one label can be active at any time.
|
Update the internal structures according to the visible data 'v' and the labels 'l'. The training is performed using Contrastive Divergence (CD). v -- a binary matrix having different variables on different columns and observations on the rows l -- a binary matrix having different variables on different columns and observations on the rows. Only one value per row should be 1. n_updates -- number of CD iterations. Default value: 1 epsilon -- learning rate. Default value: 0.1 decay -- weight decay term. Default value: 0. momentum -- momentum term. Default value: 0.
|
Home | Trees | Indices | Help |
|
---|
Generated by Epydoc 3.0.1 on Fri Oct 9 06:08:53 2009 | http://epydoc.sourceforge.net |