Helmholtz machine
The Helmholtz machine (named after Hermann von Helmholtz and his concept of Helmholtz free energy) is a type of artificial neural network that can account for the hidden structure of a set of data by being trained to create a generative model of the original data.[1][2] The hope is that by learning economical representations of the data, the underlying structure of the generative model should reasonably approximate the hidden structure of the data set.
A Helmholtz machine contains two networks, a bottom-up recognition network that takes the data as input and produces a distribution over hidden variables, and a top-down generative network that generates values of the hidden variables and the data itself. Helmholtz machines are a type of learning architectures using both feedback and feedforward to ensure quality of learned models.[3]
Helmholtz machines are usually trained using an unsupervised learning algorithm, such as the wake-sleep algorithm.[4] They are a precursor to variational autoencoders, which are instead trained using backpropagation. Helmholtz machines may also be used in applications requiring a supervised learning algorithm (e.g. character recognition, or position-invariant recognition of an object within a field).
See also
[edit]References
[edit]- ↑ Peter, Dayan; Hinton, Geoffrey E.; Neal, Radford M.; Zemel, Richard S. (1995). "The Helmholtz machine". Neural Computation. 7 (5): 889–904. doi:10.1162/neco.1995.7.5.889. hdl:21.11116/0000-0002-D6D3-E. PMID 7584891. S2CID 1890561.
- ↑ Hinton, Geoffrey E; Zemel, Richard (1993). "Autoencoders, Minimum Description Length and Helmholtz Free Energy". Advances in Neural Information Processing Systems. 6. Morgan-Kaufmann.
- ↑ Luttrell S.P. (1994). A Bayesian analysis of self-organizing maps. Neural Computation. 1994 Sep 1;6(5):767-94.
- ↑ Hinton, Geoffrey E.; Dayan, Peter; Frey, Brendan J.; Neal, Radford (1995-05-26). "The wake-sleep algorithm for unsupervised neural networks". Science. 268 (5214): 1158–1161. Bibcode:1995Sci...268.1158H. doi:10.1126/science.7761831. PMID 7761831. S2CID 871473.
External links
[edit]- http://www.cs.utoronto.ca/~hinton/helmholtz.html — Hinton's papers on Helmholtz machines
- https://www.nku.edu/~kirby/docs/HelmholtzTutorialKoeln.pdf - A tutorial on Helmholtz machines
 | This artificial neural network-related article is a stub. You can help Wikipedia by adding missing information. |