COVID-19 Information
Public health information (CDC)
Research information (NIH)
SARS-CoV-2 data (NCBI)
Prevention and treatment information (HHS)
Log in

Modeling language and cognition with deep unsupervised learning: a tutorial overview
Marco Zorzi et al. Front Psychol. 2013.
Free PMC article
Deep unsupervised learning in stochastic recurrent neural networks with many layers of hidden units is a recent breakthrough in neural computation research. These networks build a hierarchy of progressively more complex distributed representations of the sensory data by fitting a hierarchical generative model. In this article we discuss the theoretical foundations of this approach and we review key issues related to training, testing and analysis of deep networks for modeling language and cognitive processing. The classic letter and word perception problem of McClelland and Rumelhart (1981) is used as a tutorial example to illustrate how structured and abstract representations may emerge from deep generative learning. We argue that the focus on deep architectures and generative (rather than discriminative) learning represents a crucial step forward for the connectionist modeling enterprise, because it offers a more plausible model of cortical learning as well as a way to bridge the gap between emergentist connectionist models and structured Bayesian models of cognition.
Keywords: connectionist modeling; deep learning; hierarchical generative models; neural networks; unsupervised learning; visual word recognition.
Figure 1 (A) A directed graphical model,…
Figure 2 Graphical representation of a Restricted…
Figure 3 (A) Architecture of the DBN…
Figure 4 Architecture of the DBN with…
Figure 5 Mean accuracy of the linear…
Figure 6 Visualization of features learned at…
Similar articles
Probabilistic Models and Generative Neural Networks: Towards an Unified Framework for Modeling Normal and Impaired Neurocognitive Functions.
Testolin A, et al. Front Comput Neurosci. 2016.PMID: 27468262 Free PMC article.
Deep Unsupervised Learning on a Desktop PC: A Primer for Cognitive Scientists.
Testolin A, et al. Front Psychol. 2013.PMID: 23653617 Free PMC article.
Learning Orthographic Structure With Sequential Generative Neural Networks.
Testolin A, et al. Cogn Sci. 2016. PMID: 26073971
[Connectionist modeling of higher-level cognitive processes].
Tsuzuki T, et al. Shinrigaku Kenkyu. 2002.PMID: 11977850 Review. Japanese.
Connectionist perspectives on language learning, representation and processing.
Joanisse MF, et al. Wiley Interdiscip Rev Cogn Sci. 2015. PMID: 26263227 Review.
See all similar articles
Cited by 22 articles
The phase space of meaning model of psychopathology: A computer simulation modelling study.
Kleinbub JR, et al. PLoS One. 2021. PMID: 33901183Free PMC article.
A Database for Learning Numbers by Visual Finger Recognition in Developmental Neuro-Robotics.
Davies S, et al. Front Neurorobot. 2021.PMID: 33737873 Free PMC article.
Emergence of Network Motifs in Deep Neural Networks.
Zambra M, et al. Entropy (Basel). 2020.PMID: 33285979 Free PMC article.
Visual sense of number vs. sense of magnitude in humans and machines.
Testolin A, et al. Sci Rep. 2020. PMID: 32572067Free PMC article.
The Challenge of Modeling the Acquisition of Mathematical Concepts.
Testolin A. Front Hum Neurosci. 2020.PMID: 32265678 Free PMC article.
See all "Cited by" articles
Ackley D., Hinton G. E., Sejnowski T. J. (1985). A learning algorithm for boltzmann machines. Cogn. Sci. 9, 147–169 10.1207/s15516709cog0901_7 - DOI
Albert A. (1972). Regression and the Moore-Penrose pseudoinverse. New York, NY: Academic Press
Andrieu C., De Freitas N., Doucet A., Jordan M. I., Freitas N., De (2003). An introduction to MCMC for machine learning. Mach. Learn. 50, 5–43 10.1023/A:1020281327116 - DOI
Baldi P. (2012). Autoencoders, unsupervised learning, and deep architectures. J. Mach. Learn. Res. 27, 37–50
Bengio Y. (2009). Learning deep architectures for AI. Found. Trends Mach. Learn. 2, 1–127 10.1561/2200000006 - DOI
LinkOut - more resources
Full Text Sources
Europe PubMed Central
Frontiers Media SA
PubMed Central
Other Literature Sources
scite Smart Citations
NCI CPTAC Assay Portal
NCBI Literature Resources
MeSH PMC BookShelf
Follow NLM
National Library of Medicine
8600 Rockville Pike
Bethesda, MD 20894
Title & authorsAbstractFiguresSimilar articlesCited byReferencesLinkOut - more resources