Content deleted Content added
Citation bot (talk | contribs) m Alter: url. Add: year, class, eprint, chapter-url, hdl, bibcode, pmid, pages, doi, issue. Removed or converted URL. Removed parameters. Formatted dashes. Some additions/deletions were actually parameter name changes. | You can use this bot yourself. Report bugs here.| Activated by User:Headbomb | via #UCB_webform |
Citation bot (talk | contribs) Altered journal. Add: authors 1-1. Removed parameters. Some additions/deletions were parameter name changes. | Use this bot. Report bugs. | Suggested by Headbomb | Linked from Wikipedia:WikiProject_Academic_Journals/Journals_cited_by_Wikipedia/Sandbox | #UCB_webform_linked 92/504 |
||
| (117 intermediate revisions by 56 users not shown) | |||
Line 1:
{{Short description|Cognitive science approach}}
[[File:Artificial_neural_network.svg|thumb|
'''Connectionism''' (coined by [[Edward Thorndike]] in the 1930s{{Citation needed|date=March 2024}}) is the name of an approach to the study of human mental processes and cognition that utilizes mathematical models known as connectionist networks or artificial neural networks.<ref>{{Cite web|url=https://iep.utm.edu/connectionism-cognition/#:~:text=Connectionism%20is%20an%20approach%20to,%2C%20neuron%2Dlike%20processing%20units.|title=Internet Encyclopedia of Philosophy|website=iep.utm.edu|access-date=2023-08-19}}</ref> Connectionism has had many 'waves' since its beginnings.
▲[[File:Artificial_neural_network.svg|thumb|Connectionist (ANN) model with a hidden layer]]
The first wave appeared 1943 with [[Warren Sturgis McCulloch]] and [[Walter Pitts]] both focusing on comprehending neural circuitry through a formal and mathematical approach,<ref>{{Cite journal |last1=McCulloch |first1=Warren S. |last2=Pitts |first2=Walter |date=1943-12-01 |title=A logical calculus of the ideas immanent in nervous activity |url=https://doi.org/10.1007/BF02478259 |journal=The Bulletin of Mathematical Biophysics |language=en |volume=5 |issue=4 |pages=115–133 |doi=10.1007/BF02478259 |issn=1522-9602}}</ref> and [[Frank Rosenblatt]] who published the 1958 book “The Perceptron: A Probabilistic Model For Information Storage and Organization in the Brain” in ''Psychological Review'', while working at the Cornell Aeronautical Laboratory.<ref name="2019TheCuriousCaseOfConnectionism">{{Cite journal|last=Berkeley |first= Istvan S. N.|date=2019|title=The Curious Case of Connectionism |journal=Open Philosophy |volume=2019 |issue=2 |pages=190–205|doi= 10.1515/opphil-2019-0018|s2cid= 201061823|doi-access=free }}</ref>
==Basic principles==▼
The first wave ended with the 1969 book about the limitations of the original perceptron idea, written by [[Marvin Minsky]] and [[Papert]], which contributed to discouraging major funding agencies in the US from investing in connectionist research.<ref name="2006_Margaret_Boden_book">{{Cite book|title=Mind as Machine: A History of Cognitive Science | url=https://archive.org/details/mindasmachinehis0002bode/page/n5/mode/thumb |url-access=limited|last=Boden |first=Margaret |publisher=Oxford U.P|year=2006|isbn=978-0-262-63268-3|location=Oxford|pages=914}}</ref> With a few noteworthy deviations, the majority of connectionist research entered a period of inactivity until the mid-1980s. The term connectionist model was reintroduced in the early 1980s in a Cognitive Science paper by Jerome Feldman and Dana Ballard.
The central connectionist principle is that mental phenomena can be described by interconnected networks of simple and often uniform units. The form of the connections and the units can vary from model to model. For example, units in the network could represent [[neurons]] and the connections could represent [[synapses]], as in the [[human brain]].▼
The second wave blossomed in the late 1980s, following the 1987 book about Parallel Distributed Processing by [[James L. McClelland]], [[David E. Rumelhart]] et al., which introduced a couple of improvements to the simple perceptron idea, such as intermediate processors (known as "[[hidden layers]]" now) alongside input and output units and used [[Sigmoid function|sigmoid]] [[activation function]] instead of the old 'all-or-nothing' function. Their work has, in turn, built upon that of [[John Hopfield]], who was a key figure investigating the mathematical characteristics of sigmoid activation functions.<ref name="2019TheCuriousCaseOfConnectionism"/> From the late 1980s to the mid-1990s, connectionism took on an almost revolutionary tone when Schneider,<ref name="1987_Paradigm_Shift_in_Psychology">{{Cite journal|last=Schneider |first=Walter |date=1987 |title=Connectionism: Is it a Paradigm Shift for Psychology? |journal=Behavior Research Methods, Instruments, & Computers |volume=19 |pages=73–83|doi=10.1515/opphil-2019-0018 |s2cid=201061823 |doi-access=free }}</ref> [[Terence Horgan]] and Tienson posed the question of whether connectionism represented a [[paradigm shift|fundamental shift]] in psychology and [[GOFAI]].<ref name="2019TheCuriousCaseOfConnectionism"/> Some advantages of the second wave connectionist approach included its applicability to a broad array of functions, structural approximation to biological neurons, low requirements for innate structure, and capacity for [[graceful degradation]].<ref name=":2">{{Cite book|title=The Algebraic Mind: Integrating Connectionism and Cognitive Science (Learning, Development, and Conceptual Change)|url=https://archive.org/details/algebraicmindint00marc_403|url-access=limited|last=Marcus|first=Gary F.|publisher=MIT Press|year=2001|isbn=978-0-262-63268-3|location=Cambridge, Massachusetts|pages=[https://archive.org/details/algebraicmindint00marc_403/page/n43 27]–28}}</ref> Some disadvantages of the second wave connectionist approach included the difficulty in deciphering how ANNs process information, or account for the compositionality of mental representations, and a resultant difficulty explaining phenomena at a higher level.<ref name=":1">{{Cite journal|last=Smolensky|first=Paul|date=1999|title=Grammar-based Connectionist Approaches to Language|journal=Cognitive Science|volume=23|issue=4|pages=589–613|doi=10.1207/s15516709cog2304_9|doi-access=free}}</ref>
The current (third) wave has been marked by advances in [[Deep Learning]] allowing for [[Large language model]]s.<ref name="2019TheCuriousCaseOfConnectionism"/> The success of [[deep learning]] networks in the past decade has greatly increased the popularity of this approach, but the complexity and scale of such networks has brought with them increased [[Explainable artificial intelligence|interpretability problems]].<ref name=":0">{{cite book|url=https://plato.stanford.edu/archives/fall2018/entries/connectionism/|title=The Stanford Encyclopedia of Philosophy|first=James|last=Garson|editor-first=Edward N.|editor-last=Zalta|date=27 November 2018|publisher=Metaphysics Research Lab, Stanford University|via=Stanford Encyclopedia of Philosophy}}</ref>
{{Main|Artificial neural networks}}▼
▲The central connectionist principle is that mental phenomena can be described by interconnected networks of simple and often uniform units. The form of the connections and the units can vary from model to model. For example, units in the network could represent [[neurons]] and the connections could represent [[synapses]], as in the [[human brain]]. This principle has been seen as an alternative to [[GOFAI]] and the classical [[theory of mind|theories of mind]] based on symbolic computation, but the extent to which the two approaches are compatible has been the subject of much debate since their inception.<ref name=":0" />
===Activation function===
{{Main|Activation function}}
Internal states of any network change over time due to neurons sending a signal to a succeeding layer of neurons in the case of a feedforward network, or to a previous layer in the case of a recurrent network. Discovery of non-linear activation functions has enabled the second wave of connectionism.
===Memory and learning===
▲{{Main|Artificial neural networks|Deep learning}}
Neural networks follow two basic principles:
# Any mental state can be described as an (N)-dimensional [[Vector (mathematics)|vector]] of numeric activation values over neural units in a network.
Most of the variety among
* ''Interpretation of units'': Units can be interpreted as neurons or groups of neurons.
* ''Definition of activation'': Activation can be defined in a variety of ways. For example, in a [[Boltzmann machine]], the activation is interpreted as the probability of generating an action potential spike, and is determined via a [[logistic function]] on the sum of the inputs to a unit.
* ''Learning algorithm'': Different networks modify their connections differently. In general, any mathematically defined change in connection weights over time is referred to as the "learning algorithm".
===Biological realism===
Connectionist work in general does not need to be biologically realistic
==
▲The weights in a neural network are adjusted according to some [[learning rule]] or algorithm, such as [[Hebbian theory|Hebbian learning]]. Thus, connectionists have created many sophisticated learning procedures for neural networks. Learning always involves modifying the connection weights. In general, these involve mathematical formulas to determine the change in weights when given sets of data consisting of activation vectors for some subset of the neural units. Several studies have been focused on designing teaching-learning methods based on connectionism.<ref>{{Cite journal|last=Novo|first=María-Luisa|last2=Alsina|first2=Ángel|last3=Marbán|first3=José-María|last4=Berciano|first4=Ainhoa|date=2017|title=Connective Intelligence for Childhood Mathematics Education|journal=Comunicar|language=es|volume=25|issue=52|pages=29–39|doi=10.3916/c52-2017-03|issn=1134-3478}}</ref>
Hopfield networks had precursors in the [[Ising model]] due to [[Wilhelm Lenz]] (1920) and [[Ernst Ising]] (1925), though the Ising model conceived by them did not involve time. [[Monte Carlo method|Monte Carlo]] simulations of Ising model required the advent of computers in the 1950s.<ref name="brush67">{{cite journal |last1=Brush |first1=Stephen G. |year=1967 |title=History of the Lenz-Ising Model |journal=Reviews of Modern Physics |volume=39 |issue=4 |pages=883–893 |bibcode=1967RvMP...39..883B |doi=10.1103/RevModPhys.39.883}}</ref>
==
The first wave begun in 1943 with [[Warren Sturgis McCulloch]] and [[Walter Pitts]] both focusing on comprehending neural circuitry through a formal and mathematical approach.<ref name="2019TheCuriousCaseOfConnectionism"/> McCulloch and Pitts showed how neural systems could implement [[first-order logic]]: Their classic paper "A Logical Calculus of Ideas Immanent in Nervous Activity" (1943) is important in this development here. They were influenced by the work of [[Nicolas Rashevsky]] in the 1930s.
[[Donald O. Hebb|Hebb]] contributed greatly to speculations about neural functioning, and proposed a learning principle, [[Hebbian learning]]. [[Karl Lashley|Lashley]] argued for distributed representations as a result of his failure to find anything like a localized [[Engram (neuropsychology)|engram]] in years of [[lesion]] experiments. [[Friedrich Hayek]] independently conceived the model, first in a brief unpublished manuscript in 1920,<ref>Hayek, Friedrich A. [1920] 1991. Beiträge zur Theorie der Entwicklung des Bewusstseins [Contributions to a theory of how consciousness develops]. Manuscript, translated by Grete Heinz.</ref><ref>{{Cite journal |last=Caldwell |first=Bruce |date=2004 |title=Some Reflections on F.A. Hayek's The Sensory Order |url=http://link.springer.com/10.1007/s10818-004-5505-9 |journal=Journal of Bioeconomics |language=en |volume=6 |issue=3 |pages=239–254 |doi=10.1007/s10818-004-5505-9 |s2cid=144437624 |issn=1387-6996}}</ref> then expanded into a book in 1952.<ref>{{Cite book |last=Hayek |first=F. A. |title=The Sensory Order: An Inquiry into the Foundations of Theoretical Psychology |date=2012-09-15 |publisher=The University of Chicago Press |edition=1st |language=en}}</ref>
The Perceptron machines were proposed and built by [[Frank Rosenblatt]], who published the 1958 paper “The Perceptron: A Probabilistic Model For Information Storage and Organization in the Brain” in ''Psychological Review'', while working at the Cornell Aeronautical Laboratory. He cited Hebb, Hayek, Uttley, and [[W. Ross Ashby|Ashby]] as main influences.
Another form of connectionist model was the [[Stratificational linguistics|relational network]] framework developed by the [[linguist]] [[Sydney Lamb]] in the 1960s
The research group led by Widrow empirically searched for methods to train two-layered [[ADALINE]] networks, with limited success.<ref>pp 124-129, Olazaran Rodriguez, Jose Miguel. ''[https://web.archive.org/web/20221111165150/https://era.ed.ac.uk/bitstream/handle/1842/20075/Olazaran-RodriguezJM_1991redux.pdf?sequence=1&isAllowed=y A historical sociology of neural network research]''. PhD Dissertation. University of Edinburgh, 1991.</ref><ref>Widrow, B. (1962) ''Generalization and information storage in networks of ADALINE "neurons"''. In M. C. Yovits, G. T. Jacobi, & G. D. Goldstein (Ed.), Self-Organizing Svstems-1962 (pp. 435-461). Washington, DC: Spartan Books.</ref>
A method to train multilayered perceptrons with arbitrary levels of trainable weights was published by [[Alexey Grigorevich Ivakhnenko]] and Valentin Lapa in 1965, called the [[Group method of data handling|Group Method of Data Handling]].<ref name="DLhistory">{{cite arXiv |eprint=2212.11279 |class=cs.NE |first=Juergen |last=Schmidhuber |author-link=Juergen Schmidhuber |title=Annotated History of Modern AI and Deep Learning |date=2022}}</ref><ref name="ivak1965">{{cite book |last=Ivakhnenko |first=A. G. |url={{google books |plainurl=y |id=FhwVNQAACAAJ}} |title=Cybernetic Predicting Devices |publisher=CCM Information Corporation |year=1973}}</ref><ref name="ivak1967">{{cite book |last1=Ivakhnenko |first1=A. G. |url={{google books |plainurl=y |id=rGFgAAAAMAAJ}} |title=Cybernetics and forecasting techniques |last2=Grigorʹevich Lapa |first2=Valentin |publisher=American Elsevier Pub. Co. |year=1967}}</ref> This method employs incremental layer by layer training based on [[regression analysis]], where useless units in hidden layers are pruned with the help of a validation set. In 1972, [[Shun'ichi Amari]] made this architecture [[Adaptation (computer science)|adaptive]].<ref name="Amari1972">{{cite journal |last1=Amari |first1=Shun-Ichi |date=1972 |title=Learning patterns and pattern sequences by self-organizing nets of threshold elements |journal=IEEE Transactions |volume=C |issue=21 |pages=1197–1206}}</ref><ref name="DLhistory" />
The first multilayered perceptrons trained by [[stochastic gradient descent]]<ref name="robbins1951">{{Cite journal |last1=Robbins |first1=H. |author-link=Herbert Robbins |last2=Monro |first2=S. |year=1951 |title=A Stochastic Approximation Method |journal=The Annals of Mathematical Statistics |volume=22 |issue=3 |page=400 |doi=10.1214/aoms/1177729586 |doi-access=free}}</ref> was published in 1967 by [[Shun'ichi Amari]].<ref name="Amari1967">{{cite journal |last1=Amari |first1=Shun'ichi |author-link=Shun'ichi Amari |date=1967 |title=A theory of adaptive pattern classifier |journal=IEEE Transactions |volume=EC |issue=16 |pages=279–307}}</ref><ref name="DLhistory" /> In computer experiments conducted by Amari's student Saito, a five layer MLP with two modifiable layers learned useful [[Knowledge representation|internal representations]] to classify non-linearily separable pattern classes.<ref name="DLhistory" />
==The second wave==
The second wave begun in late 1980s, following the 1987 two-volume book about the ''Parallel Distributed Processing'' (PDP) by [[James L. McClelland]], [[David E. Rumelhart]] et al., which has introduced a couple of improvements to the simple perceptron idea, such as intermediate processors (known as "[[hidden layers]]" now) alongside input and output units and using [[Sigmoid function|sigmoid]] [[activation function]] instead of the old 'all-or-nothing' function. Their work has, in turn, built upon [[John Hopfield]], who was a key figure investigating the mathematical characteristics of sigmoid activation functions.<ref name="2019TheCuriousCaseOfConnectionism"/>
▲Many connectionist principles can be traced to early work in [[psychology]], such as that of [[William James]].<ref>{{cite book |last1=Anderson |first1= James A.|last2=Rosenfeld |first2= Edward |date= 1989|title= Neurocomputing: Foundations of Research|chapter-url= |location= |publisher= A Bradford Book |page= 1|chapter = Chapter 1: (1890) William James ''Psychology (Brief Course)'' |isbn=978-0262510486 |accessdate= }}</ref> Psychological theories based on knowledge about the human brain were fashionable in the late 19th century. As early as 1869, the neurologist [[John Hughlings Jackson]] argued for multi-level, distributed systems. Following from this lead, [[Herbert Spencer]]'s ''Principles of Psychology'', 3rd edition (1872), and [[Sigmund Freud]]'s ''Project for a Scientific Psychology'' (composed 1895) propounded connectionist or proto-connectionist theories. These tended to be speculative theories. But by the early 20th century, [[Edward Thorndike]] was experimenting on learning that posited a connectionist type network.
▲Another form of connectionist model was the [[Stratificational linguistics|relational network]] framework developed by the [[linguist]] [[Sydney Lamb]] in the 1960s. Relational networks have been only used by linguists, and were never unified with the PDP approach. As a result, they are now used by very few researchers.
==Connectionism vs. computationalism debate==
Line 78 ⟶ 65:
* Computationalists often posit [[domain specificity|domain specific]] symbolic sub-systems designed to support learning in specific areas of cognition (e.g., language, intentionality, number), whereas connectionists posit one or a small set of very general learning-mechanisms.
Despite these differences, some theorists have proposed that the connectionist architecture is simply the manner in which organic brains happen to implement the symbol-manipulation system. This is logically possible, as it is well known that connectionist models can implement symbol-manipulation systems of the kind used in computationalist models,<ref name=":3">{{Cite journal|last=Chang|first=Franklin|date=2002|title=Symbolically speaking: a connectionist model of sentence production|journal=Cognitive Science|language=en|volume=26|issue=5|pages=609–651|doi=10.1207/s15516709cog2605_3|issn=1551-6709|doi-access=free}}</ref>
The debate was largely centred on logical arguments about whether connectionist networks could produce the syntactic structure observed in this sort of reasoning. This was later achieved although using fast-variable binding abilities outside of those standardly assumed in connectionist models.<ref name=":3" /><ref>{{Cite journal|
Part of the appeal of computational descriptions is that they are relatively easy to interpret, and thus may be seen as contributing to our understanding of particular mental processes, whereas connectionist models are in general more opaque, to the extent that they may be describable only in very general terms (such as specifying the learning algorithm, the number of units, etc.), or in unhelpfully low-level terms. In this sense, connectionist models may instantiate, and thereby provide evidence for, a broad theory of cognition (i.e., connectionism), without representing a helpful theory of the particular process that is being modelled. In this sense, the debate might be considered as to some extent reflecting a mere difference in the level of analysis in which particular theories are framed. Some researchers suggest that the analysis gap is the consequence of connectionist mechanisms giving rise to [[Emergence|emergent phenomena]] that may be describable in computational terms.<ref>{{Cite journal|last=Ellis|first=Nick C.|date=1998|title=Emergentism, Connectionism and Language Learning|url=http://www-personal.umich.edu/~ncellis/NickEllis/Publications_files/Emergentism.pdf|journal=Language Learning|volume=48
In the 2000s, the popularity of [[Cognitive Model#Dynamical systems|dynamical systems]] in [[philosophy of mind]] have added a new perspective on the debate;<ref>{{Citation |last=Van Gelder |first=Tim |year=1998 |title=The dynamical hypothesis in cognitive science |journal=Behavioral and Brain Sciences |volume=21 |issue=5 |pages=615–28; discussion 629–65 |doi=10.1017/S0140525X98001733 |pmid=10097022 |url= https://www.cambridge.org/core/services/aop-cambridge-core/content/view/S0140525X98271731 |access-date=28 May 2022}}</ref><ref>{{Cite journal |last=Beer |first=Randall D. |date=March 2000 |title=Dynamical approaches to cognitive science |journal=Trends in Cognitive Sciences |volume=4 |issue=3 |pages=91–99 |doi=10.1016/s1364-6613(99)01440-0 |pmid=10689343 |s2cid=16515284 |issn=1364-6613}}</ref> some authors{{which|date=February 2016}} now argue that any split between connectionism and computationalism is more conclusively characterized as a split between computationalism and [[Cognitive Model#Dynamical systems|dynamical systems]].
In 2014, [[Alex Graves (computer scientist)|Alex Graves]] and others from [[DeepMind]] published a series of papers describing a novel Deep Neural Network structure called the [[Neural Turing Machine]]<ref>{{cite
▲The debate was largely centred on logical arguments about whether connectionist networks could produce the syntactic structure observed in this sort of reasoning. This was later achieved although using fast-variable binding abilities outside of those standardly assumed in connectionist models<ref name=":3" /><ref>{{Cite journal|last=Shastri|first=Lokendra|last2=Ajjanagadde|first2=Venkat|date=1993/09|title=From simple associations to systematic reasoning: A connectionist representation of rules, variables and dynamic bindings using temporal synchrony|journal=Behavioral and Brain Sciences|language=en|volume=16|issue=3|pages=417–451|doi=10.1017/S0140525X00030910|issn=1469-1825}}</ref>. {{As of | 2016}}, progress in neurophysiology and general advances in the understanding of neural networks have led to the successful modelling of a great many of these early problems, and the debate about fundamental cognition has, thus, largely been decided among neuroscientists in favour of connectionism.{{citation needed|date=March 2012}} However, these fairly recent developments have yet to reach consensus acceptance among those working in other fields, such as psychology or philosophy of mind.
==Symbolism vs. connectionism debate==
▲Part of the appeal of computational descriptions is that they are relatively easy to interpret, and thus may be seen as contributing to our understanding of particular mental processes, whereas connectionist models are in general more opaque, to the extent that they may be describable only in very general terms (such as specifying the learning algorithm, the number of units, etc.), or in unhelpfully low-level terms. In this sense connectionist models may instantiate, and thereby provide evidence for, a broad theory of cognition (i.e., connectionism), without representing a helpful theory of the particular process that is being modelled. In this sense the debate might be considered as to some extent reflecting a mere difference in the level of analysis in which particular theories are framed. Some researchers suggest that the analysis gap is the consequence of connectionist mechanisms giving rise to [[Emergence|emergent phenomena]] that may be describable in computational terms.<ref>{{Cite journal|last=Ellis|first=Nick C.|date=1998|title=Emergentism, Connectionism and Language Learning|url=http://www-personal.umich.edu/~ncellis/NickEllis/Publications_files/Emergentism.pdf|journal=Language Learning|volume=48:4|pages=631–664|via=}}</ref>
Smolensky's Subsymbolic Paradigm<ref>P. Smolensky: On the proper treatment of connectionism. In: Behavioral and Brain Sciences. Band 11, 1988, S. 1-74.</ref><ref>P. Smolensky: The constituent structure of connectionist mental states: a reply to Fodor and Pylyshyn. In: T. Horgan, J. Tienson (Hrsg.): Spindel Conference 1987: Connectionism and the Philosophy of Mind. The Southern Journal of Philosophy. Special Issue on Connectionism and the Foundations of Cognitive Science. Supplement. Band 26, 1988, S. 137-161.</ref> has to meet the Fodor-Pylyshyn challenge<ref>J.A. Fodor, Z.W. Pylyshyn: Connectionism and cognitive architecture: a critical analysis. Cognition. Band 28, 1988, S. 12-13, 33-50.</ref><ref>J.A. Fodor, B. McLaughlin: Connectionism and the problem of systematicity: why Smolensky's solution doesn't work. Cognition. Band 35, 1990, S. 183-184.</ref><ref>B. McLaughlin: The connectionism/classicism battle to win souls. Philosophical Studies, Band 71, 1993, S. 171-172.</ref><ref>B. McLaughlin: Can an ICS architecture meet the systematicity and productivity challenges? In: P. Calvo, J. Symons (Hrsg.): The Architecture of Cognition. Rethinking Fodor and Pylyshyn's Systematicity Challenge. MIT Press, Cambridge/MA, London, 2014, S. 31-76.</ref> formulated by classical symbol theory for a convincing theory of cognition in modern connectionism. In order to be an adequate alternative theory of cognition, Smolensky's Subsymbolic Paradigm would have to explain the existence of systematicity or systematic relations in language cognition without the assumption that cognitive processes are causally sensitive to the classical constituent structure of mental representations. The subsymbolic paradigm, or connectionism in general, would thus have to explain the existence of systematicity and compositionality without relying on the mere implementation of a classical cognitive architecture. This challenge implies a dilemma: If the Subsymbolic Paradigm could contribute nothing to the systematicity and compositionality of mental representations, it would be insufficient as a basis for an alternative theory of cognition. However, if the Subsymbolic Paradigm's contribution to systematicity requires mental processes grounded in the classical constituent structure of mental representations, the theory of cognition it develops would be, at best, an implementation architecture of the classical model of symbol theory and thus not a genuine alternative (connectionist) theory of cognition.<ref>J.A. Fodor, B. McLaughlin: Connectionism and the problem of systematicity: Why Smolensky's solution doesn't work. Cognition. Band 35, 1990, S. 183-184.</ref> The classical model of symbolism is characterized by (1) a combinatorial syntax and semantics of mental representations and (2) mental operations as structure-sensitive processes, based on the fundamental principle of syntactic and semantic constituent structure of mental representations as used in Fodor's "Language of Thought (LOT)".<ref>J.A. Fodor: The language of thought. Harvester Press, Sussex, 1976, ISBN 0-85527-309-7.</ref><ref>J.A. Fodor: LOT 2: The language of thought revisited. Clarendon Press, Oxford, 2008, ISBN 0-19-954877-3.</ref> This can be used to explain the following closely related properties of human cognition, namely its (1) productivity, (2) systematicity, (3) compositionality, and (4) inferential coherence.<ref>J.A. Fodor, Z.W. Pylyshyn (1988), S. 33-48.</ref>
This challenge has been met in modern connectionism, for example, not only by Smolensky's "Integrated Connectionist/Symbolic (ICS) Cognitive Architecture",<ref>P. Smolenky: Reply: Constituent structure and explanation in an integrated connectionist / symbolic cognitive architecture. In: C. MacDonald, G. MacDonald (Hrsg.): Connectionism: Debates on psychological explanation. Blackwell Publishers. Oxford/UK, Cambridge/MA. Vol. 2, 1995, S. 224, 236-239, 242-244, 250-252, 282.</ref><ref>P. Smolensky, G. Legendre: The Harmonic Mind: From Neural Computation to Optimality-Theoretic Grammar. Vol. 1: Cognitive Architecture. A Bradford Book, The MIT Press, Cambridge, London, 2006a, ISBN 0-262-19526-7, S. 65-67, 69-71, 74-75, 154-155, 159-202, 209-210, 235-267, 271-342, 513.</ref> but also by Werning and Maye's "Oscillatory Networks".<ref>M. Werning: Neuronal synchronization, covariation, and compositional representation. In: M. Werning, E. Machery, G. Schurz (Hrsg.): The compositionality of meaning and content. Vol. II: Applications to linguistics, psychology and neuroscience. Ontos Verlag, 2005, S. 283-312.</ref><ref>M. Werning: Non-symbolic compositional representation and its neuronal foundation: towards an emulative semantics. In: M. Werning, W. Hinzen, E. Machery (Hrsg.): The Oxford Handbook of Compositionality. Oxford University Press, 2012, S. 633-654.</ref><ref>A. Maye und M. Werning: Neuronal synchronization: from dynamics feature binding to compositional representations. Chaos and Complexity Letters, Band 2, S. 315-325.</ref> An overview of this is given for example by Bechtel & Abrahamsen,<ref>Bechtel, W., Abrahamsen, A.A. ''Connectionism and the Mind: Parallel Processing, Dynamics, and Evolution in Networks.'' 2nd Edition. Blackwell Publishers, Oxford. 2002</ref> Marcus<ref>G.F. Marcus: The algebraic mind. Integrating connectionism and cognitive science. Bradford Book, The MIT Press, Cambridge, 2001, ISBN 0-262-13379-2.</ref> and Maurer.<ref>H. Maurer: Cognitive science: Integrative synchronization mechanisms in cognitive neuroarchitectures of the modern connectionism. CRC Press, Boca Raton/FL, 2021, ISBN 978-1-351-04352-6. https://doi.org/10.1201/9781351043526</ref>
▲In 2014, [[Alex Graves (computer scientist)|Alex Graves]] and others from [[DeepMind]] published a series of papers describing a novel Deep Neural Network structure called the [[Neural Turing Machine]]<ref>{{cite arxiv|last1=Graves|first1=Alex|title=Neural Turing Machines|eprint=1410.5401|class=cs.NE|year=2014}}</ref> able to read symbols on a tape and store symbols in memory. Relational Networks, another Deep Network module published by DeepMind are able to create object-like representations and manipulate them to answer complex questions. Relational Networks and Neural Turing Machines are further evidence that connectionism and computationalism need not be at odds.
==See also==
{{div col|colwidth=20em}}
* [[Associationism]]
* [[Artificial intelligence]]
* [[Behaviorism]]
* [[Catastrophic interference]]
* [[Calculus of relations]]
* [[Cybernetics]]
* [[Deep learning]]
* [[Eliminative materialism]]
* [[Feature integration theory]]
* [[Genetic algorithm]]
* [[Harmonic grammar]]
* [[Machine learning]]
* [[Pandemonium architecture]]
Line 103 ⟶ 100:
==Notes==
{{
==References==
* Feldman, Jerome and Ballard, Dana. Connectionist models and their properties(1982). Cognitive Science. V6, Iissue 3 , pp205-254.
* Rumelhart, D.E., J.L. McClelland and the PDP Research Group (1986). ''Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Volume 1: Foundations'', Cambridge, Massachusetts: [[MIT Press]], {{ISBN|978- * McClelland, J.L., D.E. Rumelhart and the PDP Research Group (1986). ''Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Volume 2: Psychological and Biological Models'', Cambridge, Massachusetts: MIT Press, {{ISBN|978-
* Pinker, Steven and Mehler, Jacques (1988). ''Connections and Symbols'', Cambridge MA: MIT Press, {{ISBN|978-
* Jeffrey L. Elman, Elizabeth A. Bates, Mark H. Johnson, Annette Karmiloff-Smith, Domenico Parisi, Kim Plunkett (1996). ''Rethinking Innateness: A connectionist perspective on development'', Cambridge MA: MIT Press, {{ISBN|978-
* Marcus, Gary F. (2001). ''The Algebraic Mind: Integrating Connectionism and Cognitive Science (Learning, Development, and Conceptual Change)'', Cambridge, Massachusetts: MIT Press, {{ISBN|978-
*{{cite journal|title=A Brief History of Connectionism|author=David A. Medler|url=http://www.blutner.de/NeuralNets/Texts/Medler.pdf|year= 1998|volume=1|pages=61–101|journal=Neural Computing Surveys}}
* Maurer, Harald (2021). ''Cognitive Science: Integrative Synchronization Mechanisms in Cognitive Neuroarchitectures of the Modern Connectionism'', Boca Raton/FL: CRC Press, https://doi.org/10.1201/9781351043526, {{ISBN|978-1-351-04352-6}}
==External links==
{{Spoken Wikipedia|En-Connectionism.ogg|date=2011-11-26}}
* [http://philosophy.uwaterloo.ca/MindDict/connectionism.html Dictionary of Philosophy of Mind entry on connectionism]
* {{cite SEP |url-id=connectionism |title=Connectionism |last=Garson |first=James |author-link=James Garson}}
* [http://srsc.ulb.ac.be/pdp/iac/IAC.html A demonstration of Interactive Activation and Competition Networks] {{Webarchive|url=https://web.archive.org/web/20150703142148/http://srsc.ulb.ac.be/pdp/iac/IAC.html |date=2015-07-03 }}
* {{cite IEP |url-id=connect |title=Connectionism}}
* [https://sapienlabs.org/the-crisis-of-computational-neuroscience/ Critique of connectionism]
{{Cybernetics}}
{{Evolutionary psychology}}
{{Authority control}}
[[Category:Cognitive science]]
| |||