Symbolism and connectionism paradigms

Lorenzo Magnani

Human and Machine perception: Emergence, Attention and Creativity

Pavia, September 14-17, 1998

Thagard gives us an analysis of all these approaches (logic, rule, concepts, analogies, images, and connections) in terms of five complex criteria (p. 15)

I propose to adopt this framework because it is very useful to correctly pose the problem of the dichotomy simbolism/connectionism from a theoretical point of view.

The first criterion, representational power, deals with the quantity of information a particular kind of representation can express. Different ways of representation vary very greatly in representational power. Moreover, a computational approach to mental representations surely has to account for three important kinds of high-level reasoning: problems solving (planning, decision making, and explanation), learning, and language. We can evaluate the computational power of a particular representation in terms of how it regards these kinds of reasoning. The ability to learn from experience is also an important part of intelligent reasoning, and the capability to explain the use of language is a desirable aspect of a general cognitive theory.

The psychological plausibility of a theory of mental representation aims to account for quantitative and qualitative results of psychological experiments concerning certain mental capabilities of humans. A theory of mental representation should be also consistent with the results of neuroscientific experiments.

Finally, the practical applicability concerns the effective results in the different areas of education (for instance, how to teach better), design (for instance, new computer interfaces), and intelligent systems (expert and knowledge-based systems - rule-based, case-based, connectionist - in many fields of artificial intelligence).

The cognitive accomplishments made in cognitive science by the approaches in terms of logic, concepts, rules, analogies, and imagery, are well-known. We have very large computational systems that employ the resources of logic (Genesereth & Nilsson, 1987), the most ancient way of representing knowledge, sometimes with the help of probabilistic theory (Pearl, 1988). Concepts (in terms of frames, scripts, semantic networks) (Minksy, 1975, Schank & Abelson, 1977), and rules (as heuristic search and rule-based chunking) (Newell, Shaw & Simon, 1958, Newell & Simon, 1972, Newell, 1990, Anderson, 1983, 1993), constitute the background of many other systems able to model many cognitive aspects of language and problem solving. Analogical thinking (also called case-based reasoning) (Holyoak & Thagard, 1995) and imagery (Kosslyn, 1994, Kosslyn & Koenig, 1992, Glasgow & Papadias, 1992) are subject to very new and interesting research that involve psychological experiments, computational programs, and neurological examinations.

Hence, concepts, analogies, and images are very useful to provide many organized clusters of representations, rules and propositions in logic are useful because of their verbal expressiveness, and images, imagery and connectionism because of their sensory richness. What exactly about connectionism? Let us devote more time to illustrate some details concerning the connectionist approach.

If we consider the two well-known classes of neural networks, local and distributed, in the first class the neuronlike structures are given an identifiable interpretation in terms of concepts or propositions, while in the second one the distributed representations in networks learn how to represent concepts or propositions in complex intricate ways that distribute meaning over clusters of neuronlike structures: both local and distributed representations can be used to perform the so-called parallel contraint satisfaction.

It is well-known the first models of parallel constraint satisfaction were first developed for computer vision (Marr and Poggio, 1976, Feldam, 1981, McClelland and Rumhelhart, 1981); it is also well-known that links between units in a distributed network are adequate to represent simple associations but lack the representational power to grasp more complicated kinds of rules such as that ones containing some uses of universal quantifiers and complex logical relations. Nevertheless, neural networks offer a more flexibility in grasping a broader range of sensory experience, for instance they make possible many more tastes and aromas than we can say in words (Churchland, 1995).

Constructing a plan is naturally understood in terms of rules and analogies, but decision among plans is easier to represent using parallel constraint satisfaction (Thagard & Millgram, 1995, Mannes & Kintsch, 1991) because of the fact we can interpret the facility relations as positive internal constraints, and the negative internal constraints as incompatibility relations, when two actions or goals cannot be performed or satisfied together. We have also to remember that connectionist networks can implement simple types of ruled-based systems (Touretzky & Hinton, 1990) but also the "mental leaps" of the analogical reasoning (Nelson, Thagard, & Hardy, 1994, Holyoak & Thagard, 1995).

In its turn explanation can be seen as activation of prototypes represented by distributed networks (Churchland, 1989); on this perspective, inference to the best explanation can be viewed as the activation of the suitable prototype. For example, the theory of explanatory coherence (Thagard, 1989, 1992), implemented in a connectionist local neural network, is able to activate the hypotheses (or propositions) that better explain (that is, "they have more explanatory coherence") some clusters of evidence.

Adding new units and changing the weights - by training - on the links between units to a connectionist network are the main ways to perform learning tasks (Rumelhart & McClelland, 1986): the so-called feedforward networks trained by backpropagation techniques have had many successful applications.

Finally, McClelland and Rumelhart have also illustrated that word recognition can be considered from the point of view of a parallel constraint satisfaction, and many other connectionist tools have implemented other cognitive linguistic tasks: disambiguation, speech perception, learning verbs by children (and producing errors in that learning ).

We can conclude that the psychological plausibility of connectionist models is very high: from the phenomena of word perception (McClelland & Rumelhart, 1981) to discourse comprehension (Rumelhart & McClelland, 1982, Kintsch, 1988) from analogical mapping (Holyoak & Thagard, 1985) to visual word recognition (Seidenberg & McClelland, 1989). Connectionist weight adjustment is a very powerful mechanism of learning language, even if we do not dispose of a unified connectionist theory of language production and comprehension. If compared with the approaches in terms of logic, rules, and concepts, the connectionism has an obvious neurological plausibility: we can say that most connectionist models are a very rough approximation to the behavior or the real (human) neurons.

At present cognitive science does not show a unified theory able to explain a full range of psychological phenomena, similar to the genetic theory in biology or quantum theory in physics: all the approaches we have illustrated above give rise to different autonomous points of view concerning the "mind". We can say that the different approaches stress different representational and inferential aspects of mind, have different psychological and neurological plausibility, and many original applications in education, design, expert and knowledge-based systems.

The consequence is that an eclectic attitude is imperative, not only from the epistemological point of view, but also in the activity of the knowledge engineer. Some researchers insistently refer to the need of an hybrid approach, an integration of symbolic and neural representation and computation, in areas such as linguistic theory (Prince & Smolesky, 1997), natural language processing (Dyer, 1995) and expert systems (Medsker, 1994), where symbolic approaches have been prevalent in the past. The opportunity of using neural techniques is usually suggested by the fact that neural networks can reduce well-known shortcomings of symbolic problem solvers: for example, brittleness caused by incomplete data, no increase in performance with experience, time consuming knowledge acquisition, sensory processing needed by data acquisition, exploitation of sensory information in the training phase to provide pictorial non-symbolic forms of explanation (Medsker, 1994, Burattini et. al., 1998).