100 %
0 %
Information about McClelland

Published on March 10, 2008

Author: Panfilo


Connectionist Models of Development: It’s (Mostly) About Change Over Developmental Time:  Connectionist Models of Development: It’s (Mostly) About Change Over Developmental Time James L. McClelland Center for the Neural Basis of Cognition &Department of Psychology Carnegie Mellon University CNBC A Joint Project of Carnegie Mellon and the University of Pittsburgh Connectionist Models:  Connectionist Models …have been applied to development to address change over developmental time, and to several other issues. This talk will stress the developmental change issue. The models can be used to simulate the dynamics of activation in real time, and, like dynamical systems, they can incorporate graded propagation of activation, attractors, decay, and noise. When applied to change over developmental time, the real time dynamics are often ‘simplified out’, so that the cumulative impact of developmental experience can be explored in a tractable way. It may be that incorporating more dynamical properties into these connectionist models might make them better at capturing some of the challenges facing them in several areas of development. Models Illustrating These Points:  Models Illustrating These Points Leaky competing accumulator model Usher and McClelland, 2001 Speech attractor learning model Vallabha and McClelland, submitted Model of differentiation of conceptual knowledge in development Rogers and McClelland, 2004 Usher and McClelland (2001) Leaky Competing Accumulator Model:  Usher and McClelland (2001) Leaky Competing Accumulator Model Addresses the process of deciding between two alternatives based on external input (r1 and r2) with leakage, self-excitation, mutual inhibition, and noise dx1/dt = r1-k(x1)+a[x1]+–b[x2]++x1 dx2/dt = r2-k(x2)+a[x2]+–b[x1]++x2 Addresses data on the time course of decision making in time-controlled and standard RT tasks. Models details of accuracy data and RT distribution shape as a function of relative value of r1 and r2 Easily extends to n alternatives. Vallabha and McClelland, Submitted:  Vallabha and McClelland, Submitted Uses within-trial dynamics of U&M, ands two-way connections, and a variant of Hebbian learning: dwij = eai(aj – wij) Slow learning within the perceptual system over developmental time creates attractors for native language categories. Faster learning within a simulation of a six-day training experiment creates new attractors that interact with attractors in slow learning system. Feedback on response accuracy increments activation of correct response unit before Hebbian weight adjustment is applied. Slide8:  Fixed Training Stimuli Initial Adaptive Stimuli lock rock English Endpoint Stimuli lock and rock Slide13:  L Stimulus index R Activity of the unit L Stimulus index R And now for something completely different…:  And now for something completely different… A model with no real-time dynamics trained using back propagation that exhibits interesting learning dynamics over developmental time Development of Conceptual Knowledge (Rogers and McClelland):  Development of Conceptual Knowledge (Rogers and McClelland) Addresses differentiation of conceptual knowledge during development Shows interesting dynamic properties including acceleration and deceleration, and U-shaped developmental trajectories. Also exhibits reorganization of conceptual knowledge over the course of training Differentiation in Development:  Differentiation in Development Slide18:  The Rumelhart Model Slide19:  The Training Data: All propositions true of items at the bottom level of the tree, e.g.: Robin can {grow, move, fly} Slide20:  The Rumelhart Model: Target output for ‘robin can’ input Slide21:  Forward Propagation of Activation Slide22:  dk ~ (tk-ak) wij di ~ Sdkwki wki aj Back Propagation of Error (d) Error-correcting learning: At the output layer: Dwki = edkai At the prior layer: Dwij = edjaj … ai Slide23:  The Rumelhart Model Slide26:  Eigenvectors of the property covariance matrix. Plant-Animal Fish-Bird Tree-Flower Effects of Coherent Variation of Properties on Learning in Connectionist Models:  Effects of Coherent Variation of Properties on Learning in Connectionist Models Attributes that vary together create the superordinate concepts that populate the taxonomic hierarchy, and determine which properties are central and which are incidental to a given concept. Where sets of attributes vary together across concepts, they exert a strong effect on learning. These properties are easier to learn than properties that do not covary. Concepts with co-varying properties stay together through semantic space and form the clusters corresponding to superordinate concepts. Arbitrary properties (those that do not co-vary with others) are very difficult to learn. They control a late stage of differentiation in which individual concepts within clusters become distinct. Slide28:  Coherence Training Environment No category labels are provided Each concept and each property occurs with equal frequency Properties Coherent Incoherent Concepts Effects of Coherence on Learning:  Effects of Coherence on Learning Coherent Properties Incoherent Properties Effect of Coherence on Representation:  Effect of Coherence on Representation Slide36:  Rogers and McClelland (2004) extend this model to address: Disintegration of conceptual knowledge in dementia Basic level, typicality, and frequency effects Differential coherence of categories and differential importance of some properties for category membership Reorganization and coalescence of conceptual knowledge Conceptual Reorganization (Carey, 1985):  Conceptual Reorganization (Carey, 1985) Carey demonstrates that young children ‘discover’ the unity of plants and animals as living things with many shared properties only around the age of 10. She suggests that the coalescence of the concept of living thing depends in part on learning about diverse aspects of both plants and animals including Nature of life sustaining processes What it means to be dead vs. alive Reproductive properties However, she has no mechanistic suggestions as to how the reorganization of conceptual knowledge actually occurs. Conceptual Reorganization in the Model:  Conceptual Reorganization in the Model Our simulation model provides a vehicle for exploring how conceptual reorganization can occur. The model is capable of forming initial representations based on superficial appearances Later, it can discover shared structure that cuts across several different relational contexts, and use the emergent common structure as a basis for a deeper organization. Reorganization Simulation:  Reorganization Simulation We consider the coalescence of the superordinate categories plant and animal, in situations where the training data initially supports a superficial organization based on appearance properties. In each training pattern, the input is an item and one of the three relations: ISA, HAS, or CAN. The target includes all of the superficial appearance properties (IS properties) plus the properties appropriate for the relation. The model quickly learns representations that capture the superficial IS properties. Later, it reorganizes these representations as it learns the relation-dependent properties. Organization of Conceptual Knowledge at Different Points in Development:  Organization of Conceptual Knowledge at Different Points in Development Toward a Synthesis:  Toward a Synthesis Rogers and McClelland model does not use real-time propagation of activation, or bi-directional connections to create attractors, although a related model of Rogers et al (2004) does incorporate these properties. Neither model incorporates intrinsic variability or Hebbian learning. A speculation: Incorporating these additional properties would improve the models’ ability to address issues such as instability at stage transitions. Such models would also combine some of the attractive properties both of dynamical systems and connectionist modeling approaches. Inference and Memory in the Theory:  Inference and Memory in the Theory A semantic representation for a new item can be derived by error propagation, using knowledge already stored in the weights. Rapid learning of new information requires a complementary system in the hippocampus (McClelland, McNaughton & O’Reilly, 1995). Slide44:  Find representation for sparrow from ‘sparrow ISA bird’ by back propagation. Slide45:  Use the representation to infer what a sparrow can do. Slide46:  sparrow Hippocampus Use the hippocampal memory system to store a memory for the learning episode. Differential Importance (Marcario, 1991):  Differential Importance (Marcario, 1991) 3-4 yr old children see a puppet and are told he likes to eat, or play with, a certain object (e.g., top object at right) Children then must choose another one that will “be the same kind of thing to eat” or that will be “the same kind of thing to play with”. In the first case they tend to choose the object with the same color. In the second case they will tend to choose the object with the same shape. Adjustments to Training Environment:  Adjustments to Training Environment Among the plants: All trees are large All flowers are small Either can be bright or dull Among the animals: All birds are bright All fish are dull Either can be small or large Testing Feature Importance:  Testing Feature Importance After partial learning, model is shown eight test objects: Four “Animals”: All have skin All combinations of bright/dull and large/small Four “Plants”: All have roots All combinations of bright/dull and large/small Representations are generated by using back-propagation to representation. Representations are then compared to see which animals are treated as most similar, and which plants are treated as most similar. Similarities of Obtained Representations:  Similarities of Obtained Representations Size is relevant for Plants Brightness is relevant for Animals Differential Feature Importance:  Differential Feature Importance The simulation suggests that domain-general learning mechanisms can learn that different features are important for different concepts. We can say that the network has acquired domain-specific knowledge of just the sort theory theorists claim children can acquire. It does so from the distributions of properties of concepts, without the aid of initial domain knowledge. Causal Inference:  Causal Inference Young children can attribute causal powers to objects based on single observations of scenarios in which the objects participate. Gopnik’s ‘blickett’ experiments Causal powers are central to children’s naming responses, leading them to assign the same name to objects with different appearance properties but similar causal powers. Do we need an innate mechanism for causal inference, as Gopnik Suggests? My Perspective:  My Perspective Domain general mechanisms that are sensitive to experience underlie the development and elaboration of conceptual and causal reasoning. These mechanisms acquire domain-specificity through gradual learning. They also provide a basis for inference and formation of episodic memories whose conceptual content is shaped by experience. Extension of the Model to Causal Inference (in my dreams):  Extension of the Model to Causal Inference (in my dreams) Gradual learning allows network to form internal representations of objects that account for their “causal powers”, based on the consequences of their participation in events. Subsequently, observation of the causal properties of novel objects can be used to assign internal representations sensitive to these causal properties. Coherent covariation of causal powers with other object properties within the domain of artifacts leads them to be treated as more central than superficial appearance properties. Item Context (External and Internal) Sequelae Conclusions:  Conclusions Sensitivity to coherent co-variation underlies conceptual development and the emergence of causal knowledge in young children. A domain general, gradual learning system, embedded in a complementary learning systems architecture, can provide the mechanism necessary for conceptual elaboration, inference, and episodic memory shaped by conceptual content. Of course we have a long way to go before we can say we’ve proven these points. Overview:  Overview The complementary learning systems perspective. Differentiation and reorganization of conceptual knowledge in development. Inference and Episodic Memory. Domain specificity. Causal inference in children. Complementary Learning Systems (McClelland, McNaughton and O’Reilly, 1995):  Complementary Learning Systems (McClelland, McNaughton and O’Reilly, 1995) Memory task performance depends on multiple contributing brain systems. Contributions of components to overall task performance depend on their neuro-mechanistic properties. Components work together so that overall performance may be better than the sum of the independent contributions of the parts. Neuropsychological Findings Motivating the Approach:  Neuropsychological Findings Motivating the Approach Hippocampal lesions produce dramatic deficits in certain types of memory tasks: Recallable memory for specific individuals and events Paired associate learning with arbitrary materials Arbitrary new factual information But in many tasks learning appears to be unaffected: Skill acquisition tasks such as mirror reading Item priming effects Hippocampal lesions also produce temporally graded retrograde amnesia: Information acquired shortly before lesion is more vulnerable than information acquired earlier. Slide60:  Time from experience to lesion in days Lesioned groups Control groups Retention Index Complementary Learning System I: Processing and learning in neocortex:  Complementary Learning System I: Processing and learning in neocortex An input and a response to it result in activation distributed across many areas in the neocortex. Small connection weight changes occur, producing Subtle item-specific effects Gradual skill acquisition Gradual acquisition of conceptual and causal knowledge. These small changes are not sufficient to support rapid acquisition of arbitrary new associations. Complementary Learning System 2: Fast Learning System in the Hippocampus:  Complementary Learning System 2: Fast Learning System in the Hippocampus Bi-directional connections produce a reduced description of the cortical pattern in the hippocampus. Large connection weight changes bind bits of reduced description together Cued recall depends on pattern completion within the hippocampal network Consolidation occurs through repeated reactivation, leading to cumulation of small changes in cortex.

Add a comment

Related presentations

Related pages

David McClelland – Wikipedia

David Clarence McClelland (* 20. Mai 1917; † 27. März 1998) war ein US-amerikanischer Verhaltens-und Sozialpsychologe und ein Vertreter der quantitiven ...
Read more

David McClelland - Wikipedia, the free encyclopedia

David Clarence McClelland (May 20, 1917 – March 27, 1998) was an American psychologist, noted for his work on motivation Need Theory. He published a ...
Read more

Wirtschaftspsychologische Gesellschaft ...

McClelland unterscheidet drei zentrale Motivgruppen, bei denen Menschen sich stark unterscheiden: Leistungsmotivation beschreibt den Antrieb Erfolg zu ...
Read more

McClelland – Wikipedia

McClelland ist der Familienname folgender Personen: Bramlette McClelland (1920–2010), US-amerikanischer Geotechniker; Charles McClelland (Cutter), Cutter ...
Read more

Human Motivation: David McClelland ...

This is an advanced, technical work written by a leading theoretician. McClelland examines motivation in the personality and behaviorist traditions.
Read more

McClelland - Wikipedia, the free encyclopedia

McClelland is a surname. Notable people with the surname include: Alyssa McClelland, Australian actress; Charles A. McClelland (born 1917), American ...
Read more

Kristin McClelland | LinkedIn

Kristin McClelland. Distributor at Juice PLUS+ International franchise. Ort Leipzig und Umgebung, Deutschland Branche Gesundheit, Wellness & Fitness
Read more

David McClelland | LinkedIn

Werden Sie Mitglied von LinkedIn und erhalten Sie Zugriff auf das vollständige Profil von David McClelland. Völlig kostenlos! Als Mitglied von LinkedIn ...
Read more

McClelland Tobacco Company | Matured Virginias and ...

The McClelland Tobacco Company specializes in manufacturing the highest quality Matured Virginias and Oriental Mixtures for tobacco smoking pipes.
Read more

McClelland's Theory of Needs - NetMBA Business Knowledge ...

Management > McClelland. McClelland's Theory of Needs. In his acquired-needs theory, David McClelland proposed that an individual's specific needs are ...
Read more