Humans accumulate knowledge and abilities that serve as building blocks for subsequent development. Such layered or sequential learning appears to be an essential mechanism, both in acquiring useful abstractions that serve intelligent behavior, and in producing essential new foundations for further development. For this seminar, we eschew single task learning, which dominates much of current research in machine learning, in favor of sustained learning of indefinitely many tasks, which accounts for ever increasing capabilites in human learning. We will start with the study of Gagne's psychological theory of cumulative learning and its practical implications for machine learning. We will then survey work that explores how learning systems can acquire the deeply layered knowledge that is necessary for intelligent systems.
The seminar is oriented toward reading and discussion, but a project is possible for anyone with such interest. For each paper that we read, a written one page (max) analysis/synopsis is required at the class in which the paper is first discussed. There will be a final exam, but no midterm.
This seminar is being co-taught by Profs Paul Utgoff and Rod Grupen.
Come to these regular hours, or schedule an appointment:
Classes are Tuesdays and Thursdays 11:15-12:30 AM, in LGRC A311 (low-rise).
You need to have background in Artificial Intelligence, preferably cmpsci 683. Speak to one of the instructors for permission to enroll in the seminar if you have background other than cmpsci 683.
This schedule lists the assigned reading and other information for each class.
Introduction, course overview.
Discuss: Many-Layered Learning, Utgoff, P. and Stracuzzi, D., Neural Computation, October 14(10):2497-2529, 2002. Reprint available. [Leader: Paul Utgoff]
Discuss: Trading spaces: Computational, representation, and the limits of uninformed learning, Clark, A. and Thornton, C. (1997), Behavioral and Brain Sciences, vol 20, pp 57-90. [Leader: Paul Utgoff]
Discuss: Learning and Transferring Action Schemas, Paul R. Cohen, Yu-Han Chang. Clayton T. Morrison. The Twentieth International Joint Conference on Artificial Intelligence (IJCAI 2007). 2007. [Leader: Rod Grupen]
Discuss: Intellectual Skills: Defined Concepts and Rules, in The Conditions of Learning and Theory of Instruction, Gagne, Robert, M., Fourth Edition (1985), Holt, Rinehart and Winston. [Leader: Paul Utgoff]
Discuss: Schema Learning: Experience-Based Construction of Predictive Action Models Holmes and Isbell, NIPS 2005. [Leader: Chris Vigorito]
Quadrupedal Walking Gaits Rod Grupen, Chapter 9, The Developmental Organization of Dexterous Robot Behavior," MIT Press, 2008. [Leader: Tulsi Vembu]
Greedy Layer-Wise Training of Deep Networks Bengio, Lamblin, Popovici, Larochelle (NIPS 06). [Leader: David Cooper]
The neural basis of cognitive development: A constructivist manifesto, Quartz, S.R. and Sejnowski, T.J. (1997), Behavioral and Brain Sciences, vol 20, pp 537-596. [Leader: Roberto Olivares]
Effective Control Knowledge Transfer Through Learning Skill and Representation Hierarchies , Manfred Huber et al. [Leader: Jacqueline Kenney]
Semiotic Schemas: A Framework for Grounding Language in Action and Perception Deb Roy, AI Journal, 2005. [Leader: Rod Grupen]
A Fast Learning Algorithm for Deep Belief Nets Hinton, Osindero, Teh (Neural Computation 06) [Leader: William Dabney]
An Emergent Framework for Self-Motivation in Developmental Robotics, Marshall, J., Blank, D., and Meeden, L. (2004). Proceedings of the Third International Conference on Development and Learning, Salk Institute for Biological Studies, La Jolla, California. [Leader: Tulsi Vembu]
Bootstrap Learning of Foundational Representations, Kuipers, Benjamin; Beeson, Patrick; Modayil, Joseph; Provost, Jefferson, Connection Science, Volume 18, Number 2, June 2006, pp. 145-158(14). [Leader: Oscar Loureiro]
Modeling the Neural Basis of Cognitive Development, Quartz, S.R. (2003), in Modeling Neural Development, van Ooyen, A. (ed), pp 291-313, MIT Press. [Leader: Roberto Olivares]
Bringing up robot: Fundamental mechanisms for creating a self-motivated, self-organizing architecture. Blank, D.S., Kumar, D., Meeden, L., and Marshall, J. (2005), Cybernetics and Systems, 36(2). [Leader: Patrick Deegan]
Everyone should read: Bootstrapping Grounded Word Semantics Steeles, Kaplan
As time permits, you may wish to look also at The Origins of Syntax in Visually Grounded Robotic Agents Luc Steels [Leader: Phil Kirlin]
The Basal Ganglia and Chunking of Action Repertoires Neurology of Learning and Memory, 70, 119-136, 1998. [Leader: ------------------]
Pfleger, K. On-Line Learning of Predictive Compositional Hierarchies by Hebbian Chunking [Leader: ------------]
Karmiloff-Smith, A. Precis of Beyond modularity: A developmental perspective on cognitive science, 1994, Behavioral and Brain Sciences, vol 17, pp 693-745. [Leaders: Chris Vigorito and David Cooper]
Intrinsic Motivation for Reinforcement Learning Systems, Barto, A.G. and Simsek, O., Proceedings of the Thirteenth Yale Workshop on Adaptive and Learning Systems, 2005. [Leader: Oscar Loureiro]
Sequence Learning with Incremental Higher-Order Neural Networks Mark Ring, AI 93-193, 1993. [Leader: Phil Kirlin]
What Should a Robot Learn from an Infant? Mechanisms of Action Interpretation and Observational Learning in Infancy Gergely, G. [Leader: Jacqueline Kenney]
Vygotsky, L.S. (1987). Section 4 of Chapter 6 of Volume 1 of The Collected Works of L.S. Vygotsky, edited by R.W. Rieber and A.S. Caron, Plenum Press. [Leader: -------------]
Hierarchical Temporal Memory Concepts, Theory, and Terminology Jeff Hawkins and Dileep George. [Leader: Chris Vigorito]
Robotic Grasping of Novel Objects Ashutosh Saxena, Justin Driemeyer, Justin Kearns, Andrew Ng. In NIPS 19, 2006. [Leader: Dov Katz]
Building Portable Options: Skill Transfer in Reinforcement Learning Konidaris et. al. [Leader: Steve Murtagh]
Main themes, wrap-up.
The final exam is in CMPS 140, 10:30AM.
This is a pool of other articles. It looks like we may not get to these.
You have an EdLab account, but you will probably not need it for this seminar. If this is a new account (you have never had an EdLab account before), then your password is your 8-digit ID number.
Formulating your 1-page synopsis provides good preparation for the discussion. It is not useful to turn these in late.
The written synopses and active participation in the discussions are most important.
Attribute correctly the source of ideas that you borrow from others. If you are in doubt, feel free to ask.
Many of the materials created for this course are the intellectual property of the instructors. This includes, but is not limited to, the syllabus, lectures and course notes. Except to the extent not protected by copyright law, any use, distribution or sale of such materials requires the permission of the instructor. Please be aware that it is a violation of university policy to reproduce, for distribution or sale, class lectures or class notes, unless copyright has been explicity waived by the faculty member.