HMC CS 152, Neural Networks

Fall 2012

Instructor: Prof. Robert M. Keller, 1253 Olin, Office Hours: MTW 2:45-4:00 pm

Text

Thomas J. Anastasio, Tutorial on Neural Systems Modeling, Sinauer (2010), ISBN 978-0-87893-339-6

(Errata for the Text)

Final Project Info (Due Tue. Nov 6, Tue. Nov. 20, Thurs. Dec. 13)

Final Projects

Co-evolution of a Neural Network Reversi Player by Joe Agajanian and Sean Campbell

Pattern Recognition with Neural Networks by Natasha Parikh and Miranda Parker

Neural FreeCell Player by Alex Eng

Testing the Efficient-Market Hypothesis using Neural Networks by Matthew Prince

Voice Identification by Lisa Gai and Ravi Kumar

OCR for Mathematical Expressions by Alistair Dobke and Mark Mann

Twitter Mood by Frank Liu and Colin Bundschu

Phishing Detection by Josh Oratz

Recurrent Networks and Backpropagation Through Time: A Cautionary Tale by John Wentworth

Instrument Identification Using Feed Forward Networks by Zachary Gaslowitz

Facial Expression Recognition by Erin Coughlan and Vivian Wehner

Email Spam Detection by Kevin Vigue

Noise Cancellation by David Ersek

Characterizing whether a twitter poster is likely sober or drunk by Matthew Toal

Assignments

Assignment 1 (Due Wed. 12 September)

Presentation Articles (for Assignment 1)

Assignment 2 (Due Wed. 19 September)

Starter Code and Data for Assignment 2

Assignment 3 (Due Wed. 26 September)

Assignment 4 (Due Tues. 7 October)

Assignment 5

Assignment 5 files

Assignment 6 (Due Wed. 31 October)

Lecture Slides

Introduction (2-up version)

Perceptrons (2-up version) (6-up version)

Difference Equations Recurrent Networks (2-up version)

Adalines (2-up version)

Real-Time Adalines (2-up version)

Lateral Inhibition Networks

Radial Basis Function Networks

Hebbian Learning (2-up version)

Hopfield Networks (2-up version)

Competitive Learning (2-up version)

Self-Organizing Maps (2-up version)

Slides from McLennan

ART Networks (2-up version)

Backpropagation (2-up version)

Backpropagation Applications (2-up version)

Backpropagation Tricks (2-up version)

Backpropagation Variations (2-up version)

Reinforcement Learning (2-up version)

Temporal Differences (2-up version)

Backpropagation through Time (2-up version)

Real-Time Recurrent Learning (2-up version)

Fuzzy Logic (2-up version)

Support Vector Machines (2-up version)

Boltzmann Machines (2-up version)

PCA and ICA

Reinforcement Learning videos (each 1-3 mins)

• 1. Introduction
• 2. Successes
• 3. Forms of Learning
• 4. Forms of Learning Question
• 5. Forms of Learning Answer
• 6. MDP Review
• 7. Solving a MDP
• 8. Agents of Reinforcement Learning
• 9. Passive vs. Active
• 10. Passive Temporal Difference
• 11. Passive Agent Results
• 12. Weaknesses Question
• 13. Weaknesses Answers
• 14. Active Reinforcement Learning
• 15. Greedy Agent Results
• 16. Balancing Policy
• 17. Errors in Utility Questions
• 18. Errors in Utility Answers
• 19. Exploration Agents
• 20. Exploration Agent Results
• 21. Q Learning 1
• 22. Q Learning 2
• 23. Pacman 1
• 24. Pacman 2
• 25. Conclusion

Overview

This year, I am attempting to change the course somewhat from prior offerings, to relate it more to biological neural models. Ideally, this will be done while retaining much of the previous material having to do with the use of neural networks to solve various AI and machine learning problems. The present outline represents an integration underway that combines more computer-science oriented material with the presentation in the text.

Outline by Topic

Headings with chapter numbers refer to the text.

* means that supplementary material will be used.

Chapter 1. Basic Neural Computations
The brain is the most complex organ known to exist, yet simple mathematical and computer programming methods can be used to simulate many neural systems.

• Neural Systems, Neural Networks, and Brain Function
• Operation and Habituation of the Gill Withdrawal Reflex of Aplysia
• The Dynamics of a Single Neural Unit with Positive Feedback
• Neural Networks: Neural Systems with Multiple Interconnected Units

 

* Perceptron Model [reference: http://page.mi.fu-berlin.de/rojas/neural/chapter/K4.pdf]
Perceptrons were the first formalized model, introduced to show how an artificial neural network could recognize visual patterns.

 

Chapter 2. Recurrent Connections and Simple Neural Circuits
Small networks with recurrent connections, forming circuits, can shape signals in time, produce oscillations, and simulate neural systems involved in low-level motor control.

• The Dynamics of Two Neural Units with Feedback in Series
• Signal Processing in the Vestibulo-Ocular Reflex (VOR)
• Primate VOR
• The Parallel Pathway Model of Velocity Storage
• The Positive Feedback Model of Velocity Storage
• Pigeon VOR: The Negative Feedback Model of Velocity Leakage
• Oculomotor Neural Integration via Reciprocal Inhibition
• Simulating the Insect Flight Central Pattern Generator

 

* Adaline Model [references: http://en.wikipedia.org/wiki/ADALINE, http://www-isl.stanford.edu/~widrow/papers/j1992feedforwardnetworks.pdf]
Adalines are a model similar to perceptrons, but with a different training method. They are extendable to general feed-forward networks, also called “multi-level perceptrons, which are much more powerful than the basic adaline or perceptron model.

 

Chapter 3. Forward and Recurrent Lateral Inhibition
Networks with forward and recurrent laterally inhibitory connectivity profiles can shape signals in space and time and simulate certain forms of sensory and motor processing.

• Simulating Edge Detection in the Early Visual System of Limulus
• Simulating Center/Surround Receptive Fields Using the Difference of Gaussians
• Simulating Activity Bubbles and Stable Pattern Formation
• Separating Signals from Noise and Modeling Target Selection in the Superior Colliculus

 

* Radial Basis Function Networks
Radial Basis Function networks are 2-layer networks based on certain characteristics of a visual field. They generalize to the very powerful Support Vector Machine model.

 

* Feed-forward Associative Learning [reference: http://www.cs.hmc.edu/courses/2010/fall/cs152/HebbianAssociative.pdf]

This model shows how memory can be constructed based purely on synaptic weights.

 

* Hopfield Networks [reference: http://www.scholarpedia.org/article/Hopfield_network]

Chapter 4. Covariation Learning and Auto-Associative Memory
Networks with recurrent connection weights that reflect the covariation between pattern elements can dynamically recall those patterns and simulate certain forms of memory.

• Four Hebbian Learning Rules for Neural Networks
• Simulating Memory Recall Using Recurrent Auto-Associator Networks
• Recalling Distinct Memories Using Negative Connections in Auto-Associators
• Synchronous versus Asynchronous Updating in Recurrent Auto-Associators
• Graceful Degradation and Simulated Forgetting
• Simulating Storage and Recall of a Sequence of Patterns
• Hebbian Learning, Recurrent Auto-Association, and Models of Hippocampus

 

Boltzmann Machines, Training by Correlation [reference: http://www.scholarpedia.org/article/Boltzmann_machine]
Boltzmann machines provide one biologically-plausible way to add learning to the Hopfield model.

 

Chapter 5. Unsupervised Learning and Distributed Representations
Unsupervised learning algorithms, given only a set of input patterns, can train neural networks to form distributed representations of those patterns that resemble brain maps.

• Learning through Competition to Specialize for Specific Inputs
• Training Few Output Neurons to Represent Many Input Patterns
• Simulating the Formation of Brain Maps using Cooperative Mechanisms
• Modeling the Formation of Tonotopic Maps in the Auditory System
• Simulating the Development of Orientation Selectivity in Visual Cortex
• Modeling a Possible Multisensory Map in the Superior Colliculus

 

* Self-Organizing Maps (Kohonen Maps) [reference: http://www.scholarpedia.org/article/Kohonen_network]

 

 

Chapter 6. Supervised Learning and Non-Uniform Representations
Supervised learning algorithms can train neural networks to associate patterns and simulate the non-uniform distributed representations found in many brain regions.

• Using the Classic Hebb Rule to Learn a Simple Labeled Line Response
• Learning a Simple Contingency Using the Covariation Rule
• Using the Delta rule to Learn a Complex Contingency
• Learning Interneuronal Representations using Back-Propagation
• Simulating Catastrophic Retroactive Interference in Learning
• Simulating the Development of Non-Uniform Distributed Representations
• Modeling Non-Uniform Distributed Representations in the Vestibular Nuclei

 

* Back-Propagation Enhancements and Variations

 

* Support-Vector Machines

 

Chapter 7. Reinforcement Learning and Associative Conditioning
Reinforcement learning algorithms can simulate certain forms of associative conditioning and can train networks to develop non-uniform distributed representations.

• Learning the Labeled-Line Task via Perturbation of One Weight at a Time
• Perturbing All Weights Simultaneously and the Importance of Structure
• Plausible Weight Modification using Perturbative Reinforcement Learning
• Reinforcement Learning and Non-Uniform Distributed Representations
• Reinforcement in a Schema Model of Avoidance Conditioning
• Exploration and Exploitation in a Model of Avoidance Conditioning

 

Chapter 8. Information Transmission and Unsupervised Learning
Unsupervised learning algorithms can train neural networks to increase the amount of information they contain about the input and simulate the properties of sensory neurons.

• Some Basic Concepts in Information Theory
• Measuring Information Transmission through a Neural Network
• Maximizing Information Transmission in a Neural Network
• Information Transmission and Competitive Learning in Neural Networks
• Information Transmission in Self-Organizing Map Networks
• Information Transmission in Stochastic Neural Networks

 

* Adaptive Resonance Theory (ART)

 

Chapter 9. Probability Estimation and Supervised Learning
Supervised learning algorithms can train neural units and networks to estimate probabilities and simulate the responses of neurons to multisensory stimulation.

• Implementing a Simple Classifier as a Three-Layered Neural Network
• Predicting Rain as an Everyday Example of Probabilistic Inference
• Implementing a Simple Classifier Using Bayes’ Rule
• Modeling Neural Responses to Sensory Input as Probabilistic Inference
• Modeling Multisensory Collicular Neurons as Probability Estimators

 

Chapter 10. Time-Series Learning and Nonlinear Signal Processing
Supervised learning through time can train neural networks to produce dynamic transformations and simulate certain forms of motor control and short-term memory.

• Training Connection Weights in Nonlinear Recurrent Neural Networks
• Training a Two-unit Network to Simulate the Oculomotor Neural Integrator
• Velocity Storage in the Primate VOR
• Training a Network of Linear Units to produce Velocity Storage
• Training Networks of Nonlinear Units to Produce Velocity Storage
• Training a Recurrent Neural Network to Simulate Short-Term Memory

 

* Back-Propagation through Time

 

* FIR (Finite-Impulse Response) Back-Propagation Networks

 

Chapter 11. Temporal-Difference Learning and Reward Prediction
Temporal-difference learning can train neural networks to estimate the future value of a current state and simulate the responses of neurons involved in reward processing.

• Learning State Values using Iterative Dynamic Programming
• Learning State Values using Least Mean Squares
• Learning State Values using the Method of Temporal Differences
• Simulating Dopamine Neuron Responses using Temporal Difference Learning

 

* Game-Playing Using Temporal Differences

 

 

Chapter 12. Predictor-Corrector Models and Probabilistic Inference
Predictor-corrector models can improve perception by combining internal expectations with sensory observations and simulate the responses of certain sensory neurons.

• Modeling Visual System Direction Selectivity using Asymmetric Inhibition
• Modeling Visual Processing as Bottom-Up/Top-Down Probabilistic Inference
• A Predictor-Corrector Model of Predictive Tracking by Midbrain Neurons
• Training a Sigmoidal Unit to Simulate Trajectory Prediction by Neurons

 

Chapter 13. The Genetic Algorithm and Simulated Evolution (time permitting)
The genetic algorithm simulates the process of evolution and can be used to optimize the structure, connectivity, and adaptability of neural systems.

• Simulating Genes and Genetic Operators
• Exploring a Simple Example of Simulated Genetic Evolution
• Evolving the Sizes of Neural Networks to Improve Learning
• Evolving Optimal Learning Rules for Auto-Associative Memories
• Evolving Connectivity Profiles for Activity-Bubble Neural Networks

 

Chapter 14. Future Directions in Neural Systems Modeling
In the future, neural systems models will become increasingly complex and will span levels from molecular interactions within units to interactions between networks.

• Neuro-Informatics and Molecular Networks
• Enhanced Learning in Neural Networks with Smart Synapses
• Combining Complementary Network Paradigms for Memory Formation
• Smart Synapses and Complementary Rules in Cerebellar Learning

 

* Fuzzy Logic (time permitting)

 

* Artificial Life (time permitting)