Harvey Mudd College

Computer Science 152

Neural Networks

Fall 2007

• Neocognitron (Anton Bakalov)
• Cribbage (Corey Hebert)
• A Better Exit Strategy (Devin Smith)
• Adaptive Parsing (Adrian Sampson)
• Reinforcement Learning Applied to Othello (George Tucker)
• SVM by SMO (Erik Kuefler)

• General information and outline
  
  
• Textbook Link
  
  
• Matlab - Getting Started
  
  
• MatLab NN Toolbox Manual (html)
• MatLab NN Toolbox Manual (pdf)
  
• Slides from a prior offering of this course (2004)


• Links to some previous term projects
  
  
• Perceptrons
  • Perceptron Demo 1
  • Perceptron Demo 2
    
  
  
• Adaptive Filtering
  • Neural Nets for Adaptive Filtering ..." (Widrow and Winter paper)
  • Adaptive Noise Cancellation Demo (Explanation)
  • Echo cancellation method comparisons (NLMS, RLS, ...)
  • Echo Cancellation Demo
  • Introduction to Adaptive Filters (web course by Douglas L. Jones)
  
  
• Backpropagation
  • Multivariate chain rule
  • Fahlman's Quickprop paper (in CiteSeer)
  • Short description of Rprop (Resilient Backpropagation, from SNNS)
  • Original Rprop paper (Riedmiller and Braun, in CiteSeer)
  • Rprop Implementation Details(Riedmiller, in CiteSeer)
  • Sarprop paper (Treadgold and Gedeon, in CiteSeer)
  • Backpropagation Applications
    • NetTalk (Senjowski and Rosenberg)
    • LeNet-5, convolutional networks
• Various Applications
  • Various links
  
  
• Time-Based Approaches and Applications
  • Recurrent Network Slides
  • Notes on Recurrent Networks (Genevieve Orr, Willamette University)
  • Temporal Difference Slides
  • Guide to recurrent neural networks and backpropagation
  • Truck-Backer paper (Widrow and Nguyen, IEEE Xplore)
  • Backpropagation Through Time (BPTT, Werbos article)
  • Applying BPTT to Inverted Pendulum
  • BPTT paper (Jesus and Hagan, with pseudo-code)
  • Werbos presentation on Control Theory and Related Techniques (1988)
  • BPTT and DAC (Derivative Adaptive Critic
  • Arabic speech recognition article (BPTT application)
  • Adaptive Heuristic Critic (based on TD)
  • Black Jack and Reinforcement Learning
  • Temporal Difference Learning and TD-Gammon (Tesauro)
  
  
• Radial Basis Functions (RBFs)
  • Brief Description (SNNS)
  • wikipedia article
  • Example code (matlab)
  • Learning rules (SNNS)
  • FastRBF Toolbox (Commercial, FarField Technology)
  
  
• (wikipedia)
• Linear Least Squares (wikipedia)
  
• k-Means Clustering
  • Demo, with Voronoi regions
  
  
• Support Vector Machines (SVMs)
  • SVMs vs NNs
  • Support Vector Machines: Hype or Hallelujah? (Bennett and Campbell)
  • Wikipedia article
  • Demo applet (SmartLab)
  • Demo (AT&T Labs)
  • Classification Application Demo (Quantio)
  • Materials for Support Vector Machine book (tutorials, etc., John Shawe-Taylor & Nello Cristianini )
  • www.kernel-machines.org
  • Learning with Kernels book (figures, etc., Bernhard Scholkopf and Alex Smola)
  • Software Links
  • Classified links, etc.
  • Lagrange Multipliers
    • Lagrange Multipliers (Wikipedia article)
    • Demo (MIT)
    • Lagrange Multipliers without Permanent Scarring (Dan Klein)
  • Quadratic Programming
    • Wikipedia article
    • Matlab Quadratic Program Solver
  • KKT Conditions
  • NeuroCOLT Archive (Neural Networks and Computational Learning Theory)
  • SVM Application links
  
  
• Unsupervised Learning
  • Wikipedia article
  • Clustering package (Matlab)
  
  
• SOM (Self-Organizing Maps)
  • SOM Toolbox (Matlab)
  
  
• Student Paper Presentations
  • Neocognitron, by Anton Bakalov
  • Cribbage Player, by Corey Hebert
  • Relevance Vector Machines, by Erik Kuefler
  • Finding Shortest Paths in Real Time, by Adrian Sampson
  • CTRNNs for the Dowry Problem, by Devin Smith
  • Reinforcement Learning in Board Games, by George Tucker
  • Paper review example: Wan and Beaufays paper
  
  
• ART (Adaptive Resonance Theory)
  • Wikipedia entry (ART)
  • Tutorial (T. Tanaka and A. Weitzenfeld)
  • ART clearinghouse
  • Category ART (David Weenink)
  
  
• Hopfield Nets
  • Demo
  • Noisy recall demo
  • 10 x 10 demo
  • TSP demo, showing net activations (note the possibility of local minima)
  
  
• Simulated Annealing
  • Wikipedia entry
  • SA demo for TSP
  • 8x8 Ising demo
  • Finer-grain Ising demo
  • Statistical Physics slides
  • Spin-glass tutorial
  
  
• Boltzmann Machine
  • Revised lecture slides
  • Wikipedia entry
  • Scholarpedia entry
  • Necker Cube Example
  • Boltzmann's Equation
  • Restricted Boltzmann Machines, Sigmoid Belief Networks (Hinton)
  
  
• PCA
  • Lecture slides
  • Correlation (Wikipedia)
  • Wikipedia entry
  • Reducing the Dimensionality of Data with Neural Networks (Hinton and Salakhutdinov)
  • The Talented Dr. Hebb, Part 2 - PCA
  • David Gleich's PCA/ICA presentation
  • Demo obtaining the first principal component using APEX neural network
  • PCA Applets
  
  
• ICA
  • ICA Central
  • ICA explanation (Helsinki University of Technology)
  • J.V. Stone's page (book, Matlab code)
  • ICA Tutorial (Helsinki University of Technology)
  • FastICA package (Matlab and others, Helsinki University of Technology)
  • Square root of a matrix
  • Blind Source Separation demo (Te-Won Lee)
  • A course on ICA
  • Cocktail Party demo (Helsinki University of Technology)
  • Video processing demos (Jiminy Peak)
  
  
• Fuzzy Logic
  • Lecture slides
  • Fuzzy Pendulum Demo
  • Michael Athans' extended critique
  
  
• Evolutionary Computation
  • Lecture Slides
  • Pagentree (Parallel Genetic Decision Trees) (Keller, Bentow, Malone)
  • Genetic Algorithms and Evolutionary Computation (Adam Marczyk)
  • Genetic Algorithms Overview
  • GP Symbolic Regression
  • GP Truck-backer
  • SANE Pole Balancer
  
  
• Physiological References
  • The Primary Visual Cortex
  • Lateral inhibition
  • Spiking Neuron Models
  • Olson and Grossberg ("instar")
  
  
• Miscellaneous
  • Inverted Pendulum controlled by Matlab (student project video)
  • Inverted pendulum on a toy car (student project video)
  • Inverted Pendulum control equations
  • Emergent (rewrite of PDP++)
  • Face Recognition Homepage
  • Brief survey of neural network varieties (html) (MLP, RBF, RNN, Jordan-Elman, Hopfield, BSB, SOM, and PCA)
  • The Talented Dr. Hebb, Part 1 - Novelty Filtering
  • Classifier Showdown (Peltarion)
  • Various Tutorials (Peltarion)
  
  
• Lists of Links
  • Applets
  • Software and Datasets