HMC CS 152, Fall 2008

• Student Projects
  • Eric Doi (Language Classification)
  • James Wendt, Ragib Morshed (TD-Mancala)
  • Greg Bickerman (Mancala Learning via Temporal Difference)
  • Dmitriy Yakovlev (Speaker Recognition)
• Syllabus
• Introduction, Perceptrons
• A Brief History of Connectionism
• Neural Nets FAQ
• Spiking Neuron Models
• Adalines
• Gradient Descent
• Adaline slides from NND
• Adaptive Filter (wikipedia)
• LMS Filter (wikipedia)
• RLS Filter (wikipedia)
• Adaptive Noise Cancellation Demo
• Backpropagation
  • Backprop slides from NND
  • More detailed backprop slides
  • Six Lectures on Backpropagation (Leslie Smith)
  • Backprop applications
  • Backprop Techniques and Tips
  • Backpropagator's Review (Donald Tveter)
  • The Pattern Recognition Basis of Artificial Intelligence (Donald Tveter)
    • Coding Data for Character Recognition
  • Methods to Speed Up Backpropagation (Dilip Sarkar, ACM Computing Surveys)
  • Optimization with Numerical Methods
  • Backprop Variations
  • LeNet-5, Convolutional Neural Networks (LeCun Demos and Papers)
  • NetTalk (Senjowski and Rosenberg, Dataset)
  • Multivariate chain rule
  • Fahlman's Quickprop paper (in CiteSeer)
  • Short description of Rprop (Resilient Backpropagation, from SNNS)
  • Original Rprop paper (Riedmiller and Braun, in CiteSeer)
  • Rprop Implementation Details(Riedmiller, in CiteSeer)
  • Sarprop paper (Treadgold and Gedeon, in CiteSeer)
  
  
• Radial Basis Functions
  • Lecture slides
  • Improving the Performance of K-Means Clustering Algorithm to Position the Centres of RBF Network
  • 3D Object Recognition demo
  
  
• Moore-Penrose Pseudo-Inverse
• Linear Regression
  
• Time-Based Approaches and Applications
  • Lecture slides
  • Guide to recurrent neural networks and backpropagation
  • Truck-Backer paper (Widrow and Nguyen, IEEE Xplore)
  • Backpropagation Through Time (BPTT, Werbos article)
  • Applying BPTT to Inverted Pendulum
  • BPTT paper (Jesus and Hagan, with pseudo-code)
  • Werbos presentation on Control Theory and Related Techniques (1988)
  • BPTT and DAC (Derivative Adaptive Critic)
  • Arabic speech recognition article (BPTT application)
  • Adaptive Heuristic Critic (based on TD)
  • Black Jack and Reinforcement Learning
  • TD Learning Project Applets
  • Temporal difference learning (Barto, scholarpedia)
  • Reinforcement learning (Woergoetter Porr, scholarpedia)
  • Reinforcement Learning: An Introduction (Sutton and Barto)
    • Tic-Tac-Toe example
    • Code
  • NeuroGo
  
  
• Radial Basis Functions (RBFs)
  • Brief Description (SNNS)
  • Example code (matlab)
  • Learning rules (SNNS)
  • FastRBF Toolbox (Commercial, FarField Technology)
  
  
• Function Minimization GA Demo
• Examples of function minimization
• Graphic TSP Demo
• GAs for NNs
• Truck backer using Genetic Programming
  
• k-Means Clustering
  • Demo, with Voronoi regions
  
  
• Competitive Networks
• Self-Organizing Maps
• SOM demos (1D and 2D)
• 3D Map demo (TSP, Jochen Froehlich)
• GNG Demo (Growing Neural Gas)
• Some Competitive Learning Methods (Bernd Fritzke)
• SOM Number Recognition Demo
• Bayesian SOM demo
• Data exploration using self-organizing maps
• GHSOM Toolbox
• ART Networks
• Adaptive Resonance Theory (wikipedia)
• Hebbian Learning and Associative Networks
• Hopfield Nets
  • Hopfield's original paper
  • Hopfield Network (scholarpedia, by John Hopfield)
  • Hopfield net (wikipedia)
  • Energy Surface
  • Notes on Associative, Hopfield, and BAM Networks
  • Hopfield net demo (Heaton Research)
  • Bit-map Hopfield demo Hopfield Demo showing iterations (Pong, shows updating)
  • Another Hopfield demo
  • Hopfield demo, with pattern overlap indicator
  • Hopfield and BAM demos (Java failure?)
  • Hopfield TSP demo
• Boltzmann Machines
  • Boltzmann machine (wikipedia)
  • Boltzmann Machine (scholarpedia, by Geoffrey Hinton)
  • Reducing the Dimensionality of Data with Neural Networks (G. E. Hinton and R. R. Salakhutdinov)
  • TSP via Simulated Annealing
  • Ising Model Applet
  • Another Ising Model
  • Yet Another Ising Model
  • The Next Generation of Neural Networks
  • Restricted Boltzmann Machines for Collaborative Filtering
  • Machine Learning with PLearn
• PCA, ICA, etc.
  • PCA slides
  • PCA demos
  • FERET Photobook
  • Eigenface Tutorial
  • ICA introduction
  • Cocktail Party Problem (Helsinki University)
  • Blind source separation (
  • Fast ICA Package
  • ICA Survey paper
  • ICA Links (Tony Bell)
• SVM (Support Vector Machines)
  • wikipedia article
  • SVM demo (AT&T)
  • libsvm (includes demo)
    Tools Hsuan-Tien Lin's webpage Datasets
  • Version Space applet (G. Gordon)
  • List of other software
  • SVM Tool (tagging parts of speech)
  • Matlab SVM Toolbox
  • support-vector-machines.org
  • Lagrange Multiplier Applet
  • Kernel PCS
• Fuzzy Logic slides
• Fuzzy Logic (wikipedia)
• Evolutionary Computation (slides)
  • Genetic Algorithms (wikipedia)
  • Genetic Route Planner demo
  • Memetic Algorithms (wikipedia)
  • Genetic Programming (wikipedia)
  • Pagentree
  • Functional genetic programming with combinators (Briggs and O'Neill)
  • Field Guide to GPArtificial Life (wikipedia)
  • GenePooL
  • Flies
  • Driving GA
  • Zooland (links)
  
  
• Statistical Pattern Recognition Tool (SVM, PCA, ...)
• Information Retrieval (applies SVM's, etc.)
• Data Mining Software Links
• Matlab NN Toolbox
• Lens: Light, Efficient Neural Network Simulator
• Sebastian Seung's Course
• Retina, LGN
• Visual Cortex
• BCM Theory

• Miscellaneous
  • UCI Machine Learning Repository
  • Inverted Pendulum controlled by Matlab (student project video)
  • Inverted pendulum on a toy car (student project video)
  • Inverted Pendulum control equations
  • Emergent (rewrite of PDP++)
  • Face Recognition Homepage
  • Brief survey of neural network varieties (html) (MLP, RBF, RNN, Jordan-Elman, Hopfield, BSB, SOM, and PCA)
  • The Talented Dr. Hebb, Part 1 - Novelty Filtering
  • Classifier Showdown (Peltarion)
  • Various Tutorials (Peltarion)
  • JOONE Java Object-Oriented Neural Net Engine
  
  
• Lists of Links
  • Applets
  • Software and Datasets