Catalog Description

Modeling, simulation, and analysis of artificial neural networks. Relationship to biological neural networks. Design and optimization of discrete and continuous neural networks. Backpropagation, and other gradient descent methods. Hopfield and Boltzmann networks. Unsupervised learning. Self-organizing feature maps. Applications chosen from function approximation, signal processing, control, computer graphics, pattern recognition, time-series analysis. Relationship to fuzzy logic, genetic algorithms, and artificial life.

Prerequisites: CS 60 and Mathematics 73, or permission of the instructor. 3 credit hours.

Instructor

• Bob Keller 1242 Olin (3-4 p.m. MW; 4-5 p.m. Tu or by drop-in or appt.), keller@cs.hmc.edu, x 18483
  
  

Texts

• Main Textbook:

  (abbreviated NAS): Jose C. Principe, Neil R. Euliano, and W. Curt Lefebvre, Neural and Adaptive Systems, John Wiley & Sons, 2000, ISBN 0471351679.

  

• Software, which is installed on turing.cs.hmc.edu:
  • Matlab help
  • SNNS: Stuttgart Neural Network Simulator.
  • Genocop III (Genetic Algorithms)

Course Requirements

There will be some homework and programming assignments, but no exams. These assignments will constitute about 40% of your grade. 40% of your grade is from a substantial final project involving either creation of a working neural network application or a research paper. The grade on the project will be determined by the comprehensiveness and degree to which you explored competing approaches. The projects will be presented orally. 10% of your grade will be based on a presentation you make related to the material of the project. Finally, 10% will be based on general participation, which includes attendance.

CS 152 Topic Outline (subject to minor revisions)

• Week 1
  • Introduction
    • Contexts for and Motivation Neural Networks: Artificial Intelligence | Biological | Physics | Engineering
    • Artificial Neural Network overview
  • Types of Problems: Recognition vs. Transformation
  • Non-neural approaches, such as regression (read NAS, ch 1)
  • Supervised Learning: Single-Layer Networks
    • Adalines (Adaptive Linear Networks) (read NAS, ch 2, except maybe for 2.2)

• Week 2, Supervised Learning: Multi-Layer Networks
  • Multi-Layer Perceptrons (MLPs) (read NAS, 3.1-3.5)
  • Historical Perceptrons
  • Delta rule
  • Adatron algorithm

• Week 3
  • Backpropagation (read NAS, 3.6)
  • Embedded Adaptive Systems (read NAS, 3.7)

• Week 4
  • Designing and Training MLPs (read NAS, 4)
  • Conjugate Gradient method
  • Levenberg-Marquardt (LM) method
  • Cascade-Correlation Networks (not in NAS)

• Week 5, Function Approximation (read NAS, 5)
  • Radial-Basis Networks
  • Support Vector Machines

• Week 6, Hebbian Learning and PCA (read NAS, 6)

• Week 7, Linear Associative Memories (LAM's)

• Week 8, Competitive Networks (read NAS, 7)
  • Simple Competitive Networks: Winner-take-all | Hamming network
  • Adaptive Resonance Theory (ART)
  • Kohonen Self-Organizing Maps (SOMs)
  • Learning Vector Quantization (LVQ)
  • Counterpropagation Networks (CPN)

• Week 9, Digital Signal Processing (read NAS, 8)
  • Time series
  • Adaptive Filters (read NAS, 9)

• Week 10, Temporal Processing (read NAS, 10)
  • Backpropagation through time
  • Finite Impulse Response (FIR) MLP
  • Temporal Differences method (TD, not in book)

• Week 11, Recurrent networks (read NAS, 11)
  • Hopfield Networks
  • Brain-State-in-a-Box (BSB)
  • Boltzmann Machines and Simulated Annealing
  • Bi-Directional Associative Memory (BAM)

• Week 12, Evolutionary computation, Fuzzy logic
  • Neural Network Approaches
  • Evolutionary Programming