Catalog Description

Modeling, simulation, and analysis of artificial neural networks. Relationship to biological neural networks. Design and optimization of discrete and continuous neural networks. Backpropagation, and other gradient descent methods. Hopfield and Boltzmann networks. Unsupervised learning. Self-organizing feature maps. Applications chosen from function approximation, signal processing, control, computer graphics, pattern recognition, time-series analysis. Relationship to fuzzy logic, genetic algorithms, and artificial life.

Prerequisites: CS 60 and Mathematics 73, or permission of the instructor. 3 credit hours.

Instructor

• Robert Keller 242 Olin (4-5 p.m. MTuW or by appt.), keller@turing, x 18483
  
  

Texts

• Main Textbook:

  MMR: K. Mehrotra, C.K. Mohan, and S. Ranka, Elements of Artificial Neural Networks, M.I.T. Press, 1997, ISBN 0-262-13328-8.

  

• Software, which is installed on turing.cs.hmc.edu:
  • Matlab help
  • Matlab cross referenced
  • SNNS: Stuttgart Neural Network Simulator.
  • Genocop III

Course Requirements

There will be some homework and programming assignments, but no exams. These assignments will constitute about 50% of the grade. The other 50% of the grade is from a substantial final project involving either a working neural network application or a research paper. The grade on the project will be determined by the comprehensiveness and degree to which you explored competing approaches. The projects will be presented orally.

Optional voluntary oral presentations on textbook material can also be made during the term. These can act to cushion your grade. They are very much encouraged, as it they really help you learn the material at a higher level than you would otherwise. Please see me if you are interested in making a presentation.

CS 152 Topic Outline

• Week 1 (read MMR chapter 1)

  Introduction

  • Contexts for and Motivation Neural Networks: Artificial Intelligence | Biological | Physics
  • Artificial Neural Network overview

• Week 2 (read MMR chapter 2)

  Supervised Learning: Single-Layer Networks

  • Perceptrons
  • Adalines

• Weeks 3-6 (read MMR chapters 3 and 4)

  Supervised Learning: Multi-Layer Networks

  • Multi-Layer Perceptrons (MLPs)
  • Backpropagation
  • Conjugate Gradient method
  • Levenberg-Marquardt (LM) method
  • Madalines
  • Radial-Basis Networks
  • Cascade-Correlation Networks
  • Polynomial Networks
  • Recurrent Networks
    • Time series
    • Backpropagation through time
    • Finite Impulse Response (FIR) MLP
    • Temporal Differences method (TD)

• Weeks 7-8 (read MMR chapter 5)

  Unsupervised Learning

  • Simple Competitive Networks: Winner-take-all | Hamming network
  • Learning Vector Quantization (LVQ)
  • Counterpropagation Networks (CPN)
  • Adaptive Resonance Theory (ART)
  • Kohonen Self-Organizing Maps (SOMs)
  • Principal Component Analysis networks (PCA)

• Weeks 9-10 (read MMR chapter 6)

  Associative Models

  • Linear Associative Memory (LAM)
  • Hopfield Networks
  • Brain-State-in-a-Box (BSB)
  • Boltzmann Machines and Simulated Annealing
  • Bi-Directional Associative Memory (BAM)

• Week 11 (read MMR chapter 7)

  Optimization Problems

  • Neural Network Approaches
  • Evolutionary Programming

• Week 12

  Fuzzy logic and its connection to NNs