Key Points
- Neural networks can be mathematically expressed as chains of matrix operations
- Training a neural network is done using gradient descent, specifically via an operation known as backpropagation, which is an application of the chain rule of calculus.
- The neural network we focused on in these lessons, with layers that strictly take input from the previous layer, is known as a feed-forward network.
- Besides feed-forward networks, there are other types of neural networks:
- The convolutional neural network, which uses a sliding window approach with a smaller weight matrix, is often used for computer vision
- The recurrent neural network allows layers to feed their output back to previous layers, and is useful in NLP for sequence-to-sequence tasks such as translation or POS tagging.
- Transformers will be covered in Week 8.
- The metrics of precision, recall, and F1 that we learned about previously can also be used to evaluate classifiers
- However, if you have more than two classes, you will need to compute each metric multiple times, once for each class.
- To average the results across all classes, you can either take the macro average which is a straight average with no weighting, or the micro average which is weighted by class distribution (more common classes get more weight).
- Accuracy is an additional metric you can use, which is computed as simply the fraction of correctly classified examples, and has the advantage of being directly applicable no matter how many classes there are.
- Research papers can be hard to read, but there are systematic strategies for approaching them!
Takeaway Skills
On a subsequent lab assignment or exam, you are expected to be able to:
- Define the bolded key terms above.
- Answer basic multiple-choice / true-or-false questions about the different types of neural networks.
- Given a set of true labels and classifier predictions, compute per-class precision, recall, and F1 scores.
(When logged in, completion status appears here.)