Lab 2: Rotations

Introduction

In this lab, you'll be creating a classifier to determine whether an image is rotated (not-rotated, left, right, or 180 degrees). This could be useful, for example, in a photo viewer to automatically rotate images that are not right-side-up.

The key to this lab is that properly labeled data is not provided. You'll need to be preparing the data.

Prepare data

Use the same data as was used for lab 1 (download).

The data has valid, train, and test folders. In order to have the most training data, train on the data in both the train and test folders. Use the data in the valid folder for validation.

Problems:

the data is categorized into giraffe and rhino classes, which isn't useful for this lab.
all the data is right-side-up. There's no 90, 180, or 270 degree rotated data to train on.

A solution is to:

Create a new directory with the desired hierarchy:

Programatically rotate each of the existing giraffe and rhino images and save them into the appropriate directory.
Create dataloaders from that new directory.

Useful methods when preparing data:

Path.mkdir (Pathlib docs)
PILImage.create: fastai utility method that creates a PIL.Image given a Path (docs).
PIL.Image.rotate (check out the expand flag)
PIL.Image.save

Train model

Now that you have your data, you can train a model. Use an existing pretrained model (like resnet34 or squeezenet1_1).

See how low you can get the validation loss. Get it to below 0.16.

Test model

What is the validation accuracy associated with the validation loss you achieved?

Which images in the validation set is the classifier least sure about? (You may find plot_top_losses useful—see fast.ai book chapter 2).

Suggestions

Make your notebook cells idempotent. That is, you should be able to rerun any cell without errors. For example, if a code cell makes a directory, it should either:
- Remove the directory first (!rm -r dir is an easy way to do so).
- or, pass a flag to the directory creation code so that it doesn't raise an exception if the directory is already there.
- or, catch the exception.
Make sure your validation data doesn't leak into the training data. For example, if you have a rhino picture 'myrhino.jpg', make sure the rhino, along with all the rotations of the rhino, is either entirely contained within the training data, or entirely contained within the validation data. Don't have the unrotated myrhino in training, along with the 90 degree rotation in validation (or vice-versa). Otherwise, the trained neural network has an unfair advantage in the test of the 90-degree rotation myrhino (because it was trained on that exact, albeit unrotated, rhino).
When you are using a pretrained model, train with learner.fine_tune. However, if your model is not pretrained, train with learner.fit or learner.fit_one_cycle.
Chapter 2 of the fastai book provides lots of information on training, data augmentation, and data loaders. When using data augmentation (like that provided by aug_transforms()), decide which types of transforms are appropriate for this task and which aren't. For example, will a horizontal flip be a good transformation?
Your notebook should be completely self-contained with reference only to the given dataset to download. Don't create the new training data on your own machine. Instead, create it as part of your notebook.
In order to make your notebooks reproducible, you can use the following fastai call:
```
set_seed(XXX, reproducible=True)
```
In addition, you'll want to pass in a seed when creating your data loaders.
There are callbacks you may find useful: EarlyStoppingCallback, SaveModelCallback.
Make sure you have GPU selected (change it using Runtime/Change runtime type menu. Your training will be orders of magnitude faster using a Graphical Processing Unit (GPU).
When you create a learner, you specify the metrics you want to be generated. Possibilities include error_rate and accuracy.
One way to find a good learning rate is to use lr_find.

Challenge Instead of pre-rotating each image, create a transform that will randomly both do the rotation and change the label (y). Thus, your dataloader can just load all the unrotated images, and the transform will do the rotation on the fly as part of data augmentation.

Note that this is a difficult problem.

Challenge Turn this into a regression problem where you are predicting the numeric degrees of rotation given a rotated image. In this case, you'll want to support rotating with increments of less than 90 degrees. You'll need:

a custom loss function (since a y of 1 and a y-hat of 359 should yield fairly low loss). Or, instead, generate 2 y's instead of 1 (sin and cos of the angle). In that case, you can just use the standard MSE.
to use the DataBlock API to generate a dataloaders that have a RegressionBlock rather than CategoryBlock as the type of y.

There may be other issues you'll need to solve as well.

Note that this is a difficult problem.

This completes the lab. Submit instructions

Make sure that the output of all cells is up-to-date.
Rename your notebook:
1. Click on notebook name at the top of the window.
2. Rename to "CS152Sp21Lab2 FirstName1/FirstName2" (using the correct lab number, along with your two first names). I need this naming so I can easily navigate through the large number of shared docs I will have by the end of the semester.
Choose File/Save
Share your notebook with me:
1. Click on the Share button at the top-right of your notebook.
2. Enter rhodes@g.hmc.edu as the email address.
3. Click the pencil icon and select Can comment.
4. Click on Done.
Enter the URL of your colab notebook in this submittal form. Do not copy the URL from the address bar (which may contain an authuser parameter and which I will not be able to open). Instead, click Share and Copy link to obtain the correct link. Enter your names in alphabetical order.
At this point, you and I will go back and forth until the lab is approved.
1. I will provide inline comments as I evaluate the submission (Google should notify you of these comments via email).
2. You will then need to address those comments. Please do not resolve or delete the comments. I will use them as a record of our conversation. You can respond to them ("Fixed" perhaps).
3. Once you have addressed all the comments in this round, fill out the submittal form again.
4. Once I am completely satisifed with your lab, I will add a LGTM (Looks Good to Me) comment
5. At that point, setup an office hour appointment with me. Ill meet with you and your partner and we'll have a short discussiona about the lab. Both of you should be able to answer questions about any part of the lab.