Lab 4: Part of Speech Tagging

In this week's lab, you'll implement the Viterbi Algorithm for doing part-of-speech tagging with Hidden Markov Models.

Let's Go!

As always, start by creating a new GitHub repo from the starter code template:

• Link to Lab 4 starter code

This lab provides the following starter files:

• HmmTagger.py
• evaluate.py
• read_tags.py

We also provide a minimalist test suite for the classes you will be implementing:

• test-tagger.py

As always, the tests are mostly just to ensure that your code is following the specifications so that you don't end up doing analysis based on results that are way off from what we intended. They are not meant to catch every possible bug, and you should feel free to do your own testing (and, if you do so, talk about it in your journal). Also note that, compared to previous labs, this lab's tests will run a bit slower (taking about a minute to run to completion on the course server).

IMPORTANT NOTE: If at any point during this lab you encounter the error ImportError: No module named 'en_core_web_sm', you can fix this by running the following incantation in the terminal:

python -m spacy download en_core_web_sm

Outline

• Introduction, Lecture Recap, and Matrix Terminology
• Part 1: Understanding the Starter Code
• Part 2: Finishing the Viterbi Implementation
• Part 3: Exploring Tag Set Collapsing
• Part 4: Exploring Vocabulary Size Effects

(When logged in, completion status appears here.)