Lab 3: Text Classification
In the same way you inherited from PCLLabels to create the BinaryLabels class, you should add your own derived class to pcl_main.py to extract features, inheriting from the PCLFeatures base class. Your class should be called MyFeatures. For now, this class should implement a bag-of-words feature set: that is, like described in class, you should treat each individual word in an example as a feature (we will expand beyond this later!). Importantly, this time around we are providing a vocabulary that defines what should count as a word! Thus, you should implement _extract_features such that it returns a list of words (features) that are in the input vocabulary. Words that are not in the input vocabulary should be ignored.
You may wish to take a moment to make sure you understand the process method of PCLFeatures, which will call your _extract_features method, to make sure that you are returning the right thing.
The vocabulary you should use to initialize your vocab is stored in /courses/cs159/data/patronize/vocab.txt. For the initial tokenization phase, you should split on whitespace (like you did in Lab 1); you should not do anything fancy like using spaCy. An extract_text helper function has already been provided for you that does this, so you don't have to re-implement your Lab 1 code.
Note: While you are only required to implement functions that match the interface of the PCLFeatures class, you’re encouraged to add extra helper functions make your code more modular and have good style.
Hint: be very careful when implementing the _get_feature_name method! As the provided comment notes, the purpose of this feature is to get the name (i.e., the text) of the ith feature in the DictVectorizer's internal vocabulary. This internal vocabulary may not exactly match the provided initial vocabulary! It can differ for two reasons. One is that the DictVectorizer may sort the features differently. Another is that some vocabulary words may not appear at all in the training data (e.g., if you reduced max_instances to a small number, you will only ever see the words that happen to appear in those few documents), and the DictVectorizer only considers features it has actually seen. You may wish to consult the DictVectorizer documentation to find out how to access the DictVectorizer's internal vocabulary.
(When logged in, completion status appears here.)