Lab 3: Text Classification
In Lab 2, you explored some sample data from the Semeval 2022 shared task on Patronizing and Condescending Language Detection, and did some basic comparisons of condescending versus neutral language. But as we discussed in class, oftentimes we want to go beyond a descriptive account of how two classes (e.g., condescending and neutral) differ from each other, and instead take a predictive approach: given a piece of text, can we estimate the likelihood that this text is condescending? That is the problem you will tackle, with the help of the full Semeval 2022 Patronizing and Condescending Language (PCL) dataset, in this lab!
To do this text classification task, we will make use of the Naive Bayes model we learned about in class. But rather than having you implement Naive Bayes from scratch, you will make use of scikit-learn's implementation, MultinomialNB. Instead, the focus of this lab will be on the science of text classification: how do we apply a classifier to (possibly messy) real-world data, and how can we draw conclusions from the results?
Reminder: the PCL dataset is provided in XML format. Individual examples in the dataset look something like this:
<example id="@@1824078" category="poor-families" country="tz" condescension="true" score="4">Camfed would like to see this trend reversed . It would like to see more girls in school . Basic Education Statistics in Tanzania ( BEST 2010 ) show that only 18 percent of girls have completed secondary school education . This is why Camfed supports girls from poor families to obtain secondary education and its efforts have seen many go to university .</example>
<example id="@@1921089" category="refugee" country="tz" condescension="false" score="0">Kagunga village was reported to lack necessary social services to meet the growing demand of refugees . The village has neither reliable , clean and safe water nor sanitation facilities that include latrines and critical medical services .</example>
The full PCL dataset can be found on the server at the following location: /courses/cs159/data/patronize/.
You may want to manually look through some of the data before you begin, in order to familiarize yourself with its structure. Although there is no coding TODO for this part, please make sure to document your data exploration process in your journal!
The FIRST thing we should see in your journal (other than starter code setup stuff if you're mentioning that) is you talking about how you explored the data!
(When logged in, completion status appears here.)