Lab 5: Information Retrieval and Vector Semantics

For this last part of the lab, we want to give you some experience finding new Python libaries and learning to use them with minimal guidance from us. This will likely be something you have to do quite a bit of in your final project!

There are two "paths" in Part 5; since this lab has already gotten quite long, you are only requrired to do ONE of the paths, Part 5a OR Part 5b. That being said, your experience with the lab may be most "satisfying" if you do both! To sweeten the deal, anyone who completes both paths is eligible to receive 10 points of extra credit on this lab! However, if you do this, you must explicitly indicate on your journal that you are intending to claim the extra credit points. Failure to do so means you are forfeiting the extra credit points, even if you complete both paths!

Both paths are related to same idea: at the end of lecture, we explained that SVD is only one approach for transforming document or word vectors into concept space, and that neural networks are an alternative approach. In practice, neural networks tend to be "better", by which we mean that they tend to capture more intuitive, humanlike concepts. Part 5 will have you explore such neural network vector models; Part 5a focuses on models for document vectors and Part 5b focuses on models for word vectors.

Hint: This part is asking you to find Python libraries that implement these neural network approaches. Whatever library you find probably won't be already installed in your environment. But the benefit of using virtual environments is that you can always install whatever libraries you want in the environment using pip, no admin permissions needed!

Part 5a: Neural network document vectors

Your task here is to do some research to find Python packages that implement models for representing documents as vectors. As a hint for your search, these are more commonly referred to as embedding models. You can use whatever resources you want for your search—search for "python embedding models" on google, do a paper search, ask ChatGPT, talk to your peers who have done summer research, or ask a prof! Whatever approach you use, make sure to document your search process in your journal. As another hint, one popular (relatively) modern embedding model is BERT; maybe that can help guide your search (but you can also find other models!)

Once you have found a library that you can use to compute document embeddings, you should integrate it into wiki_search.py by implementing a new subclass of WikiDocumentVectors that uses the library you found to create vector representations of documents (and queries). You will also want to modify main to allow the script to use this new subclass. Then, answer the following analysis questions:

Analysis Question #7a

Re-run the same queries from Analysis Question #1/#3 using your new neural network vector model, and paste the new results in your journal. What changed? For each query, do you think the new results are better or worse than before? Why?

Analysis Question #8a

Re-run your queries from Analysis Question #2/#4 using your new neural network vector model, and paste the new results in your journal. What changed? Do the new results better align with your expectations? For the two queries you originally expected the IR system to get wrong, does the system now get them right?

---

Part 5b: Neural network word vectors

Your task here is to do some research to find Python packages that implement models for representing words as vectors. As a hint for your search, these are more commonly referred to as word embedding models. You can use whatever resources you want for your search—search for "python word embedding models" on google, do a paper search, ask ChatGPT, talk to your peers who have done summer research, or ask a prof! Whatever approach you use, make sure to document your search process in your journal. As another hint, one famous word embedding model is word2vec; maybe that can help guide your search (but you can also find other models!)

Once you have found a library that you can use to compute word embeddings, you should integrate it into wiki_word_lookup.py by implementing a new subclass of WikiWordVectors that uses the library you found to create vector representations of documents (and queries). You will also want to modify main to allow the script to use this new subclass. Then, answer the following analysis questions:

Analysis Question #7b

Repeat Analysis Question #5 using your new neural network vector model, and paste the results in your journal. How do these results compare to the ones from Analysis Question #5? Do you think that this model capture more "humanlike" concepts than SVD? Are there any word similarities you still find confusing or incorrect?

Analysis Question #8b

Repeat Analysis Question #6 using your new neural network vector model. Have the results significantly improved, in terms of things you were originally troubled or unsatisfied by? Or perhaps have the results gotten worse? In either case, why do you think this is?

(When logged in, completion status appears here.)

Previous Page Go to Parent Page