Syllabus
The Basics
Instructor: Prof. Jonathan Chang
Grutor: Diya Sanghi
Time and Place: T/R 1:15-2:30pm; SHAN 2450
Office Hours: TBD, McGregor 322
Grutoring Hours: See Grutoring Center
For information about office hours and links to the course resources, visit the Home page.
Motivation
Original comic by Zach Wienersmith [source]; extremely professional and seamless edit by your prof
We find ourselves in a world that is constantly being reshaped by natural language processing (NLP) technologies. This is perhaps most outwardly prominent right now in the form of chatbots and virtual assistants, but NLP is in fact all around us in more subtle ways—from search engines to spellcheckers to social media sorting algorithms. How do these technologies work? What is their relation to how we humans use language? What are their limitations? NLP is a rapidly changing field and the specific answers to these questions may change even within the next year (or sooner). Yet even today's most cutting-edge NLP models are grounded in foundational concepts and methods that have been with us for decades, and having a solid grasp of these foundational concepts can go a long way towards helping you sort out the hype from the reality. With NLP taking an ever more prominent position in daily life, a fluency in these key concepts will be a valuable skill for anyone who seeks to be a "leader in their field with a clear understanding of the impact of their work on society" (as the HMC mission statement calls you to be)!
Course Goals
This NLP class has two complementary goals: explaining core concepts and problems in NLP centering on how text is stored, predicted, categorized, dismantled, and interpreted by computers; and showing how those core concepts are connected to, and still very much present within, today's cutting-edge NLP technologies. By the end of this semester, you should be able to:
- Implement and use classic algorithms for natural language processing such as text normalization, statistical language models, vector representations of words, information retrieval, part-of-speech tagging, and text classification;
- Read and analyze primary literature (that, is, academic papers) in the field of NLP in popular subfields;
- Choose and interpret appropriate metrics of evaluation of how NLP technologies perform in practice;
- Critique the assumptions and approximations that are made about language when developing NLP problems, algorithms, datasets, and evaluations;
- Develop, execute, and communicate a research agenda, including experimental design and implementation, literature review, and effective use of LaTeX and slide decks.
Prerequisites
This course expects programming maturity (i.e., at the level of CS 70) and a course covering introductory topics in probability (which can include MATH 62, MATH 157, or another appropriate course at the instructor's discretion). If this doesn’t fit your prior courses, please reach out to me (Prof. Chang) immediately.
Course Structure
CS 159 can be roughly divided into two parts. Weeks 1-10 comprise the "core concepts" part of the class, which involves traditional lectures and readings with weekly lab assignments, focused on foundational NLP algorithms and concepts. Weeks 12-15 comprise the "special topics" part of the class, during which you will be reading and discussing research papers about modern developments in NLP, as well as working on a final project.
Class sessions
CS 159 uses a "semi-flipped" format. Of the two class sessions per week, one session (generally the Tuesday session) will be dedicated to a lecture covering the core topics for that week. You will also be assigned supplementary readings (mostly from the textbook, but sometimes mixed with other sources) which build upon the material from lecture. Supplementary readings should be completed before the following week's lecture at latest, but I encourage you to do them early, since lab assignments will rely on concepts from both the lecture and the readings.
The second class session will be in-class work time for you to work on that week's lab assigment (details below). Before coming to an in-class work session, you will be asked to complete a short check-in on Gradescope asking about your level of comfort with the concepts from lecture (and any parts of the readings you may have already completed). These will help me (Prof. Chang) better address any concerns that might come up during the work session. Check-ins are required and are graded based on completion.
During the "special topics" part of the class, lecture time will instead be devoted to in-class paper discussions. The second class session will remain as work time, though you will be working on your final project rather than a weekly lab.
Labs
Labs are an opportunity for you to get practice with the algorithms, tools, and concepts discussed in the lectures and readings. There are six total lab assignments; starting in Week 2, labs will be released weekly on Thursday morning (i.e., before the in-class work session) and due at midnight (11:59pm) the following Wednesday. All labs are implemented in Python; to ensure compatibility, we will provide access to a shared server environment with all necessary software and packages already installed.
Labs are also an opportunity for you to practice good record-keeping and documentation (important skills in both research and industry careers!). To this end, labs are not graded directly based on your code, but instead based on the quality of a work journal you will be asked to record for each lab assignment. Journaling instructions and a grading rubric can be found here.
Pair programming is strongly encouraged, but not required, for lab assignments.
Midterm
There will be one midterm exam covering concepts from the "core concepts" part of the class. The midterm is currently tentatively scheduled for Week 11. The midterm will consist of a solo portion containing multiple-choice and short-answer questions, and a group portion in which you will write an open-ended evaluation/critique of a provided example NLP application. Further details will be provided on the midterm page closer to the day of.
Special Topic Presentation
NLP is a constantly evolving field, and the "special topics" part of the class is designed to reflect that. In this part of the class, you will work with a small team of 3-4 students to lead a class discussion of a modern topic in natural language processing, in which you will presenting the core problem and some ideas from recent work. Information about signing up for topics and expectations for these will be shared when we reach that point.
Final Project
During the "special topics" part of the class, you will also work on a final project in a group of 1-3 people. The final deliverables of this project will be (1) an short 3-4 page ACL-style paper describing your project and (2) a short presentation of the highlights of your project. Your final project grade will be determined by these two deliverables, as well as several smaller milestone assignments in the second half of the semester. Final deliverables will be due on Gradescope at the end of the corresponding final exam period for our class.
Schedule
The tentative course schedule can be found here. Links will be added and updated for lab assignments, supplementary readings, and special topic signups as we progress through the semester.
Class Policies
Grading
Your grade will be computed using the following components:
- 10% Class participation and check-ins
- 35% Lab assignments
- 15% Midterm exam
- 15% Special topic presentation
- 25% Final project
Final grades are calculated based on the following scale (which, following HMC practices, does not include A+ or D-):
| Letter grade | Minimum numeric grade |
|---|---|
| A | 95 |
| A- | 90 |
| B+ | 87 |
| B | 83 |
| B- | 80 |
| C+ | 77 |
| C | 73 |
| C- | 70 |
| D+ | 67 |
| D | 65 |
Extensions
If you need a 1-day extension in this class for any reason (lots of exams, busy with a family obligation, minor illness, or just tired), please email the instructor (jpchang@hmc.edu) at least 24 hours before the deadline. Although I may not reply to confirm, any request for a 1-day extension on something besides an exam or the final project submission will be automatically honored, no matter the reason, as long as you send the email on time. If you’re working in a group, only one group member needs to request the late day.
If you encounter a situation requiring more than a 1-day extension (e.g., major illness or family emergency), please reach out to me as soon as you can so we can together determine a healthy plan for you to continue your coursework when it makes sense. My priority is always to make sure you can focus on the situation at hand while ensuring you won't spend the rest of the semester working to catch up again.
Accommodations
HMC is committed to providing an inclusive learning environment and support for all students. Students with a disability (including mental health, or chronic or temporary medical conditions) who may need accommodations in order to fully participate in this class are encouraged to contact their home college's accessible-education officer:
Honor Code
All students—even those from other colleges—are expected to understand and comply with Harvey Mudd College's expectations of academic integrity as outlined in our Honor Code and in the CS department's interpretation of that policy. Specifically,
- You must not exchange literal copies of material, whether that material consists of code, program output, or English-language text (e.g., documentation). You also may not copy material from published or online sources, with or without cosmetic changes (such as altering variable names), without explicit permission. If you do have permission to use externally written material, you must attribute it properly and clearly indicate which material is yours and which material is not yours. Publishing your own homework or exams from this class on the web (e.g., in a public GitHub repository) violates this policy.
- You should not do anything that a reasonable student peer would describe as "subverting the clear intent of the assignment," unless you have asked for and received permission to do so. Finding open-sourced code that you can use to solve an assigned problem, for example, would typically be subverting the intent of the assignment because your shortcut means that you do not learn what the assignment aims to teach.
- If you use any sources to assist you, you must document them. For example, if you use code from an existing researcher's code repository, paper, or blog post to build your class project, you must attribute that source in a sufficiently specific way for the course staff to easily find the original source.
- If you aren't sure whether something you've done or plan to do is allowed, you should explicitly document what you did and—if at all possible—consult with the course staff, ideally before you take the questionable action. Similarly, document any extensive or particularly important help you obtain, even if that help seems legitimate. If you've been helped so much that we can't consider the work truly your own, you might not be able to get full credit for it, but proper attribution will avoid an Honor Code violation (or academic integrity case at your institution).
- Academic integrity also involves being careful enough to avoid unintentionally breaking the rules. Thus, you must read instructions in assignments and exams carefully so that you are aware of any limitations they place on you, such as time restrictions or restrictions on information sources you may consult. Similarly, if you see something that plausibly seems like it ought to be off-limits to you, such as a GitHub directory belonging to another student or files from a previous semester, you should immediately contact us to let us know that something doesn't seem right, rather than looking further at something that perhaps should have been off-limits.
Special note regarding Generative AI assistants
In the age of Generative AI (e.g., ChatGPT, Google Gemini, etc.), students are frequently left wondering what (if any) uses of such tools are considered acceptable in each of their classes—and this is perhaps a particularly topical concern in an NLP course!
The short answer is that there is no specific Generative AI policy in CS159; for the purposes of this class, we consider Generative AI tools to fall under the same category as any other external resource. This means that your usage of Generative AI is simply governed by the same Honor Code considerations outlined above.
A good rule of thumb I like to use is, whenever you are asking a question to ChatGPT or any other Generative AI tool, you should consider: is this something that the Honor Code would consider acceptable if I had asked my roommate instead of ChatGPT? Asking your roommate to just solve an entire homework problem for you, and then submitting their solution as your own, would be a clear violation of the Honor Code. On the other hand, if you have figured out on your own that solving the problem requires multiplying two matrices and you just don't know the syntax to do that in NumPy, and you know your roommate is a NumPy expert, asking them about that specific syntax seems perfectly acceptable. And the same logic applies when you substitute "your roommate" with "ChatGPT" in that scenario.
All that having been said, there are a few additional nuances with Generative AI tools that interact with the Honor Code policies in unique ways that you should fully understand before using such tools:
- The policy states that external resources must be attributed "in a sufficiently specific way for the course staff to easily find the original source." In the case of Generative AI tools, the response often depends both on the specific model and the prompt that you used, so attribution must mention both the model and the exact text of your prompt (or prompts, if multiple were used).
- The policy also cautions about unintentional breaking of the rules. This is a particularly salient issue when it comes to Generative AI, since Generative AI models don't always perfectly follow instructions (for reasons you will hopefully understand better after taking this class!). You may intend to only get syntax help and end up receiving a large code snippet that solves more of the assignment than you intended. In such cases, you should immediately stop and contact the course staff so we can work together to figure out how to proceed—which is exactly the same thing you should do when, for example, you accidentally encounter a full solution in a "traditional" online resource.
As for journal writing, similar principles apply. Asking an LLM to completely generate your journal (or large chunks of it) from scratch is a clear Honor Code violation (and defeats the purpose of journaling). On the other hand, if you just wanted to use an AI tool like Grammarly to polish up the wording of a journal that you already wrote in your own words, that may be acceptable as long as you explicitly acknowledge the use of the tool and provide the text of your prompt (if any). That being said, I will point out that the journal is intended to be a "brain dump" of your thoughts and therefore I don't need it to be super polished. It may be preferable (and less work) to turn in an unpolished "raw" journal than to spend unnecessary time prompting an LLM just right to polish it!
(When logged in, completion status appears here.)