Lab Journaling Instructions and Grading Rubric

Motivation

Oftentimes, what distinguishes highly successful NLP projects from less successful ones is not the quality of the code or the sophistication of the mathematics. Rather, it is the ability of the project developers to successfully interpret their model and their findings, often by drawing insights from linguistics, social sciences, and biology (for example: modern-day LLMs are fundamentally powered by a mechanism known as "attention"—to be briefly discussed later in this class—which was very loosely inspired by similar mechanisms in the human brain!). Yes, it's important to know how to write code for a part-of-speech tagger, but it's far more important to be able to analyze why, based on your understanding of the model and the data it was trained on, the tagger is likely to make certain kinds of mistakes.

All of this is to say: I care less about your actual code and more about your underlying thought process. And the best way for me to see your thought process is, well, to have you literally write down your thought process!

Journaling Instructions

To this end, for each lab assignment you will be asked to record and submit a real-time journal that details all the things you tried as you worked on the assignment.

"All the things you tried" is the key point here: when working on an assignment (or any other coding project), it is often the case that the first few things you try don't quite work. Therefore, in a traditional class environment, the code/writeup you end up submitting only reflects the thing that ended up working. But here it's the opposite: I want to know all the details of "how you cooked" your final submission, how you got from a not-working to a working state. And importantly, I want to see your debugging process: when you see that something has gone wrong, I don't just want to hear you say "it didn't work", I also want to hear your speculation about why it didn't work.

(Of course, it is also the case that some things you try will work. When that happens, I also want to hear it; after all, success is worth celebrating!)

To be more specific, your journal should include all of the following:

• High-level ideas you have as you figure out how to solve a problem in the assignment. (example: "My first thought was to use Python's Counter class to handle the counting of tokens")
• Any difficulties you encountered. Note that "difficulties" doesn't only include bugs, but also covers things like being unsure how to even start, having code that works but gives results that don't make sense, or even being confused about what the problem is asking. (examples: "The function worked for most test cases but gave counts that were off by one on the file example.txt"; "I know the first thing we need to do is tokenize but I'm not sure how to further filter the tokens")
• Steps you took to resolve the difficulties. This can include things like (concisely) outlining your debugging steps, or citing resources that helped you solve the problem. (examples: "I opened example.txt and compared it to another input file that the function worked correctly on"; "I went to grutoring hours and the grutor told me to pay attention to how I handle line endings")

There are no requirements about how you choose to format your journal; you can go with paragraphs or bullet points based on personal preference. Likewise, there are no requirements about how the journal is organized, other than it is somehow organized so that it's readable (ideas include making separate sections for each day you worked on the assignment, or sections corresponding to sections in the assignment instructions). There is only one formatting requirement for the journal: it must be written in a Google Doc that is shared between you and the course staff (see below for more details).

Real-time feedback via Google Docs

At the start of the semester, Prof Chang will create one shared Google doc per student. The Google doc will already contain (blank) tabs for every lab; all your journaling should be done in this doc. (If you are working with a partner on a given week's lab, only one partner's tab for that lab needs to be filled out; the other partner can simply write "I worked with [partner name]" in their tab so the course staff knows where to look.)

Why a Google doc? Well, as mentioned above, part of the motivation behind the journaling system is that I want to see your thought process. And having all the work happen in a shared doc lets this literally happen in real time! And the neatest part is that this goes both ways: I can also give feedback and guidance in real time! To this end, Prof Chang may occasionally leave comments in your lab journal; you are not required to respond to these comments, but it's probably a good idea to take them seriously since they may give hints to a problem you're struggling with, or even let you know that you're going in the wrong direction.

Journaling dos and don'ts

DO:

• Be complete. Include all ideas you had even if they didn't pan out, and include sufficient details about your problem-solving process that course staff can get a good sense of what you did.
• Be concise. This may seem to be in tension with the previous point, but the idea here is that you should be complete without including extraneous details. We don't need to hear something like "and then in the middle of working on this, I went to dinner".
• State when you got help from an outside source, and cite that source properly.
• Draw direct connections to material from the lectures or readings.
• Include example outputs where appropriate.
• Maintain your journal in real-time, as you work through the assignment.

DON'T:

• Include large blocks of code in your journal. You will submit your code separately so the course staff will have access to it. You should only include small snippets or plain-english summaries of your code as needed to explain your ideas.
• Blindly copy-paste entire output printouts. Oftentimes, the code will print a bunch of stuff but only part of that output is relevant to the problem at hand. Selecting and including only the relevant output both demonstrates that you truly understand what's being printed, and helps keep your journal more readable for us.
• Focus only on the code. In NLP, understanding the cause of a problem usually requires an understanding not only of the code itself, but also of the input data that the code is running on.
• Wait until the assignment is done to write your journal all at once. This defeats the purpose of journaling, and I guarantee you there will be details you forget if you wait until the end to write everything up.

Example journal

Looking for more guidance? Here's an example of (an excerpt from) a journal that was kept by some of my research students during Summer 2025. Note that the task these students were working on was more of a web development project than an NLP one, so the specific technical insights are different from what you'll have in your journals. Nonetheless, I think this excerpt demonstrates a lot of the good practices listed above, such as explicitly showing planning outlines and general thought processes, outlining debugging steps taken to solve problems, and including figures and example outputs.

Analysis Questions

In each lab assignment, there will be specifically marked analysis questions we want you to answer. These are distributed throughout each assignment (not all at the end of the assignment) and are purposely placed to get you to reflect more on the code you just wrote / experiments you just ran. Separate from your main lab journal, you will also be expected to write up your answers to the analysis questions (there will be separate tabs in the Google doc for each lab's analysis questions).

Even though analysis questions are separate from the main journal, remember that their purpose is to spur thinking! So if answering an analysis question helps you figure something out, gives you ideas, or otherwise provides insights that are relevant to your overall thought process, you can—and should—mention this in your journal.

Journal submission and grading

For each lab, you will be asked to submit three things: a link to the GitHub repo for your code, a PDF copy of your journal, and a PDF copy of your analysis question answers. If you were working with a partner, you only need to submit one journal for both of you, but note that on the Honor Code, both partners must contribute roughly equally to the submitted journal.

You may notice that we are having you simply provide a link to your GitHub repo rather than directly submitting your code to Gradescope. This is because we will not be running an autograder on your code, and in fact will not be directly grading your code at all. Instead, your lab grade will be determined purely based on your journal. I recognize that this may be a somewhat alien experience to many of you, but my hope is that this adds some flexibility and nuance to the grading process. In fact, with a good journal, you may be able to get a lot of credit on an assignment even if your code doesn't work perfectly!

Journal grading is based on five criteria: Insight, Growth, Completeness, Technical proficiency, and Organization. Each criterion is judged on a 4-point scale of Needs significant improvement, Room for growth, Acceptable, and Outstanding. Note that "Acceptable" is considered the default and is the score that a student adequately keeping up with the content should expect to get most often, and it translates to the A-/B+ range. "Outstanding" is reserved for work that especially stands out above and beyond the expectations of the course.

One minor exception: since "Completeness" is pretty open-and-shut (you can't really do _more than fully complete), the "Outstanding" category does not apply to "Completeness", and "Acceptable" directly translates to the A range._

For more details on how these categories and scale are defined, click here to view the official rubric.

(When logged in, completion status appears here.)

Previous Page Go to Parent Page