Lab 1: Tokenization and Segmentation

In this lab, you will practice writing and evaluating rule-based systems for tokenization and sentence segmentation.

Let's Go!

All starter code in CS 159 is distributed through GitHub. Since this is your first real lab, please take a moment to view the help page on starter code for instructions on how to get the starter code.

Once you're ready, you can find the Lab 1 starter code here:

• Link to Lab 1 starter code

Outline

• Part 1: Tokenizer Basics
• Part 2: Through the Rabbit Hole
• Part 3: Sentence Segmentation

Submitting Your Journal and Analysis Questions

When you're done, submit your journal and analysis questions on Gradescope:

• Gradescope Submission Link

Reminders

• Remember that you need to activate your conda environment every time you log in to the course server. If you need a reminder on how to do this, see Part 2 of the Lab Setup Help Page.
• If you are working with a partner (which is strongly recommended), remember that you only need to fill out one journal doc; the other partner can just write "worked with (partner name)" in their doc. However, also don't forget to make a joint submission on Gradescope (like in CS 70, the partner making the Gradescope submission should add their partner to the submission).
• Analysis Questions are scattered throughout the lab. Please answer them as soon as you reach them, and don't save them all for the end; they are strategically placed to get you to think about the code you just wrote or the results you just saw.

(When logged in, completion status appears here.)

Previous Page Next Page