Reading Assignment: A Recent NLP Paper
Given how quickly the field of NLP moves, all papers we will be discussing in class are recent (from the past year or so). So, for this "prep" reading, we've picked out an extremely recent paper, "Prompt Injection as Role Confusion" by Charles Ye, Jasmine Cui, and Dylan Hadfield-Menell (just posted over the summer, in June of 2026!). We find this fairly representative of the kind of work NLP researchers are doing now, and the kind of writing style you'll tend to see.
Background About the Paper
This paper examines how LLMs parse "system roles". "System roles" are another name for the special tokens you learned about in the Week 3 online lesson that act as control sequences for LLMs in chatbot and agent settings.
As a quick recap, remember in Week 3 online lesson we showed you the following real example of raw output from an LLM in a chatbot application:
<channel|><|tool_call>call:run_javascript{javascript:<|"|>var x = [1,2,3];
var y = x;
y += [4];
console.log("Value of x:", x);<|"|>}<tool_call|><|tool_response>response:run_javascript{value:<|"|>{"stdout":"Value of x: [ 1, 2, 3 ]","stderr":""}<|"|>}<tool_response|><|channel>
The <|tool_call> token is an example of a special token that indicates the purpose of the following text: it will represent a command to be run, rather than actual text to output to the user. Other examples of special tokens we discussed in the lesson are <user> to denote text that arrived as input typed in by the human user, and <assistant> to denote the chatbot's actual reply as it should be shown to the human user.
If you need a refresher, feel free to head back to the Week 3 online lesson, specifically the page on special tokens in LLMs.
The paper you're about to read addresses the following research question: how strictly does an LLM actually follow the "roles" denoted by the special tokens?
Read the Paper!
Now that you know what to expect, let's go ahead and read the paper!
IMPORTANT: When reading the paper, do NOT click on any of the links inside. This is to prevent accidental "spoilers" for the next part of the lesson...
Reflection
Once you have finished reading the paper, answer the following questions. These are the same questions you'll be asked to fill out before every in-class paper discussion.
(When logged in, completion status appears here.)