Week 12 Activity: Colloquium talk @ Pomona
This week, Pomona College hosts a talk by Leilani Gilpin, from UC Santa Cruz, in Seaver North Auditorium on the Pomona campus. The talk begins at 4:15 PM on Thursday.
For students enrolled in CS Colloquium (CS 195):
- If you’re in Section 1, we expect you to attend the event when it occurs (synchronously). You will need to log your in-person attendance at the event using one of the following messages:
- Via smartphone: There will be a QR code at the event that you can scan to log your attendance. It will use this website, so it will help to already be logged in on your phone beforehand to make the process smoother.
- Via paper sign-in: If you don't have a working smartphone, or you have technical difficulties, you can sign in on a paper form at the event. We will manually enter your attendance into the system later.
- If you're in Section 2, our colleagues at Pomona are hoping to record the talk, but it will take a few days to process the recording (assuming it is recorded successfully). We will post the recording here when it becomes available, so please check back regularly to see if it has been posted.
Accountability Layers: Explaining and Stress Testing Complex Systems
Abstract
Complex systems are prone to errors and failures without knowing why. In critical domains like driving, these autonomous counterparts must be able to recount their actions for safety, liability, and trust. In this talk, I will argue that an explanation: a model-dependent reason or justification for the decision of the autonomous agent being assessed, is a key component for post-mortem failure analysis and pre-deployment verification. I will present a framework that uses a model and commonsense knowledge to detect and explain unreasonable vehicle scenarios, even if it has not seen that error before. In the second part of the talk, I will motivate the use of new testing frameworks for complex systems including Large Language Models (LLMs). I will conclude by discussing new challenges in developing trustworthy AI towards complex systems that are explainable by design.
About Leilani Gilpin
Leilani H. Gilpin is an Assistant Professor in the Department of Computer Science and Engineering at UC Santa Cruz. Her research focuses on the design and analysis of methods for autonomous systems to explain themselves. Her work has applications to robust decision-making, system debugging, and accountability. She holds a PhD in Computer Science from MIT, an M.S. in Computational and Mathematical Engineering from Stanford University, and a B.S. in Mathematics (with honors), B.S. in Computer Science (with highest honors), and a music minor from UC San Diego. Outside of research, Leilani enjoys swimming, cooking, hiking, and org-mode.
When and How to Attend
- Thursday, April 11
- Location: Seaver North Auditorium, Pomona College
- Talk runs from 4:15–5:30 PM
Recording for Section 2
Unfortunatley, the talk was not recorded, so instead we're providing a different talk by Zac Hatfield-Dodds of Anthropic, recorded at the Strange Loop Conference in St. Louis, MO, on September 21, 2023.
Please indicate on your assessment that you watched this video instead of the Pomona talk.
Is my Large Language Model a Strange Loop?
Abstract
Large Language Models aren't just the new hotness - they also offer fascinating glimpses of an alternative approach to software which is evolved instead of written. We'll explore emergent representations and composition of concepts from the inside outwards; and then turn around and consider some emergent behaviors and feedback loops from the outside of the training process. Whether you're curious about superposition, induction, self-evaluation or self-correction... by the end of the talk you'll have more questions than you started with!
About Zac Hatfield-Dodds
Zac Hatfield-Dodds is an Assurance Team Lead at Anthropic. Zac is an Australian software engineer and researcher working in San Francisco, on AI safety research by day and on property-based testing by night. He co-maintains open-source projects such as Hypothesis, Trio, and Pytest; has been elected a Fellow of the Python Software Foundation; and swears that he'll finish writing his dissertation someday. If he's not at a computer, Zac is probably some combination of camping in a national park, reading a book or three, and munching on dark chocolate.
Required Assessment
To receive full credit for attending this colloquium, complete the assessment:
Please do so at your soonest convenience, within 24 hours of seeing the talk.
(When logged in, completion status appears here.)