Speaker: Ekdeep Singh Lubana, Postdoc Fellow at CBS-NTT Program at Harvard University.
Title: Formal Models for Sudden Learning of Capabilities in Neural Networks
Date and Time: January 09, 2025, 04:00 PM
Venue: #102, CDS Seminar Hall.
Abstract: Neural networks’ scaling has been argued to yield sudden learning of capabilities (a.k.a. emergent abilities). In this talk, I will first summarize our recent work on formal models that help explain the mechanisms underlying such sudden learning via data scaling, implicating the compositional nature of a task and formation of structured representations that are shared across several tasks involved in the broader data composition. Then, focusing on in-context learning (ICL)—one such suddenly learned capability—I will demonstrate the precise configurations used for training can lead to learning of fundamentally different algorithms for performing an ICL task. This indicates the phenomenology of ICL established in past work may not be universal. Further, I will discuss how merely scaling the context size can lead to a crossover between different ICL algorithms used by the model. This can be explained via a competition of algorithms lens, which also yields a new theory on the transient nature of ICL. The talk will be based on a mix of published (https://arxiv.org/abs/2310.
Bio of Speaker: Ekdeep Singh Lubana is a Physics of Intelligence Postdoc Fellow at CBS-NTT Program at Harvard University. Broadly, his research is focused on using model systems for identifying novel challenges and better understanding existing challenges in alignment of AI systems. His recent work has revolved around developing mechanistic explanations for emergent capabilities in neural networks, demonstrating the brittleness of fine-tuning based approaches (e.g., RLHF) for alignment, and tools for risk monitoring in post-deployment settings.
Host Faculty: Prof. Venkatesh Babu
ALL ARE WELCOME
