BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//wp-events-plugin.com//7.2.3.1//EN
TZID:Asia/Kolkata
X-WR-TIMEZONE:Asia/Kolkata
BEGIN:VEVENT
UID:96@cds.iisc.ac.in
DTSTART;TZID=Asia/Kolkata:20250109T160000
DTEND;TZID=Asia/Kolkata:20250109T170000
DTSTAMP:20250102T133802Z
URL:https://cds.iisc.ac.in/events/cds-kiac-seminar-cds-102-09th-january-fo
 rmal-models-for-sudden-learning-of-capabilities-in-neural-networks/
SUMMARY:CDS-KIAC {Seminar} @ CDS: #102 : 09th January : "Formal Models for 
 Sudden Learning of Capabilities in Neural Networks"
DESCRIPTION:\n\nSpeaker: Ekdeep Singh Lubana\, Postdoc Fellow at CBS-NTT Pr
 ogram at Harvard University.\nTitle: Formal Models for Sudden Learning of 
 Capabilities in Neural Networks\nDate and Time: January 09\, 2025\, 04:00 
 PM\nVenue: #102\, CDS Seminar Hall.\n\n\n\nAbstract: Neural networks’ sc
 aling has been argued to yield sudden learning of capabilities (a.k.a. eme
 rgent abilities). In this talk\, I will first summarize our recent work on
  formal models that help explain the mechanisms underlying such sudden lea
 rning via data scaling\, implicating the compositional nature of a task an
 d formation of structured representations that are shared across several t
 asks involved in the broader data composition. Then\, focusing on in-conte
 xt learning (ICL)---one such suddenly learned capability---I will demonstr
 ate the precise configurations used for training can lead to learning of f
 undamentally different algorithms for performing an ICL task. This indicat
 es the phenomenology of ICL established in past work may not be universal.
  Further\, I will discuss how merely scaling the context size can lead to 
 a crossover between different ICL algorithms used by the model. This can b
 e explained via a competition of algorithms lens\, which also yields a new
  theory on the transient nature of ICL. The talk will be based on a mix of
  published (https://arxiv.org/abs/2310.09336\, https://arxiv.org/abs/2406.
 19370)\, in-submission (https://arxiv.org/abs/2412.01003\, https://arxiv.o
 rg/abs/2408.12578\, https://arxiv.org/abs/2410.08309)\, and currently unpu
 blished work.\n\nBio of Speaker: Ekdeep Singh Lubana is a Physics of Intel
 ligence Postdoc Fellow at CBS-NTT Program at Harvard University. Broadly\,
  his research is focused on using model systems for identifying novel chal
 lenges and better understanding existing challenges in alignment of AI sys
 tems. His recent work has revolved around developing mechanistic explanati
 ons for emergent capabilities in neural networks\, demonstrating the britt
 leness of fine-tuning based approaches (e.g.\, RLHF) for alignment\, and t
 ools for risk monitoring in post-deployment settings.\n\nHost Faculty: Pro
 f. Venkatesh Babu\n\n\n\nALL ARE WELCOME
CATEGORIES:Events,Talks
END:VEVENT
BEGIN:VTIMEZONE
TZID:Asia/Kolkata
X-LIC-LOCATION:Asia/Kolkata
BEGIN:STANDARD
DTSTART:20240110T160000
TZOFFSETFROM:+0530
TZOFFSETTO:+0530
TZNAME:IST
END:STANDARD
END:VTIMEZONE
END:VCALENDAR