Why Adam work in optimizing language models: heavy-tailed data imbalance
Adam and related methods are the default optimizers in training language models because gradient descent is so slow it is thought it simply does not work. Many methods have been proposed to improve on Adam, but we have a poor understanding of why it works better in the first plac
Dieses Event ist auch Teil der öffentlichen Kalenderströme unten, sodass Besucher zur ganzen Reihe zurückkehren können statt nur zu dieser Einzelseite.
Zeit
12 Nov 2026 11:30 bis 12 Nov 2026 12:30
Ort
OAT, Room 11, ETH AI Center Foyer, Zurich
OAT, Room 11, ETH AI Center Foyer, Zurich, ch
Veröffentlicher
ETH AI Center
Öffentliche Kalender mit diesem Event
Dauer
1 Stunde
Verantwortlich
ETH AI Center
Source details
- Import status
- Regional Scan
- Source kind
- Nativ
- Updated
- vor 1 Stunde
- Public calendars
- Switzerland Tech Events
Eventkarte
OAT, Room 11, ETH AI Center Foyer, Zurich, ch
Kartendaten © OpenStreetMap-Mitwirkende
Adam and related methods are the default optimizers in training language models because gradient descent is so slow it is thought it simply does not work. Many methods have been proposed to improve on Adam, but we have a poor understanding of why it works better in the first place. This talk will be about understanding what makes language models hard for gradient descent, and how Adam fixes it. We Price / conditions: free