Create accountLogin

Recent searches

No recent searches

Search options

Only available when logged in.
ohai.social is one of the many independent Mastodon servers you can use to participate in the fediverse.
A cozy, fast and secure Mastodon server where everyone is welcome. Run by the folks at ohai.is.

Administered by:

Server stats:

1.9K
active users

ohai.social: AboutStatusProfiles directoryPrivacy policy

Mastodon: AboutGet the appKeyboard shortcutsView source codev4.4.0-nightly.2025-03-25

#with

10 posts8 participants0 posts today
Public
Hacker News

Optimizing ML Training with Metagradient Descent

arxiv.org/abs/2503.13751

arXiv logo
arXiv.orgOptimizing ML Training with Metagradient DescentA major challenge in training large-scale machine learning models is configuring the training process to maximize model performance, i.e., finding the best training setup from a vast design space. In this work, we unlock a gradient-based approach to this problem. We first introduce an algorithm for efficiently calculating metagradients -- gradients through model training -- at scale. We then introduce a "smooth model training" framework that enables effective optimization using metagradients. With metagradient descent (MGD), we greatly improve on existing dataset selection methods, outperform accuracy-degrading data poisoning attacks by an order of magnitude, and automatically find competitive learning rate schedules.
#HackerNews#Optimizing#ML
ExploreLive feeds
About