Reactive Optimizers: rewrite the optimizer's source code during training
22 points by matt_d
A new approach to optimization involves an agent that monitors training dynamics and adapts the optimizer’s source code in real time. This method allows for dynamic adjustments to the optimization process based on current training conditions, potentially improving training efficiency and outcomes.