Company
Date Published
Sept. 5, 2024
Author
Sparsh Bhasin
Word count
703
Language
English
Hacker News points
None

Summary

Grokkfast is a new optimization algorithm that accelerates the learning process in neural networks by speeding up the generalization process. It has been implemented in MonsterAPI, a finetuning platform for machine-learning projects. Experimental results show promising improvements across various tasks, with significant gains when training models from scratch or tackling more challenging problems. Grokkfast can be used in MonsterAPI by setting the optimizer to "grokadamw" and sending the payload to the endpoint.