GitHub - Carloscodix/qapla: A char-level transformer trained from scratch on an $8 ESP32-S3. Not inference: the chip runs the full training loop, with backprop written by hand in C.

An SLM trained on $8 ESP32-S3

A char-level transformer trained from scratch on an $8 ESP32-S3. Not inference: the chip runs the full training loop, with backprop written by hand in C. - Carloscodix/qapla

56 points by pavelai 78 days ago github.com 17 comments

Summary

A low-cost ESP32-S3 has been used to train a character-level transformer from scratch, with the entire training process, including backpropagation, implemented manually in C. This project showcases the potential of inexpensive hardware for machine learning tasks.

Comments (17 of ) ⌛

Loading more comments... ⌛

Scroll down to load more comments

No comments available.