The goal of this exercise is to map out the foundational principles of machine learning in a “simple,” Occam’s Razor type of way. They say the b

Engineering the Future

submited by

Style Pass

2024-07-01 04:00:06

The goal of this exercise is to map out the foundational principles of machine learning in a “simple,” Occam’s Razor type of way. They say the best way to learn is to teach. I have been reading Deep Learning by Ian Goodfellow and have made it through the foundations of these models. Chapter 9 onward appears to be much more practical applications and it makes sense for me to put down the book and synthesize the 320 pages of hard math/concepts by writing and working on projects (I learn by doing).

What follows below is an overview of chapters 1-8, some derived principles of deep learning that we can use to properly frame the future we are entering, and lastly an annoyingly over-philosophical take on all that is occurring that I couldn't help but write down.

In the above quote, Einstein echoes Occam’s Razor, the principle that if you have two competing ideas to explain the same phenomenon, you should prefer the simpler one. This concept is also foundational to machine learning. If your model is too complex, you overfit, if it is too simple and abstract, you don’t capture the phenomenon accurately. I’ve written about Abstraction vs First Principles before.