From Pigeons to Artificial Neural Networks¶
Artificial neural networks draw inspiration from real neural networks. In order to understand deep learning using a "top-down" approach, we'll start with a familiar system capable of learning: a pigeon.
In 2015, researchers Levenson et al. from the University of California Davis trained pigeons to detect breast cancer. They show that the common pigeon can reliably distinguish between benign versus malignant tumors. The experimental setup is shown below.
The researchers trained pigeons by showing an image of a magnified biopsy to a pigeon. The pigeon then pecks at one of two answer buttons, labelling the image as malignant (cancerous) or benign (not cancerous). If the pigeon chooses correctly, researchers reward it with a tasty food pellet.
You can imagine that at the very beginning, the pigeons might peck randomly, perhaps not even pecking at the buttons at all. Eventually, the pigeon might accidentally peck at the correct button, and see a food pellet. This food pellet is extremely important, and is what guides the pigeon to change its behaviour.
In this chapter, we'll build an artificial pigeon instead of using a real one. Also, instead of using pigeons to detect cancer, we'll work on a simpler problem of categorizing digits. In order to use an artificial pigeon to solve our classification problem, we will need to:
