Learning a neural network – Part 1

This is Chapter 3 of the book titled "Deep Learning": a nine-part easy-to-grasp textbook written with the goal of demystifying the science behind contemporary AI. The book attempts to explain the fundamentals of Deep  Learning to readers from a wide range of backgrounds in an intuitive manner,  while navigating its strong mathematical foundations in a highly pictorial style.  This chapter is also the first in the series of chapters under Part II of this book,  dealing with neural network training. In Chapters 1 and 2, we saw that  any function can be modeled by an appropriately architected neural network. However, for the network to model a function accurately, its parameters must  be properly assigned. In this chapter we discuss the problem of how the   parameters of a network can be learned from training data, and how the  learning can be cast as an optimization problem that can be solved through  the method of gradient descent.