Your author is a long-time artificial-intelligence researcher whose field of expertise, natural-language processing, has been revolutionized by deep learning. Unfortunately, it took him (me) a long time to catch on to this fact. I can rationalize this since this is the third time neural networks have threatened a revolution but only the first time they have delivered. Nevertheless, I suddenly found myself way behind the times and struggling to catch up. So I did what any self-respecting professor would do, scheduled myself to teach the stuff, started a crash course by surfing the web, and got my students to teach it to me. (This last is not a joke. In particular, the head undergraduate teaching assistant for the course, Siddarth (Sidd) Karramcheti, deserves special mention.)
This explains several prominent features of this book. First, it is short. I am a slow learner. Second, it is very much project driven. Many texts, particularly in computer science, have a constant tension between topic organization and organizing material around specific projects. Splitting thedifference is often a good idea, but I find I learn computer science materialbest by sitting down and writing programs, so my book largely re ects mylearning habits. It was the most convenient way to put it down, and I amhoping many in the expected audience will find it helpful as well.
Which brings up the question of the expected audience. While I hopemany CS practitioners will find this book useful for the same reason I wroteit, as a teacher my first loyalty is to my students, so this book is primarilyintended as a textbook for a course on deep learning. The course I teach atBrown is for both graduate and undergraduates and covers all the materialherein, plus some \culture” lectures (for graduate credit a student must adda significant final project). Both linear algebra and multivariate calculus arerequired. While the actual quantity of linear-algebra material is not thatgreat, students have told me that without it they would have found thinkingabout multilevel networks, and the tensors they require, quite dificult.Multivariate calculus, however, was a much closer call. It appears explicitly only in Chapter 1, when we build up to back-propagation from scratch andI would not be surprised if an extra lecture on partial derivatives would do.Last, there is a probability and statistics prerequisite. This simplifies theexposition and I certainly want to encourage students to take such a course.I also assume a rudimentary knowledge of programming in Python. I do notinclude this in the text, but my course has an extra \lab” on basic Python.That your author was playing catch-up when writing this book alsoexplains the fact that in almost every chapter’s section on further readingyou will find, beyond the usual references to important research papers,many reference to secondary sources |others’ educational writings. I wouldnever have learned this material without them.
