Neural Networks and Deep Learning – Charu C.Aggarwal

Neural networks were developed to simulate the human nervous system for machine learning tasks by treating the computational units in a learning model in a manner similar to human neurons. The grand vision of neural networks is to create artificial intelligence by building machines whose architecture simulates the computations in the human nervous system. This is obviously not a simple task because the computational power of the fastest computer today is a minuscule fraction of the computational power of a human brain. Neural networks were developed soon after the advent of computers in the fifties and sixties. Rosenblatt’s perceptron algorithm was seen as a fundamental cornerstone of neural networks, which caused an initial excitement about the prospects of artificial intelligence. However, after the initial euphoria, there was a period of disappointment in which the data hungry and computationally intensive nature of neural networks was seen as an impediment to their usability. Eventually, at the turn of the century, greater data availability and increasing computational power lead to increased successes of neural networks, and this area was reborn under the new label of “deep learning.” Although we are still far from the day that artificial intelligence (AI) is close to human performance, there are specific domains like image recognition, self-driving cars, and game playing, where AI has matched or exceeded human performance. It is also hard to predict what AI might be able to do in the future. For example, few computer vision experts would have thought two decades ago that any automated system could ever perform an intuitive task like categorizing an image more accurately than a human.

Neural networks are theoretically capable of learning any mathematical function with sufficient training data, and some variants like recurrent neural networks are known to be Turi ng compl ete. Turing completeness refers to the fact that a neural network can simulate any learning algorithm, given sufficient training data. The sticking point is that the amount of data required to learn even simple tasks is often extraordinarily large, which causes a corresponding increase in training time (if we assume that enough training data is available in the first place). For example, the training time for image recognition, which is a simple task for a human, can be on the order of weeks even on high-performance systems. Furthermore, there are practical issues associated with the stability of neural network training, which are being resolved even today. Nevertheless, given that the speed of computers is expected to increase rapidly over time, and fundamentally more powerful paradigms like quantum computing are on the horizon, the computational issue might not eventually turn out to be quite as critical as imagined.

Related posts:

Understanding Machine Learning from theory to algorithms - Shai Shalev-Shwartz & Shai Ben-David
Hands-on Machine Learning with Scikit-Learn, Keras & TensorFlow - Aurelien Geron
Deep Learning in Python - LazyProgrammer
Machine Learning with spark and python - Michael Bowles
Python Machine Learning Eqution Reference - Sebastian Raschka
Deep Learning with Hadoop - Dipayan Dev
Artificial Intelligence by example - Denis Rothman
Neural Networks - A visual introduction for beginners - Michael Taylor
Medical Image Segmentation Using Artificial Neural Networks
Machine Learning Applications Using Python - Cases studies form Healthcare, Retail, and Finance - Pu...
Building Machine Learning Systems with Python - Willi Richert & Luis Pedro Coelho
Python Data Analytics with Pandas, NumPy and Matplotlib - Fabio Nelli
Python Artificial Intelligence Project for Beginners - Joshua Eckroth
Python Machine Learning - Sebastian Raschka
Deep Learning - Ian Goodfellow & Yoshua Bengio & Aaron Courville
Natural Language Processing with Python - Steven Bird & Ewan Klein & Edward Loper
Natural Language Processing in action - Hobson Lane & Cole Howard & Hannes Max Hapke
Deep Learning with Theano - Christopher Bourez
Introduction to Deep Learning - Eugene Charniak
Superintelligence - Paths, Danges, Strategies - Nick Bostrom
Learn Keras for Deep Neural Networks - Jojo Moolayil
Deep Learning with Keras - Antonio Gulli & Sujit Pal
R Deep Learning Essentials - Dr. Joshua F.Wiley
Deep Learning dummies second edition - John Paul Mueller & Luca Massaronf
An introduction to neural networks - Kevin Gurney & University of Sheffield
Machine Learning Mastery with Python - Understand your data, create accurate models and work project...
Machine Learning with Python for everyone - Mark E.Fenner
Deep Learning for Natural Language Processing - Palash Goyal & Sumit Pandey & Karan Jain
Deep Learning with Python - Francois Cholletf
Data Science and Big Data Analytics - EMC Education Services
Generative Deep Learning - Teaching Machines to Paint, Write, Compose and Play - David Foster
Statistical Methods for Machine Learning - Disconver how to Transform data into Knowledge with Pytho...