Fundamentals of Deep Learning

Your Account

Shopping Cart 0 items $0.00

Your shopping cart is empty.

See how Facebook, Airbnb, Target, and IBM did it. Introducing O'Reilly case studies, a new way to learn, only on Safari. Learn more

Search Inside and Read

Designing Next-Generation Machine Intelligence Algorithms

Larger Cover

Fundamentals of Deep Learning

Designing Next-Generation Machine Intelligence Algorithms

By Nikhil Buduma

Publisher: O'Reilly Media

Final Release Date: May 2017

Pages: 304

With the reinvigoration of neural networks in the 2000s, deep learning has become an extremely active area of research that is paving the way for modern machine learning. This book uses exposition and examples to help you understand major concepts in this complicated field.

Large companies such as Google, Microsoft, and Facebook have taken notice, and are actively growing in-house deep learning teams. For the rest of us however, deep learning is still a pretty complex and difficult subject to grasp. If you have a basic understanding of what machine learning is, have familiarity with the Python programming language, and have some mathematical background with calculus, this book will help you get started.

Chapter 1The Neural Network
1. Building Intelligent Machines
2. The Limits of Traditional Computer Programs
3. The Mechanics of Machine Learning
4. The Neuron
5. Expressing Linear Perceptrons as Neurons
6. Feed-Forward Neural Networks
7. Linear Neurons and Their Limitations
8. Sigmoid, Tanh, and ReLU Neurons
9. Softmax Output Layers
10. Looking Forward
Chapter 2Training Feed-Forward Neural Networks
1. The Fast-Food Problem
2. Gradient Descent
3. The Delta Rule and Learning Rates
4. Gradient Descent with Sigmoidal Neurons
5. The Backpropagation Algorithm
6. Stochastic and Minibatch Gradient Descent
7. Test Sets, Validation Sets, and Overfitting
8. Preventing Overfitting in Deep Neural Networks
9. Summary
Chapter 3Implementing Neural Networks in TensorFlow
1. What Is TensorFlow?
2. How Does TensorFlow Compare to Alternatives?
3. Installing TensorFlow
4. Creating and Manipulating TensorFlow Variables
5. TensorFlow Operations
6. Placeholder Tensors
7. Sessions in TensorFlow
8. Navigating Variable Scopes and Sharing Variables
9. Managing Models over the CPU and GPU
10. Specifying the Logistic Regression Model in TensorFlow
11. Logging and Training the Logistic Regression Model
12. Leveraging TensorBoard to Visualize Computation Graphs and Learning
13. Building a Multilayer Model for MNIST in TensorFlow
14. Summary
Chapter 4Beyond Gradient Descent
1. The Challenges with Gradient Descent
2. Local Minima in the Error Surfaces of Deep Networks
3. Model Identifiability
4. How Pesky Are Spurious Local Minima in Deep Networks?
5. Flat Regions in the Error Surface
6. When the Gradient Points in the Wrong Direction
7. Momentum-Based Optimization
8. A Brief View of Second-Order Methods
9. Learning Rate Adaptation
10. The Philosophy Behind Optimizer Selection
11. Summary
Chapter 5Convolutional Neural Networks
1. Neurons in Human Vision
2. The Shortcomings of Feature Selection
3. Vanilla Deep Neural Networks Don’t Scale
4. Filters and Feature Maps
5. Full Description of the Convolutional Layer
6. Max Pooling
7. Full Architectural Description of Convolution Networks
8. Closing the Loop on MNIST with Convolutional Networks
9. Image Preprocessing Pipelines Enable More Robust Models
10. Accelerating Training with Batch Normalization
11. Building a Convolutional Network for CIFAR-10
12. Visualizing Learning in Convolutional Networks
13. Leveraging Convolutional Filters to Replicate Artistic Styles
14. Learning Convolutional Filters for Other Problem Domains
15. Summary
Chapter 6Embedding and Representation Learning
1. Learning Lower-Dimensional Representations
2. Principal Component Analysis
3. Motivating the Autoencoder Architecture
4. Implementing an Autoencoder in TensorFlow
5. Denoising to Force Robust Representations
6. Sparsity in Autoencoders
7. When Context Is More Informative than the Input Vector
8. The Word2Vec Framework
9. Implementing the Skip-Gram Architecture
10. Summary
Chapter 7Models for Sequence Analysis
1. Analyzing Variable-Length Inputs
2. Tackling seq2seq with Neural N-Grams
3. Implementing a Part-of-Speech Tagger
4. Dependency Parsing and SyntaxNet
5. Beam Search and Global Normalization
6. A Case for Stateful Deep Learning Models
7. Recurrent Neural Networks
8. The Challenges with Vanishing Gradients
9. Long Short-Term Memory (LSTM) Units
10. TensorFlow Primitives for RNN Models
11. Implementing a Sentiment Analysis Model
12. Solving seq2seq Tasks with Recurrent Neural Networks
13. Augmenting Recurrent Networks with Attention
14. Dissecting a Neural Translation Network
15. Summary
Chapter 8Memory Augmented Neural Networks
1. Neural Turing Machines
2. Attention-Based Memory Access
3. NTM Memory Addressing Mechanisms
4. Differentiable Neural Computers
5. Interference-Free Writing in DNCs
6. DNC Memory Reuse
7. Temporal Linking of DNC Writes
8. Understanding the DNC Read Head
9. The DNC Controller Network
10. Visualizing the DNC in Action
11. Implementing the DNC in TensorFlow
12. Teaching a DNC to Read and Comprehend
13. Summary
Chapter 9Deep Reinforcement Learning
1. Deep Reinforcement Learning Masters Atari Games
2. What Is Reinforcement Learning?
3. Markov Decision Processes (MDP)
4. Explore Versus Exploit
5. Policy Versus Value Learning
6. Pole-Cart with Policy Gradients
7. Q-Learning and Deep Q-Networks
8. Improving and Moving Beyond DQN
9. Summary

Title:

Fundamentals of Deep Learning

By:

Nikhil Buduma

With:

Nicholas Locascio

Publisher:

O'Reilly Media

Formats:

Print
Ebook
Safari Books Online

Print:

June 2017 (est.)

Ebook:

May 2017

Pages:

304 (est.)

Print ISBN:

978-1-4919-2561-4

| ISBN 10:

1-4919-2561-2

Ebook ISBN:

978-1-4919-2555-3

| ISBN 10:

1-4919-2555-8

Table of Contents

Product Details

About the Author

Colophon

Recommended for You

Customer Reviews

REVIEW SNAPSHOT®

by PowerReviews

oreillyFundamentals of Deep Learning

3.3

(based on 22 reviews)

64%

of respondents would recommend this to a friend.

Pros

Easy to understand (11)
Helpful examples (11)
Well-written (10)
Accurate (9)
Concise (7)

Cons

Not comprehensive enough (3)

Best Uses

Intermediate (11)
Novice (8)
Student (5)

- Reviewer Profile:
- Developer (10), Educator (6)

Reviewed by 22 customers

Sort by

Displaying reviews 1-10

Previous | Next »

5/2/2017

3.0

Margins eat code

By JAP

from Illinois

About Me Educator

Pros

Helpful examples

Cons

Best Uses

Intermediate

Comments about oreilly Fundamentals of Deep Learning:

Book is great but this release has many pages with code incomplete because the right margin eats a portion of the lines of code!! can you fix please?

4/11/2017

1.0

Cash grab

By "I am not mad I am disappointed"

from Warsaw, Poland

About Me Data Scientist

Pros

Cons

Not comprehensive enough
Too basic
Too late
Too many errors

Best Uses

Comments about oreilly Fundamentals of Deep Learning:

This book really makes me doubt O'Reilly's professionalism as a publisher.

1/3/2017

(6 of 10 customers found this review helpful)

1.0

I feel like I have been scammed

By YCeron

from Somewhere over the rainbow

Comments about oreilly Fundamentals of Deep Learning:

I know this is an early release book but the release date keeps changing. I feel like I have been scammed.At this point it seems this book will never be released.
I am very disappointed.

12/7/2016

(1 of 2 customers found this review helpful)

4.0

perfect balance between intution and math

By Mircea

from Deutschland

Comments about oreilly Fundamentals of Deep Learning:

I like the book because it goes enough into the theoretical explanations but also builds your intuition with nice examples, it's a perfect balance between theory and practice. I guess it's best suited for the curious software engineer. I only which there were more chapters ready!

12/1/2016

(4 of 6 customers found this review helpful)

1.0

Too little, too slow

By atreo1

from United Kingdom

About Me Data Scientist, Designer

Pros

Accurate
Concise
Easy to understand

Cons

Not comprehensive enough
Too many errors
Unfinished

Best Uses

Novice
Student

Comments about oreilly Fundamentals of Deep Learning:

Good introduction to neural networks, going straight to the point and with examples. But the main problem is that two or three well selected tutorials will cover the chapters so far with equally good quality. And the book is depressingly delayed until March 2017. It seems it will never be finished, at this rate, and even if it is, how good and updated can it be after years?

10/6/2016

(0 of 1 customers found this review helpful)

4.0

So far, so good!

By Speaker to Computers

from Washington DC

About Me Developer, Sys Admin

Pros

Accurate
Helpful examples
Well-written

Cons

Best Uses

Intermediate

Comments about oreilly Fundamentals of Deep Learning:

Based on the content available so far (Chs. 1-6), this is an excellent introduction to deep neural network techniques, and in particular clarifies why deep networks are operationally different from their simpler precursors like perceptrons. The recently-updated Chapter 6 includes a long-awaited, cogent and complete description of the deep-network auto-encoder strategy, which is how deep neural networks are used to do feature extraction.

I have one minor complaint, which is that many of the examples are very strongly tool-centered. This is high value for reproducing the results in TensorFlow (the tool used in this book), which is open-source and widely available, but I feel that the examples would be enhanced if they were also accompanied by a more detailed tool-agnostic description of the steps. The danger is that this might bog down the narrative, of course, so perhaps I should be careful what I wish for, but that's what has struck me so far.

9/13/2016

(8 of 9 customers found this review helpful)

1.0

This book is a scam

By Frank

from Dublin

About Me Developer

Comments about oreilly Fundamentals of Deep Learning:

As a number of other reviewers have noted, this book is not even 50% complete. O'Reilly, what are you going to do about this? You can't just offer books for pre-release purchase and then not finish the book - and this is after a full year of waiting. What's going on?

Will you be offering a refund?

Regards,
A usually happy O'Reilly customer, who will be looking elsewhere in future for his tech book needs, if this is not dealt with.

8/25/2016

(9 of 10 customers found this review helpful)

1.0

Full Price; Half Book

By Richard

from Tampa, FL

Pros

Easy to understand

Cons

Best Uses

Comments about oreilly Fundamentals of Deep Learning:

I purchased this book with the impression that it would be completed soon since it is on a rapidly evolving topic which has a limited lifetime.

14 months later and the book is not even 50% complete and the web page that a link to see the release history has no information about upcoming releases.

I believe that Oreilly should refund my money and quit offering it for sale even with the disclaimer that it is an early release version. 14 months for a current technical topic is a lifetime.

7/5/2016

(1 of 5 customers found this review helpful)

5.0

Great book

By mcasl

from León, Spain

About Me Educator, Researcher

Pros

Accurate
Concise
Easy to understand
Well-written

Cons

Not comprehensive enough

Best Uses

Intermediate
Novice

Comments about oreilly Fundamentals of Deep Learning:

I am reading the early released version of the book. So far the book provides very clear explanations and I am looking forward to read more chapters as they are ready.

5/5/2016

(6 of 7 customers found this review helpful)

4.0

Pretty good so far

By Robert

from San Jose, CA

Comments about oreilly Fundamentals of Deep Learning:

It's pretty good so far, but it clearly needs some proofreading and it really needs to be finished. Is there a schedule?

Displaying reviews 1-10

Previous | Next »

Immediate Access - Go Digital what's this?

Ebook: $37.99

Formats: DAISY, ePub, Mobi, PDF

Print & Ebook: $48.39

Pre-Order Print: $43.99

June 2017 (est.)

Safari Books Online - Read now >

Chapter 1The Neural Network

Building Intelligent Machines

The Limits of Traditional Computer Programs

The Mechanics of Machine Learning

The Neuron

Expressing Linear Perceptrons as Neurons

Feed-Forward Neural Networks

Linear Neurons and Their Limitations

Sigmoid, Tanh, and ReLU Neurons

Softmax Output Layers

Looking Forward

Chapter 2Training Feed-Forward Neural Networks

The Fast-Food Problem

Gradient Descent

The Delta Rule and Learning Rates

Gradient Descent with Sigmoidal Neurons

The Backpropagation Algorithm

Stochastic and Minibatch Gradient Descent

Test Sets, Validation Sets, and Overfitting

Preventing Overfitting in Deep Neural Networks

Summary

Chapter 3Implementing Neural Networks in TensorFlow

What Is TensorFlow?

How Does TensorFlow Compare to Alternatives?

Installing TensorFlow

Creating and Manipulating TensorFlow Variables

TensorFlow Operations

Placeholder Tensors

Sessions in TensorFlow

Navigating Variable Scopes and Sharing Variables

Managing Models over the CPU and GPU

Specifying the Logistic Regression Model in TensorFlow

Logging and Training the Logistic Regression Model

Leveraging TensorBoard to Visualize Computation Graphs and Learning

Building a Multilayer Model for MNIST in TensorFlow

Summary

Chapter 4Beyond Gradient Descent

The Challenges with Gradient Descent

Local Minima in the Error Surfaces of Deep Networks

Model Identifiability

How Pesky Are Spurious Local Minima in Deep Networks?

Flat Regions in the Error Surface

When the Gradient Points in the Wrong Direction

Momentum-Based Optimization

A Brief View of Second-Order Methods

Learning Rate Adaptation

The Philosophy Behind Optimizer Selection

Summary

Chapter 5Convolutional Neural Networks

Neurons in Human Vision

The Shortcomings of Feature Selection

Vanilla Deep Neural Networks Don’t Scale

Filters and Feature Maps

Full Description of the Convolutional Layer

Max Pooling

Full Architectural Description of Convolution Networks

Closing the Loop on MNIST with Convolutional Networks

Image Preprocessing Pipelines Enable More Robust Models

Accelerating Training with Batch Normalization

Building a Convolutional Network for CIFAR-10

Visualizing Learning in Convolutional Networks

Leveraging Convolutional Filters to Replicate Artistic Styles

Learning Convolutional Filters for Other Problem Domains

Summary

Chapter 6Embedding and Representation Learning

Learning Lower-Dimensional Representations

Principal Component Analysis

Motivating the Autoencoder Architecture

Implementing an Autoencoder in TensorFlow

Denoising to Force Robust Representations

Sparsity in Autoencoders

When Context Is More Informative than the Input Vector

The Word2Vec Framework

Implementing the Skip-Gram Architecture

Summary

Chapter 7Models for Sequence Analysis

Analyzing Variable-Length Inputs

Tackling seq2seq with Neural N-Grams

Implementing a Part-of-Speech Tagger

Dependency Parsing and SyntaxNet