Tools, Techniques, and Workflows to Train Deep Neural Networks
By O'Reilly Media, Inc.
Publisher: O'Reilly Media
Final Release Date: May 2016
Run time: 3 hours 15 minutes
Quickly dive-in to the most current topics in the field of deep learning, with this curated collection from Strata + Hadoop World 2016. This video features presentations from speakers in both academia and industry, covering the essentials of deep learning, and how to use the latest tools and techniques. Viewers will learn:
Benefits of deep learning over other machine learning techniques
Advances in the field
Elements of deep learning workflows, including how to address common challenges
How to handle large-scale distributed training of neural networks, using TensorFlow
A scalable implementation of deep neural networks for Spark
How to use SparkNet to construct deep networks using existing libraries
Over the last year, machine learning (ML) has become the hottest topic in the data world, and most companies are now investigating how to build and deploy intelligent applications backed by ML. However, the journey to build such applications starts with understanding the math underlying ML models, rather than focusing on how embedding such ML models can make an app intelligent. The resulting process is hugely time consuming, requires deep subject expertise, and slows the speed of creation of new intelligent applications.
Naveen Rao discusses deep learning, a form of machine learning loosely inspired by the brain. Naveen explores the benefits of deep learning over other machine-learning techniques, recent advances in the field, the deep learning workflow, challenges in developing and deploying deep learning-based solutions, and the need for standardized tools for building and scaling deep learning solutions.
What is Deep Learning? - Naveen Rao (Nervana) 01m 37s
Deep Learning Model Architectures - Naveen Rao (Nervana) 03m 43s
Automating Human Tasks - Naveen Rao (Nervana) 07m 19s
Challenges in Deep Learning - Naveen Rao (Nervana) 03m 44s
TensorFlow is an open source software library for numerical computation using data flow graphs. Nodes in the graph represent mathematical operations, while the graph edges represent the multidimensional data arrays (tensors) communicated between them. The flexible architecture allows you to deploy computation to one or more CPUs or GPUs in a desktop, server, or mobile or embedded device with a single API.
Pushing the Boundaries of Machine Learning - Rajat Monga (Google) 02m 39s
What is TensorFlow? - Rajat Monga (Google) 03m 16s
TensorFlow: Under the Hood - Rajat Monga (Google) 02m 59s
Machine Learning in TensorFlow - Rajat Monga (Google) 05m 53s
Updates to TensorFlow - Rajat Monga (Google) 03m 58s
TensorFlow Applications, including Spark - Rajat Monga (Google) 04m 42s
Google Cloud Machine Learning Platform - Rajat Monga (Google) 01m 38s
TensorFlow is an open source software library for machine learning, based on previous generations of software within Google for training and deploying neural networks. BigQuery is Google’s fully managed, low-cost analytics data warehouse, which lets you do interactive queries on petabyte-sized datasets. Google Cloud Dataflow (now Beam, an Apache incubator project) is a unified programming model and service for developing and executing a wide range of data processing and analytics patterns. Together, they enable a powerful workflow for distributed machine learning.
Deep Learning and Distributed Training - Kazunori Sato and Amy Unruhand (Google) 05m 40s
Google Brain, Google Cloud - Kazunori Sato and Amy Unruhand (Google) 04m 57s
Neural Networks in Production - Kazunori Sato and Amy Unruhand (Google) 03m 07s
Demo: Cloud-vision API Explorer - Kazunori Sato and Amy Unruhand (Google) 02m 33s
Demo: Cloud-speech API - Kazunori Sato and Amy Unruhand (Google) 02m 36s
TensorFlow for Deep Learning - Kazunori Sato and Amy Unruhand (Google) 05m 05s
Learning New Image Classes using Apache Beam - Kazunori Sato and Amy Unruhand (Google) 05m 51s
Demo: Using Cloud Datalab - Kazunori Sato and Amy Unruhand (Google) 06m 32s
Artificial neural networks (ANN) are popular models for machine learning, in particular for deep learning. The models that are used in practice for image classification and speech recognition contain a huge number of weights and are trained with big datasets, but training such models is challenging in terms of computation and data processing. Alexander Ulanov proposes a scalable implementation of deep neural networks for Spark.
What is an Artificial Neural Network? - Alexander Ulanov (Hewlett-Packard Labs) 03m 58s
Implementing the Multilayer Perceptron in Spark - Alexander Ulanov (Hewlett-Packard Labs) 04m 04s
Basic Linear Algebra Subprograms (BLAS) - Alexander Ulanov (Hewlett-Packard Labs) 07m 52s
Scalability - Alexander Ulanov (Hewlett-Packard Labs) 05m 51s
Scalability Testing - Alexander Ulanov (Hewlett-Packard Labs) 04m 19s
While there has been a lot of recent progress, deep learning presents a very different workload from what systems like Spark are optimized for. In particular, these workloads are often bottlenecked by communication. While the cost of communication between machines can be improved with better hardware, this bottleneck limits the benefit of distributed training in settings like EC2.
Sparknet for Deep Learning - Robert Nishihara (UC Berkeley) 11m 13s
How to Read Data into Neural Nets - Robert Nishihara (UC Berkeley) 03m 40s
How to Use SparkNet - Robert Nishihara (UC Berkeley) 07m 59s
Distributed Training for Neutral Nets - Robert Nishihara (UC Berkeley) 08m 08s
Experiment Using AlexNet - Robert Nishihara (UC Berkeley) 02m 33s
Experiment Using GoogleNet - Robert Nishihara (UC Berkeley) 01m 59s
Parallelization Schemes - Robert Nishihara (UC Berkeley) 02m 32s
SparkNet Code & Paper - Robert Nishihara (UC Berkeley) 04m 44s