Ready to use statistical and machine-learning techniques across large data sets? This practical guide shows you why the Hadoop ecosystem is perfect for the job. Instead of deployment, operations, or software development usually associated with distributed computing, you’ll focus on particular analyses you can build, the data warehousing techniques that Hadoop provides, and higher order data workflows this framework can produce.
Data scientists and analysts will learn how to perform a wide range of techniques, from writing MapReduce and Spark applications with Python to using advanced modeling and data management with Spark MLlib, Hive, and HBase. You’ll also learn about the analytical processes and data systems available to build and empower data products that can handle—and actually require—huge amounts of data.
Understand core concepts behind Hadoop and cluster computing
Use design patterns and parallel analytical algorithms to create distributed data analysis jobs
Learn about data management, mining, and warehousing in a distributed context using Apache Hive and HBase
Use Sqoop and Apache Flume to ingest data from relational databases
Program complex Hadoop and Spark applications with Apache Pig and Spark DataFrames
Perform machine learning techniques such as classification, clustering, and collaborative filtering with Spark’s MLlib
Introduction to Distributed Computing
Chapter 1The Age of the Data Product
What Is a Data Product?
Building Data Products at Scale with Hadoop
The Data Science Pipeline and the Hadoop Ecosystem
Chapter 2An Operating System for Big Data
Working with a Distributed File System
Working with Distributed Computation
Submitting a MapReduce Job to YARN
Chapter 3A Framework for Python and Hadoop Streaming
A Framework for MapReduce with Python
Chapter 4In-Memory Computing with Spark
Interactive Spark Using PySpark
Writing Spark Applications
Chapter 5Distributed Analysis and Patterns
Computing with Keys
Toward Last-Mile Analytics
Workflows and Tools for Big Data Science
Chapter 6Data Mining and Warehousing
Structured Data Queries with Hive
Chapter 7Data Ingestion
Importing Relational Data with Sqoop
Ingesting Streaming Data with Flume
Chapter 8Analytics with Higher-Level APIs
Spark’s Higher-Level APIs
Chapter 9Machine Learning
Scalable Machine Learning with Spark
Chapter 10Summary: Doing Distributed Data Science
Data Product Lifecycle
Machine Learning Lifecycle
Appendix Creating a Hadoop Pseudo-Distributed Development Environment
Setting Up Linux
Appendix Installing Hadoop Ecosystem Products
Packaged Hadoop Distributions
Self-Installation of Apache Hadoop Ecosystem Products
Benjamin Bengfort is a Data Scientist who lives inside the beltway but ignores politics (the normal business of DC) favoring technology instead. He is currently working to finish his PhD at the University of Maryland where he studies machine learning and distributed computing. His lab does have robots (though this field of study is not one he favors) and, much to his chagrin, they seem to constantly arm said robots with knives and tools; presumably to pursue culinary accolades. Having seen a robot attempt to slice a tomato, Benjamin prefers his own adventures in the kitchen where he specializes in fusion French and Guyanese cuisine as well as BBQ of all types. A professional programmer by trade, a Data Scientist by vocation, Benjamin's writing pursues a diverse range of subjects from Natural Language Processing, to Data Science with Python to analytics with Hadoop and Spark.
Jenny Kim is an experienced big data engineer who works in both commercial software efforts as well as in academia. She has significant experience in working with large scale data, machine learning, and Hadoop implementations in production and research environments. Jenny (with Benjamin Bengfort) previously built a large scale recommender system that used a web crawler to gather ontological information about apparel products and produce recommendations from transactions. Currently, she is working with the Hue team at Cloudera, to help build intuitive interfaces for analyzing big data with Hadoop.
Comments about oreilly Data Analytics with Hadoop:
I really like this book. It is a great overview of a plethora of topics around doing scalable data analytics and data science. It is extremely up-to date, going through techniques that have existed for many years now like MapReduce, but also newer systems like Spark, all in the context of the Hadoop eco-system. They go into machine learning techniques, data management, and overall paint a nice picture around what data science is, and why data products are important, while teaching you how to make them!
Every single concept is explained in a clear and concise manner, and wherever details are omitted there is always a citation to a source where the reader can continue reading more about it, which I think is great. Although I wouldn't classify myself as a beginner, I believe it is friendly to both professionals and beginners, as it is centered around python which makes most examples (that are conveniently uploaded in a nice github repository) really easy to simply run and play around with. After describing something, whether that would be a technique for data analysis, or just the in-and outer workings of some analysis platform like HBase, Hive etc, the authors provide examples so that while you're reading about this stuff you can also run it, play around with it and really explore how these systems function; I believe this is a crucial part of familiarizing ones' self with new platforms.
Another thing I enjoyed a lot was the ending of this book. After you really dive into all of these systems and get your feet wet with each one of them, the authors wrap it all up in a nice bow by taking a step back and describing the entire end-to-end process of how you would go about productively using the knowledge you've gotten from this book to build data analytics workflows! I highly recommend this to anyone who both knows that they want to learn how to deploy scalable analytics workflows in 2016, but also to readers who are simply just curious about data science; this book will suck you in!
Bottom Line Yes, I would recommend this to a friend