This guide is an ideal learning tool and reference for Apache Pig, the open source engine for executing parallel data flows on Hadoop. With Pig, you can batch-process data without having to create a full-fledged application—making it easy for you to experiment with new datasets.
Programming Pig introduces new users to Pig, and provides experienced users with comprehensive coverage on key features such as the Pig Latin scripting language, the Grunt shell, and User Defined Functions (UDFs) for extending Pig. If you need to analyze terabytes of data, this book shows you how to do it efficiently with Pig.
Delve into Pig’s data model, including scalar and complex data types
Write Pig Latin scripts to sort, group, join, project, and filter your data
Use Grunt to work with the Hadoop Distributed File System (HDFS)
Build complex data processing pipelines with Pig’s macros and modularity features
Embed Pig Latin in Python for iterative processing and other advanced tasks
Create your own load and store functions to handle data formats and storage mechanisms
Get performance tips for running scripts on Hadoop clusters in less time
Chapter 1 Introduction
What Is Pig?
Chapter 2 Installing and Running Pig
Downloading and Installing Pig
Chapter 3 Grunt
Entering Pig Latin Scripts in Grunt
HDFS Commands in Grunt
Controlling Pig from Grunt
Chapter 4 Pig’s Data Model
Chapter 5 Introduction to Pig Latin
Input and Output
User Defined Functions
Chapter 6 Advanced Pig Latin
Advanced Relational Operations
Integrating Pig with Legacy Code and MapReduce
Nonlinear Data Flows
Pig Latin Preprocessor
Chapter 7 Developing and Testing Pig Latin Scripts
Testing Your Scripts with PigUnit
Chapter 8 Making Pig Fly
Writing Your Scripts to Perform Well
Writing Your UDF to Perform
Tune Pig and Hadoop for Your Job
Using Compression in Intermediate Results
Data Layout Optimization
Bad Record Handling
Chapter 9 Embedding Pig Latin in Python
Chapter 10 Writing Evaluation and Filter Functions
Writing an Evaluation Function in Java
Writing Filter Functions
Chapter 11 Writing Load and Store Functions
Chapter 12 Pig and Other Members of the Hadoop Community
Pig and Hive
Metadata in Hadoop
Appendix Built-in User Defined Functions and Piggybank
Alan is an original member of the engineering team that took Pig from a Yahoo! Labs research project to a successful Apache open source project. In this role he oversaw the implementation of the language, including programming interfaces and the overall design. He has presented Pig at numerous conferences and user groups, universities, and companies. Alan is a member of the Apache Software Foundation and a co-founder of Hortonworks. He has a BS in Mathematics from Oregon State University and a MA in Theology from Fuller Theological Seminary.
The animal on the cover of Programming Pig is adomestic pig (Sus scrofa domesticus or Susdomesticus). While the larger pig family is naturally distributedin Africa, Asia, and Europe, domesticated pigs can now be found in nearlyevery part of the world that people inhabit. In fact, some pigs have beenspecifically bred to best equip them for various climates; for example,heavily coated varieties have been bred in colder climates. People havebrought pigs with them almost wherever they go for good reason: in additionto their primary use as a source of food, humans have been using the skin,bones, and hair of pigs to make various tools and implements formillennia.Domestic pigs are directly descended from wild boars, and evidencesuggests that there have been three distinct domestication events; the firsttook place in the Tigris River Basin as early as 13,000 BC, the second inChina, and the third in Europe, though the last likely occurred afterEuropeans were introduced to domestic pigs from the Middle East. Despite thelong history, however, taxonomists do not agree as to the properclassification for the domestic pig. Some believe that domestic pigs remainsimply a subspecies of the larger pig group including the wild boar(Sus scrofa), while others insist that they belong to aspecies all their own. In either case, there are several hundred breeds ofdomestic pig, each with its own particular characteristics.Perhaps because of their long history and prominent role in humansociety, and their tendency toward social behavior, domestic pigs haveappeared in film, literature, and other cultural media with regularity.Examples include “The Three Little Pigs,” Miss Piggy, and Porky the Pig.Additionally, domestic pigs have recently been recognized for theirintelligence and their ability to be trained (similar to dogs), and haveconsequently begun to be treated as pets.
While very informative, I felt that much of it was a rationalization of the decisions made creating Pig. The writing style was difficult to read at times as it made assumptions about the reader. I think it could have spent more time providing an introduction to Pig and less time providing an anatomical definition of each part of the language.
Bottom Line No, I would not recommend this to a friend
Programming Pig, the early release version, is a decent introduction to using Pig to process large scale data sets. As an early release version, it has some rough edges that will undoubtably be smoothed over before the final release, but some of the explanations of Pig's more complex areas don't have a satisfactory, simple explanation.
Bottom Line Yes, I would recommend this to a friend
I've been reading an early draft, so it's still a bit rough around the edges, BUT this is an excellent book that covers the details of using Pig, from basic to advanced features. It saved my bacon (if you'll pardon the expression...) numerous times on a recent, challenging project. It has lots of good examples and debugging tips, too.
Bottom Line Yes, I would recommend this to a friend