Book description
Get up to speed with Apache Drill, an extensible distributed SQL query engine that reads massive datasets in many popular file formats such as Parquet, JSON, and CSV. Drill reads data in HDFS or in cloud-native storage such as S3 and works with Hive metastores along with distributed databases such as HBase, MongoDB, and relational databases. Drill works everywhere: on your laptop or in your largest cluster.
In this practical book, Drill committers Charles Givre and Paul Rogers show analysts and data scientists how to query and analyze raw data using this powerful tool. Data scientists today spend about 80% of their time just gathering and cleaning data. With this book, you’ll learn how Drill helps you analyze data more effectively to drive down time to insight.
- Use Drill to clean, prepare, and summarize delimited data for further analysis
- Query file types including logfiles, Parquet, JSON, and other complex formats
- Query Hadoop, relational databases, MongoDB, and Kafka with standard SQL
- Connect to Drill programmatically using a variety of languages
- Use Drill even with challenging or ambiguous file formats
- Perform sophisticated analysis by extending Drill’s functionality with user-defined functions
- Facilitate data analysis for network security, image metadata, and machine learning
Publisher resources
Table of contents
- Preface
- 1. Introduction to Apache Drill
- 2. Installing and Running Drill
- 3. Overview of Apache Drill
- 4. Querying Delimited Data
- 5. Analyzing Complex and Nested Data
-
6. Connecting Drill to Data Sources
-
Querying Multiple Data Sources
- Configuring a New Storage Plug-in
- Connecting Drill to a Relational Database
- Querying Data in Hadoop from Drill
- Connecting to and Querying HBase from Drill
- Querying Hive Data from Drill
- Connecting to and Querying Streaming Data with Drill and Kafka
- Connecting to and Querying Kudu
- Connecting to and Querying MongoDB from Drill
- Connecting Drill to Cloud Storage
- Querying Time Series Data from Drill and OpenTSDB
- Conclusion
-
Querying Multiple Data Sources
- 7. Connecting to Drill
- 8. Data Engineering with Drill
- 9. Deploying Drill in Production
- 10. Setting Up Your Development Environment
- 11. Writing Drill User-Defined Functions
-
12. Writing a Format Plug-in
- The Example Regex Format Plug-in
- Creating the “Easy” Format Plug-in
- Creating the Regex Plug-in Configuration Class
- Creating the Format Plug-in Class
-
The Record Reader
- Testing the Reader Shell
- Logging
- Error Handling
- Setup
- Regex Parsing
- Defining Column Names
- Projection
- Column Projection Accounting
- Project None
- Project All
- Project Some
- Opening the File
- Record Batches
- Drill’s Columnar Structure
- Defining Vectors
- Reading Data
- Loading Data into Vectors
- Releasing Resources
- Testing the Reader
- Additional Details
- Conclusion
- 13. Unique Uses of Drill
- A. List of Drill Functions
- B. Drill Formatting Strings
- Index
Product information
- Title: Learning Apache Drill
- Author(s):
- Release date: November 2018
- Publisher(s): O'Reilly Media, Inc.
- ISBN: 9781492032793
You might also like
book
The Definitive Guide to Terracotta: Cluster the JVM™ for Spring, Hibernate, and POJO Scalability
Get the definitive guide on all the fundamentals of Terracotta as well as user secrets, recipes, …
book
Packet Analysis with Wireshark
Leverage the power of Wireshark to troubleshoot your networking issues by using effective packet analysis techniques …
book
Deploying a Modern Security Data Lake
Cybersecurity experts are overwhelmed, constantly on guard against malicious activity on their networks. Further complicating these …
book
Network Maintenance and Troubleshooting Guide: Field-Tested Solutions for Everyday Problems, Second Editon
Network Maintenance and Troubleshooting Guide Field-Tested Solutions for Everyday Problems, Second Edition Neal Allen The 100% …