Books & Videos

Data Science at the Command Line

Facing the Future with Time-Tested Tools

By Jeroen Janssens

Publisher: O'Reilly Media

Release Date: October 2014

Pages: 212

Read on Safari with a 10-day trial

Start your free trial now Buy on Amazon

Where’s the cart? Now you can get everything on Safari. To purchase books, visit Amazon or your favorite retailer. Questions? See our FAQ or contact customer service:

1-800-889-8969 / 707-827-7019
support@oreilly.com

Download Example Code

View/Submit Errata

This hands-on guide demonstrates how the flexibility of the command line can help you become a more efficient and productive data scientist. You’ll learn how to combine small, yet powerful, command-line tools to quickly obtain, scrub, explore, and model your data.

To get you started—whether you’re on Windows, OS X, or Linux—author Jeroen Janssens introduces the Data Science Toolbox, an easy-to-install virtual environment packed with over 80 command-line tools.

Discover why the command line is an agile, scalable, and extensible technology. Even if you’re already comfortable processing data with, say, Python or R, you’ll greatly improve your data science workflow by also leveraging the power of the command line.

Obtain data from websites, APIs, databases, and spreadsheets
Perform scrub operations on plain text, CSV, HTML/XML, and JSON
Explore data, compute descriptive statistics, and create visualizations
Manage your data science workflow using Drake
Create reusable tools from one-liners and existing Python or R code
Parallelize and distribute data-intensive pipelines using GNU Parallel
Model data with dimensionality reduction, clustering, regression, and classification algorithms

Chapter 1 Introduction
1. Overview
2. Data Science Is OSEMN
3. Intermezzo Chapters
4. What Is the Command Line?
5. Why Data Science at the Command Line?
6. A Real-World Use Case
7. Further Reading
Chapter 2 Getting Started
1. Overview
2. Setting Up Your Data Science Toolbox
3. Essential Concepts and Tools
4. Further Reading
Chapter 3 Obtaining Data
1. Overview
2. Copying Local Files to the Data Science Toolbox
3. Decompressing Files
4. Converting Microsoft Excel Spreadsheets
5. Querying Relational Databases
6. Downloading from the Internet
7. Calling Web APIs
8. Further Reading
Chapter 4 Creating Reusable Command-Line Tools
1. Overview
2. Converting One-Liners into Shell Scripts
3. Creating Command-Line Tools with Python and R
4. Further Reading
Chapter 5 Scrubbing Data
1. Overview
2. Common Scrub Operations for Plain Text
3. Working with CSV
4. Working with HTML/XML and JSON
5. Common Scrub Operations for CSV
6. Further Reading
Chapter 6 Managing Your Data Workflow
1. Overview
2. Introducing Drake
3. Installing Drake
4. Obtain Top Ebooks from Project Gutenberg
5. Every Workflow Starts with a Single Step
6. Well, That Depends
7. Rebuilding Specific Targets
8. Discussion
9. Further Reading
Chapter 7 Exploring Data
1. Overview
2. Inspecting Data and Its Properties
3. Computing Descriptive Statistics
4. Creating Visualizations
5. Further Reading
Chapter 8 Parallel Pipelines
1. Overview
2. Serial Processing
3. Parallel Processing
4. Distributed Processing
5. Discussion
6. Further Reading
Chapter 9 Modeling Data
1. Overview
2. More Wine, Please!
3. Dimensionality Reduction with Tapkee
4. Clustering with Weka
5. Regression with SciKit-Learn Laboratory
6. Classification with BigML
7. Further Reading
Chapter 10 Conclusion
1. Let’s Recap
2. Three Pieces of Advice
3. Where to Go from Here?
4. Getting in Touch
Appendix List of Command-Line Tools
1. alias
2. awk
3. aws
4. bash
5. bc
6. bigmler
7. body
8. cat
9. cd
10. chmod
11. cols
12. cowsay
13. cp
14. csvcut
15. csvgrep
16. csvjoin
17. csvlook
18. csvsort
19. csvsql
20. csvstack
21. csvstat
22. curl
23. curlicue
24. cut
25. display
26. drake
27. dseq
28. echo
29. env
30. export
31. feedgnuplot
32. fieldsplit
33. find
34. for
35. git
36. grep
37. head
38. header
39. in2csv
40. jq
41. json2csv
42. less
43. ls
44. man
45. mkdir
46. mv
47. parallel
48. paste
49. pbc
50. pip
51. pwd
52. python
53. R
54. Rio
55. Rio-scatter
56. rm
57. run_experiment
58. sample
59. scp
60. scrape
61. sed
62. seq
63. shuf
64. sort
65. split
66. sql2csv
67. ssh
68. sudo
69. tail
70. tapkee
71. tar
72. tee
73. tr
74. tree
75. type
76. uniq
77. unpack
78. unrar
79. unzip
80. wc
81. weka
82. which
83. xml2json
Appendix Bibliography

Books & Videos

Data Science at the Command Line

Facing the Future with Time-Tested Tools

Read on Safari with a 10-day trial

Table of Contents

Chapter 1 Introduction

Overview

Data Science Is OSEMN

Intermezzo Chapters

What Is the Command Line?

Why Data Science at the Command Line?

A Real-World Use Case

Further Reading

Chapter 2 Getting Started

Overview

Setting Up Your Data Science Toolbox

Essential Concepts and Tools

Further Reading

Chapter 3 Obtaining Data

Overview

Copying Local Files to the Data Science Toolbox

Decompressing Files

Converting Microsoft Excel Spreadsheets

Querying Relational Databases

Downloading from the Internet

Calling Web APIs

Further Reading

Chapter 4 Creating Reusable Command-Line Tools

Overview

Converting One-Liners into Shell Scripts

Creating Command-Line Tools with Python and R

Further Reading

Chapter 5 Scrubbing Data

Overview

Common Scrub Operations for Plain Text

Working with CSV

Working with HTML/XML and JSON

Common Scrub Operations for CSV

Further Reading

Chapter 6 Managing Your Data Workflow

Overview

Introducing Drake

Installing Drake

Obtain Top Ebooks from Project Gutenberg

Every Workflow Starts with a Single Step

Well, That Depends

Rebuilding Specific Targets

Discussion

Further Reading

Chapter 7 Exploring Data

Overview

Inspecting Data and Its Properties

Computing Descriptive Statistics

Creating Visualizations

Further Reading

Chapter 8 Parallel Pipelines

Overview

Serial Processing

Parallel Processing

Distributed Processing

Discussion

Further Reading

Chapter 9 Modeling Data

Overview

More Wine, Please!

Dimensionality Reduction with Tapkee

Clustering with Weka

Regression with SciKit-Learn Laboratory

Classification with BigML

Further Reading

Chapter 10 Conclusion

Let’s Recap

Three Pieces of Advice

Where to Go from Here?

Getting in Touch

Appendix List of Command-Line Tools

alias

awk

aws

bash