Books & Videos

Table of Contents

  1. Chapter 1 Setting the Pace: What Is Bad Data?

  2. Chapter 2 Is It Just Me, or Does This Data Smell Funny?

    1. Understand the Data Structure

    2. Field Validation

    3. Value Validation

    4. Physical Interpretation of Simple Statistics

    5. Visualization

    6. Keyword PPC Example

    7. Search Referral Example

    8. Recommendation Analysis

    9. Time Series Data

    10. Conclusion

  3. Chapter 3 Data Intended for Human Consumption, Not Machine Consumption

    1. The Data

    2. The Problem: Data Formatted for Human Consumption

    3. The Solution: Writing Code

    4. Postscript

    5. Other Formats

    6. Summary

  4. Chapter 4 Bad Data Lurking in Plain Text

    1. Which Plain Text Encoding?

    2. Guessing Text Encoding

    3. Normalizing Text

    4. Problem: Application-Specific Characters Leaking into Plain Text

    5. Text Processing with Python

    6. Exercises

  5. Chapter 5 (Re)Organizing the Web’s Data

    1. Can You Get That?

    2. General Workflow Example

    3. The Real Difficulties

    4. The Dark Side

    5. Conclusion

  6. Chapter 6 Detecting Liars and the Confused in Contradictory Online Reviews

    1. Weotta

    2. Getting Reviews

    3. Sentiment Classification

    4. Polarized Language

    5. Corpus Creation

    6. Training a Classifier

    7. Validating the Classifier

    8. Designing with Data

    9. Lessons Learned

    10. Summary

    11. Resources

  7. Chapter 7 Will the Bad Data Please Stand Up?

    1. Example 1: Defect Reduction in Manufacturing

    2. Example 2: Who’s Calling?

    3. Example 3: When “Typical” Does Not Mean “Average”

    4. Lessons Learned

    5. Will This Be on the Test?

  8. Chapter 8 Blood, Sweat, and Urine

    1. A Very Nerdy Body Swap Comedy

    2. How Chemists Make Up Numbers

    3. All Your Database Are Belong to Us

    4. Check, Please

    5. Live Fast, Die Young, and Leave a Good-Looking Corpse Code Repository

    6. Rehab for Chemists (and Other Spreadsheet Abusers)

    7. tl;dr

  9. Chapter 9 When Data and Reality Don’t Match

    1. Whose Ticker Is It Anyway?

    2. Splits, Dividends, and Rescaling

    3. Bad Reality

    4. Conclusion

  10. Chapter 10 Subtle Sources of Bias and Error

    1. Imputation Bias: General Issues

    2. Reporting Errors: General Issues

    3. Other Sources of Bias

    4. Conclusions

    5. References

  11. Chapter 11 Don’t Let the Perfect Be the Enemy of the Good: Is Bad Data Really Bad?

    1. But First, Let’s Reflect on Graduate School …

    2. Moving On to the Professional World

    3. Moving into Government Work

    4. Government Data Is Very Real

    5. Service Call Data as an Applied Example

    6. Moving Forward

    7. Lessons Learned and Looking Ahead

  12. Chapter 12 When Databases Attack: A Guide for When to Stick to Files

    1. History

    2. Consider Files as Your Datastore

    3. File Concepts

    4. A Web Framework Backed by Files

    5. Reflections

  13. Chapter 13 Crouching Table, Hidden Network

    1. A Relational Cost Allocations Model

    2. The Delicate Sound of a Combinatorial Explosion…

    3. The Hidden Network Emerges

    4. Storing the Graph

    5. Navigating the Graph with Gremlin

    6. Finding Value in Network Properties

    7. Think in Terms of Multiple Data Models and Use the Right Tool for the Job

    8. Acknowledgments

  14. Chapter 14 Myths of Cloud Computing

    1. Introduction to the Cloud

    2. What Is “The Cloud”?

    3. The Cloud and Big Data

    4. Introducing Fred

    5. At First Everything Is Great

    6. They Put 100% of Their Infrastructure in the Cloud

    7. As Things Grow, They Scale Easily at First

    8. Then Things Start Having Trouble

    9. They Need to Improve Performance

    10. Higher IO Becomes Critical

    11. A Major Regional Outage Causes Massive Downtime

    12. Higher IO Comes with a Cost

    13. Data Sizes Increase

    14. Geo Redundancy Becomes a Priority

    15. Horizontal Scale Isn’t as Easy as They Hoped

    16. Costs Increase Dramatically

    17. Fred’s Follies

    18. Myth 1: Cloud Is a Great Solution for All Infrastructure Components

    19. Myth 2: Cloud Will Save Us Money

    20. Myth 3: Cloud IO Performance Can Be Improved to Acceptable Levels Through Software RAID

    21. Myth 4: Cloud Computing Makes Horizontal Scaling Easy

    22. Conclusion and Recommendations

  15. Chapter 15 The Dark Side of Data Science

    1. Avoid These Pitfalls

    2. Know Nothing About Thy Data

    3. Thou Shalt Provide Your Data Scientists with a Single Tool for All Tasks

    4. Thou Shalt Analyze for Analysis’ Sake Only

    5. Thou Shalt Compartmentalize Learnings

    6. Thou Shalt Expect Omnipotence from Data Scientists

    7. Final Thoughts

  16. Chapter 16 How to Feed and Care for Your Machine-Learning Experts

    1. Define the Problem

    2. Fake It Before You Make It

    3. Create a Training Set

    4. Pick the Features

    5. Encode the Data

    6. Split Into Training, Test, and Solution Sets

    7. Describe the Problem

    8. Respond to Questions

    9. Integrate the Solutions

    10. Conclusion

  17. Chapter 17 Data Traceability

    1. Why?

    2. Personal Experience

    3. Immutability: Borrowing an Idea from Functional Programming

    4. An Example

    5. Conclusion

  18. Chapter 18 Social Media: Erasable Ink?

    1. Social Media: Whose Data Is This Anyway?

    2. Control

    3. Commercial Resyndication

    4. Expectations Around Communication and Expression

    5. Technical Implications of New End User Expectations

    6. What Does the Industry Do?

    7. What Should End Users Do?

    8. How Do We Work Together?

  19. Chapter 19 Data Quality Analysis Demystified: Knowing When Your Data Is Good Enough

    1. Framework Introduction: The Four Cs of Data Quality Analysis

    2. Complete

    3. Coherent

    4. Correct

    5. aCcountable

    6. Conclusion

  1. Colophon