Python for Secret Agents - Volume II

Table of contents

1. Python for Secret Agents Volume II
   1. Table of Contents
   2. Python for Secret Agents Volume II
   3. Credits
   4. About the Author
   5. About the Reviewer
   6. www.PacktPub.com
      1. Support files, eBooks, discount offers, and more
         1. Why subscribe?
         2. Free access for Packt account holders
   7. Preface
      1. What this book covers
      2. What you need for this book
      3. Who this book is for
      4. Conventions
      5. Reader feedback
      6. Customer support
         1. Downloading the example code
         2. Errata
         3. Piracy
         4. Questions
   8. 1. New Missions – New Tools
      1. Background briefing on tools
         1. Doing a Python upgrade
         2. Preliminary mission to upgrade pip
      2. Background briefing: review of the Python language
         1. Using variables to save results
         2. Using the sequence collections: strings
         3. Using other common sequences: tuples and lists
         4. Using the dictionary mapping
         5. Comparing data and using the logic operators
         6. Using some simple statements
         7. Using compound statements for conditions: if
         8. Using compound statements for repetition: for and while
         9. Defining functions
         10. Creating script files
      3. Mission One – upgrade Beautiful Soup
         1. Getting an HTML page
         2. Navigating the HTML structure
      4. Doing other upgrades
      5. Mission to expand our toolkit
         1. Scraping data from PDF files
         2. Sidebar on the ply package
         3. Building our own gadgets
            1. Getting the Arduino IDE
            2. Getting a Python serial interface
      6. Summary
   9. 2. Tracks, Trails, and Logs
      1. Background briefing – web servers and logs
         1. Understanding the variety of formats
         2. Getting a web server log
      2. Writing a regular expression for parsing
         1. Introducing some regular expression rules and patterns
         2. Finding a pattern in a file
         3. Using regular expression suffix operators
         4. Capturing characters by name
         5. Looking at the CLF
      3. Reading and understanding the raw data
         1. Reading a gzip compressed file
      4. Reading remote files
      5. Studying a log in more detail
      6. What are they downloading?
      7. Trails of activity
      8. Who is this person?
         1. Using Python to run other programs
         2. Processing whois queries
         3. Breaking a request into stanzas and lines
         4. Alternate stanza-finding algorithm
         5. Making bulk requests
      9. Getting logs from a server with ftplib
         1. Building a more complete solution
      10. Summary
   10. 3. Following the Social Network
       1. Background briefing – images and social media
          1. Accessing web services with urllib or http.client
       2. Who's doing the talking?
          1. Starting with someone we know
          2. Finding our followers
       3. What do they seem to be talking about?
       4. What are they posting?
       5. Deep Under Cover – NLTK and language analysis
       6. Summary
   11. 4. Dredging up History
       1. Background briefing–Portable Document Format
       2. Extracting PDF content
          1. Using generator expressions
          2. Writing generator functions
          3. Filtering bad data
          4. Writing a context manager
          5. Writing a PDF parser resource manager
          6. Extending the resource manager
       3. Getting text data from a document
          1. Displaying blocks of text
       4. Understanding tables and complex layouts
          1. Writing a content filter
          2. Filtering the page iterator
          3. Exposing the grid
          4. Making some text block recognition tweaks
          5. Emitting CSV output
       5. Summary
   12. 5. Data Collection Gadgets
       1. Background briefing: Arduino basics
          1. Organizing a shopping list
          2. Getting it right the first time
       2. Starting with the digital output pins
          1. Designing an external LED
          2. Assembling a working prototype
       3. Mastering the Arduino programming language
          1. Using the arithmetic and comparison operators
          2. Using common processing statements
          3. Hacking and the edit, download, test and break cycle
       4. Seeing a better blinking light
       5. Simple Arduino sensor data feed
       6. Collecting analog data
       7. Collecting bulk data with the Arduino
          1. Controlling data collection
       8. Data modeling and analysis with Python
          1. Collecting data from the serial port
          2. Formatting the collected data
          3. Crunching the numbers
          4. Creating a linear model
       9. Reducing noise with a simple filter
       10. Solving problems adding an audible alarm
       11. Summary
   13. Index