Mastering PostgreSQL 9.6

Mastering PostgreSQL 9.6

by Hans-Jurgen Schonig
Released May 2017
Publisher(s): Packt Publishing
ISBN: 9781783555352

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Master the capabilities of PostgreSQL 9.6 to efficiently manage and maintain your database

About This Book

  • Your one-stop guide to mastering the advanced concepts in PostgreSQL with ease

  • Master query optimization, replication, and high availability with PostgreSQL

  • Extend the functionalities of PostgreSQL to suit your organizational needs with minimum effort

    • Who This Book Is For

    If you are a PostgreSQL data architect or an administrator who wants to understand how to implement advanced functionalities and master complex administrative tasks with PostgreSQL, then this book is perfect for you. Prior experience of administrating a PostgreSQL database and a working knowledge of SQL is required to make the best use of this book.

    What You Will Learn

  • Get to grips with the advanced features of PostgreSQL 9.6 and handle advanced SQL

  • Make use of the indexing features in PostgreSQL and fine-tune the performance of your queries

  • Work with the stored procedures and manage backup and recovery

  • Master the replication and failover techniques

  • Troubleshoot your PostgreSQL instance for solutions to the common and not-so-common problems

  • Learn how to migrate your database from MySQL and Oracle to PostgreSQL without any hassle

    • In Detail

    PostgreSQL is an open source database used for handling large datasets (Big Data) and as a JSON document database. It also has applications in the software and web domains. This book will enable you to build better PostgreSQL applications and administer databases more efficiently.

    We begin by explaining the advanced database design concepts in PostgreSQL 9.6, along with indexing and query optimization. You will also see how to work with event triggers and perform concurrent transactions and table partitioning, along with exploring SQL and server tuning. We will walk you through implementing advanced administrative tasks such as server maintenance and monitoring, replication, recovery and high availability, and much more. You will understand the common and not-so-common troubleshooting problems and how you can overcome them.

    By the end of this book, you will have an expert-level command of the advanced database functionalities and will be able to implement advanced administrative tasks with PostgreSQL.

    Style and Approach

    This book is a comprehensive guide covering all the concepts you need to master PostgreSQL. Packed with hands-on examples, tips and tricks, even the most advanced concepts are explained in a very easy-to-follow manner. Every chapter in the book does not only focus on how each task is performed, but also why.

    Table of contents

    1. Preface
      1. What this book covers
      2. What you need for this book
      3. Who this book is for
      4. Conventions
      5. Reader feedback
      6. Customer support
        1. Errata
        2. Piracy
        3. Questions
    2. PostgreSQL Overview
      1. What is new in PostgreSQL 9.6?
        1. Understanding new database administration functions
          1. Killing idle sessions
          2. Finding more detailed information in pg_stat_activity
          3. Tracking vaccum progress
          4. Improving vacuum speed
        2. Digging into new SQL and developer-related functions
        3. Using new backup and replication functionality
          1. Streamlining wal_level and monitoring
          2. Using multiple synchronous standby servers
        4. Understanding performance-related features
          1. Improving relation extensions
          2. Checkpoint sorting and kernel interaction
          3. Using more advanced foreign data wrappers
          4. Introducing parallel queries
          5. Adding snapshot too old
      2. Summary
    3. Understanding Transactions and Locking
      1. Working with PostgreSQL transactions
        1. Handling errors inside a transaction
        2. Making use of savepoints
        3. Transactional DDLs
      2. Understanding basic locking
        1. Avoiding typical mistakes and explicit locking
          1. Considering alternative solutions
      3. Making use of FOR SHARE and FOR UPDATE
      4. Understanding transaction isolation levels
        1. Considering SSI transactions
      5. Observing deadlocks and similar issues
      6. Utilizing advisory locks
      7. Optimizing storage and managing cleanup
        1. Configuring VACUUM and autovacuum
          1. Digging into transaction wraparound-related issues
          2. A word on VACUUM FULL
        2. Watching VACUUM at work
        3. Making use of snapshot too old
      8. Summary
    4. Making Use of Indexes
      1. Understanding simple queries and the cost model
        1. Making use of EXPLAIN
        2. Digging into the PostgreSQL cost model
        3. Deploying simple indexes
        4. Making use of sorted output
        5. Using more than one index at a time
          1. Using bitmap scans effectively
        6. Using indexes in an intelligent way
      2. Improving speed using clustered tables
        1. Clustering tables
        2. Making use of index only scans
      3. Understanding additional B-tree features
        1. Combined indexes
        2. Adding functional indexes
        3. Reducing space consumption
        4. Adding data while indexing
      4. Introducing operator classes
        1. Hacking up an operator class for a B-tree
          1. Creating new operators
          2. Creating operator classes
          3. Testing custom operator classes
      5. Understanding PostgreSQL index types
        1. Hash indexes
        2. GiST indexes
          1. Understanding how GiST works
          2. Extending GiST
        3. GIN indexes
          1. Extending GIN
        4. SP-GiST indexes
        5. BRIN indexes
          1. Extending BRIN indexes
        6. Adding additional indexes
      6. Achieving better answers with fuzzy searching
        1. Taking advantage of pg_trgm
        2. Speed up LIKE queries
        3. Handling regular expressions
      7. Understanding full-text search - FTS
        1. Comparing strings
        2. Defining GIN indexes
        3. Debugging your search
        4. Gathering word statistics
        5. Taking advantage of exclusion operators
      8. Summary
    5. Handling Advanced SQL
      1. Introducing grouping sets
        1. Loading some sample data
        2. Applying grouping sets
          1. Investigating performance
        3. Combining grouping sets with the FILTER clause
      2. Making use of ordered sets
      3. Understanding hypothetical aggregates
      4. Utilizing windowing functions and analytics
        1. Partitioning data
        2. Ordering data inside a window
        3. Using sliding windows
        4. Abstracting window clauses
        5. Making use of onboard windowing functions
          1. rank and dense_rank functions
          2. ntile() function
          3. lead() and lag() functions
          4. first_value(), nth_value(), and last_value() functions
          5. row_number() function
      5. Writing your own aggregates
        1. Creating simple aggregates
        2. Adding support for parallel queries
        3. Improving efficiency
        4. Writing hypothetical aggregates
      6. Summary
    6. Log Files and System Statistics
      1. Gathering runtime statistics
        1. Working with PostgreSQL system views
          1. Checking live traffic
          2. Inspecting databases
            1. Inspecting tables
          3. Making sense of pg_stat_user_tables
          4. Digging into indexes
          5. Tracking the background worker
          6. Tracking, archiving, and streaming
          7. Checking SSL connections
          8. Inspecting transactions in real time
          9. Tracking vacuum progress
          10. Using pg_stat_statements
      2. Creating log files
        1. Configuring postgresql.conf file
          1. Defining log destination and rotation
          2. Configuring syslog
          3. Logging slow queries
          4. Defining what and how to log
      3. Summary
    7. Optimizing Queries for Good Performance
      1. Learning what the optimizer does
        1. Optimizations by example
          1. Evaluating join options
            1. Nested loops
            2. Hash joins
            3. Merge joins
          2. Applying transformations
            1. Inlining the view
            2. Flattening subselects
          3. Applying equality constraints
          4. Exhaustive searching
          5. Trying it all out
          6. Making the process fail
          7. Constant folding
          8. Understanding function inlining
          9. Join pruning
          10. Speedup set operations
      2. Understanding execution plans
        1. Approaching plans systematically
          1. Making EXPLAIN more verbose
        2. Spotting problems
          1. Spotting changes in runtime
          2. Inspecting estimates
          3. Inspecting buffer usage
          4. Fixing high buffer usage
      3. Understanding and fixing joins
        1. Getting joins right
        2. Processing outer joins
        3. Understanding the join_collapse_limit variable
      4. Enabling and disabling optimizer settings
        1. Understanding genetic query optimization
      5. Partitioning data
        1. Creating partitions
        2. Applying table constraints
        3. Modifying inherited structures
        4. Moving tables in and out of partitioned structures
        5. Cleaning up data
      6. Adjusting parameters for good query performance
        1. Speeding up sorting
        2. Speedup administrative tasks
      7. Summary
    8. Writing Stored Procedures
      1. Understanding stored procedure languages
        1. The anatomy of a stored procedure
          1. Introducing dollar quoting
          2. Making use of anonymous code blocks
          3. Using functions and transactions
      2. Understanding various stored procedure languages
        1. Introducing PL/pgSQL
          1. Handling quoting
          2. Managing scopes
          3. Understanding advanced error handling
          4. Making use of GET DIAGNOSTICS
          5. Using cursors to fetch data in chunks
          6. Utilizing composite types
          7. Writing triggers in PL/pgSQL
        2. Introducing PL/Perl
          1. Using PL/Perl for datatype abstraction
          2. Deciding between PL/Perl and PL/PerlU
          3. Making use of the SPI interface
          4. Using SPI for set returning functions
          5. Escaping in PL/Perl and support functions
          6. Sharing data across function calls
          7. Writing triggers in Perl
        3. Introducing PL/Python
          1. Writing simple PL/Python code
          2. Using the SPI interface
          3. Handling errors
      3. Improving stored procedure performance
        1. Reducing the number of function calls
          1. Using cached plans
          2. Assigning costs to functions
      4. Using stored procedures
      5. Summary
    9. Managing PostgreSQL Security
      1. Managing network security
        1. Understanding bind addresses and connections
          1. Inspecting connections and performance
          2. Living in a world without TCP
        2. Managing pg_hba.conf
          1. Handling SSL
        3. Handling instance-level security
          1. Creating and modifying users
        4. Defining database-level security
        5. Adjusting schema-level permissions
        6. Working with tables
        7. Handling column-level security
        8. Configuring default privileges
      2. Digging into row-level security - RLS
      3. Inspecting permissions
      4. Reassigning objects and dropping users
      5. Summary
    10. Handling Backup and Recovery
      1. Performing simple dumps
        1. Running pg_dump
        2. Passing passwords and connection information
          1. Using environment variables
          2. Making use of .pgpass
          3. Using service files
        3. Extracting subsets of data
        4. Handling various data formats
      2. Replaying backups
      3. Handling global data
      4. Summary
    11. Making Sense of Backups and Replication
      1. Understanding the transaction log
        1. Looking at the transaction log
        2. Understanding checkpoints
        3. Optimizing the transaction log
      2. Transaction log archiving and recovery
        1. Configuring for archiving
        2. Confguring the pg_hba.conf file
        3. Creating base backups
          1. Reducing the bandwidth of a backup
          2. Mapping tablespaces
          3. Using different formats
          4. Testing transaction log archiving
        4. Replaying the transaction log
          1. Finding the right timestamp
        5. Cleaning up the transaction log archive
      3. Setting up asynchronous replication
        1. Performing a basic setup
          1. Improving security
        2. Halting and resuming replication
        3. Checking replication to ensure availability
        4. Performing failovers and understanding timelines
        5. Managing conflicts
        6. Making replication more reliable
      4. Upgrading to synchronous replication
        1. Adjusting durability
      5. Making use of replication slots
        1. Handling physical replication slots
        2. Handling logical replication slots
          1. Use cases of logical slots
      6. Summary
    12. Deciding on Useful Extensions
      1. Understanding how extensions work
        1. Checking for available extensions
      2. Making use of contrib modules
        1. Using the adminpack
        2. Applying bloom filters
        3. Deploying btree_gist and btree_gin
        4. Dblink - consider phasing out
        5. Fetching files with file_fdw
        6. Inspecting storage using pageinspect
        7. Investigating caching with pg_buffercache
        8. Encrypting data with pgcrypto
        9. Prewarming caches with pg_prewarm
        10. Inspecting performance with pg_stat_statements
        11. Inspecting storage with pgstattuple
        12. Fuzzy searches with pg_trgm
        13. Connecting to remote servers using postgres_fdw
          1. Handling mistakes and typos
      3. Other useful extensions
      4. Summary
    13. Troubleshooting PostgreSQL
      1. Approaching an unknown database
      2. Inspecting pg_stat_activity
        1. Querying pg_stat_activity
          1. Treating Hibernate statements
          2. Figuring out where queries come from
      3. Checking for slow queries
        1. Inspecting individual queries
        2. Digging deeper with perf
      4. Inspecting the log
      5. Checking for missing indexes
      6. Checking for memory and I/O
      7. Understanding noteworthy error scenarios
        1. Facing clog corruption
        2. Understanding checkpoint messages
        3. Managing corrupted data pages
        4. Careless connection management
        5. Fighting table bloat
      8. Summary
    14. Migrating to PostgreSQL
      1. Migrating SQL statements to PostgreSQL
        1. Using lateral joins
          1. Supporting lateral
        2. Using grouping sets
          1. Supporting grouping sets
        3. Using WITH clause - common table expressions
          1. Supporting WITH clause
        4. Using WITH RECURSIVE clause
          1. Supporting WITH RECURSIVE clause
        5. Using FILTER clause
          1. Supporting FILTER clause
        6. Using windowing functions
          1. Supporting windowing and analytics
        7. Using ordered sets - WITHIN GROUP clause
          1. Supporting WITHIN GROUP clause
        8. Using TABLESAMPLE clause
          1. Supporting TABLESAMPLE clause
        9. Using limit/offset
          1. Supporting FETCH FIRST clause
        10. Using OFFSET
          1. Supporting OFFSET clause
        11. Using temporal tables
          1. Supporting temporal tables
        12. Matching patterns in time series
      2. Moving from Oracle to PostgreSQL
        1. Using the oracle_fdw extension to move data
        2. Using ora2pg to migrate from Oracle
        3. Common pitfalls
      3. Moving from MySQL or MariaDB to PostgreSQL
        1. Handling data in MySQL and MariaDB
          1. Changing column definitions
          2. Handling null values
          3. Expecting problems
        2. Migrating data and schema
          1. Using pg_chameleon
          2. Using foreign data wrappers
      4. Summary

    Product information

    • Title: Mastering PostgreSQL 9.6
    • Author(s): Hans-Jurgen Schonig
    • Release date: May 2017
    • Publisher(s): Packt Publishing
    • ISBN: 9781783555352