Expert Python Programming - Third Edition

Book description

Refine your Python programming skills and build professional-grade applications with this comprehensive guide

Key Features

  • Create manageable code that can run in various environments with different sets of dependencies
  • Implement effective Python data structures and algorithms to write optimized code
  • Discover the exciting new features of Python 3.7

Book Description

Python is a dynamic programming language that's used in a wide range of domains thanks to its simple yet powerful nature. Although writing Python code is easy, making it readable, reusable, and easy to maintain is challenging. Complete with best practices, useful tools, and standards implemented by professional Python developers, the third edition of Expert Python Programming will help you overcome this challenge.

The book will start by taking you through the new features in Python 3.7. You'll then learn the advanced components of Python syntax, in addition to understanding how to apply concepts of various programming paradigms, including object-oriented programming, functional programming, and event-driven programming. This book will also guide you through learning the naming best practices, writing your own distributable Python packages, and getting up to speed with automated ways to deploy your software on remote servers. You'll discover how to create useful Python extensions with C, C++, Cython, and CFFI. Furthermore, studying about code management tools, writing clear documentation, and exploring test-driven development will help you write clean code.

By the end of the book, you will have become an expert in writing efficient and maintainable Python code.

What you will learn

  • Explore modern ways of setting up repeatable and consistent development environments
  • Package Python code effectively for community and production use
  • Learn modern syntax elements of Python programming such as f-strings, enums, and lambda functions
  • Demystify metaprogramming in Python with metaclasses
  • Write concurrent code in Python
  • Extend and integrate Python with code written in different languages

Who this book is for

This book will appeal to you if you're a programmer looking to take your Python knowledge to the next level by writing efficient code and learning the latest features of version 3.7 and above.

Table of contents

  1. Title Page
  2. Copyright and Credits
    1. Expert Python Programming Third Edition
  3. Dedication
  4. About Packt
    1. Why subscribe?
    2. Packt.com
  5. Contributors
    1. About the authors
    2. About the reviewer
    3. Packt is searching for authors like you
  6. Preface
    1. Who this book is for
    2. What this book covers
    3. To get the most out of this book
      1. Download the example code files
      2. Download the color images
      3. Conventions used
    4. Get in touch
      1. Reviews
  7. Section 1: Before You Start
  8. Current Status of Python
    1. Technical requirements
    2. Where are we now and where we are going to?
    3. Why and how Python changes
    4. Being up-to-date with changes ­by following PEP documents
    5. Python 3 adoption at the time of writing this book
    6. The main differences between Python 3 and Python 2
      1. Why should I care?
      2. The main syntax differences and common pitfalls
        1. Syntax changes
        2. Changes in the standard library
        3. Changes in data types and collections and string literals
      3. The popular tools and techniques used for maintaining cross-version compatibility
    7. Not only CPython
      1. Why should I care?
      2. Stackless Python
      3. Jython
      4. IronPython
      5. PyPy
      6. MicroPython
    8. Useful resources
    9. Summary
  9. Modern Python Development Environments
    1. Technical requirements
    2. Installing additional Python packages using pip
    3. Isolating the runtime environment
      1. Application-level isolation versus system-level isolation
    4. Python's venv
      1. venv versus virtualenv
    5. System-level environment isolation
      1. Virtual development environments using Vagrant
      2. Virtual environments using Docker
        1. Containerization versus virtualization
        2. Writing your first Dockerfile
        3. Running containers
        4. Setting up complex environments
        5. Useful Docker recipes for Python
          1. Reducing the size of containers
          2. Addressing services inside of a Compose environment
          3. Communicating between multiple Compose environments
    6. Popular productivity tools
      1. Custom Python shells – ipython, bpython, ptpython, and so on
        1. Setting up the PYTHONSTARTUP environment variable
        2. IPython
        3. bpython
        4. ptpython
      2. Incorporating shells in your own scripts and programs
      3. Interactive debuggers
    7. Summary
  10. Section 2: Python Craftsmanship
  11. Modern Syntax Elements - Below the Class Level
    1. Technical requirements
    2. Python's built-in types
      1. Strings and bytes
        1. Implementation details
        2. String concatenation
          1. Constant folding, the peephole optimizer, and the AST optimizer
        3. String formatting with f-strings
      2. Containers
        1. Lists and tuples
          1. Implementation details
          2. List comprehensions
          3. Other idioms
        2. Dictionaries
          1. Implementation details
          2. Weaknesses and alternatives
        3. Sets
          1. Implementation details
    3. Supplemental data types and containers
      1. Specialized data containers from the collections module
      2. Symbolic enumeration with the enum module
    4. Advanced syntax
      1. Iterators
      2. Generators and yield statements
      3. Decorators
        1. General syntax and possible implementations
          1. As a function
          2. As a class
          3. Parametrizing decorators
          4. Introspection preserving decorators
        2. Usage and useful examples
          1. Argument checking
          2. Caching
          3. Proxy
          4. Context provider
      4. Context managers – the with statement
        1. The general syntax and possible implementations
          1. As a class
          2. As a function – the contextlib module
    5. Functional-style features of Python
      1. What is functional programming?
      2. Lambda functions
      3. map(), filter(), and reduce()
      4. Partial objects and partial() functions
      5. Generator expressions
    6. Function and variable annotations
      1. The general syntax
      2. The possible uses
      3. Static type checking with mypy
    7. Other syntax elements you may not know of yet
      1. The for ... else ... statement
      2. Keyword-only arguments
    8. Summary
  12. Modern Syntax Elements - Above the Class Level
    1. Technical requirements
    2. The protocols of the Python language – dunder methods and attributes
    3. Reducing boilerplate with data classes
    4. Subclassing built-in types
    5. MRO and accessing methods from superclasses
      1. Old-style classes and super in Python 2
      2. Understanding Python's Method Resolution Order
      3. Super pitfalls
        1. Mixing super and explicit class calls
        2. Heterogeneous arguments
      4. Best practices
    6. Advanced attribute access patterns
      1. Descriptors
        1. Real-life example – lazily evaluated attributes
      2. Properties
      3. Slots
    7. Summary
  13. Elements of Metaprogramming
    1. Technical requirements
    2. What is metaprogramming?
      1. Decorators – a method of metaprogramming
      2. Class decorators
      3. Using __new__() for overriding the instance creation process
      4. Metaclasses
        1. The general syntax
        2. New Python 3 syntax for metaclasses
        3. Metaclass usage
        4. Metaclass pitfalls
      5. Code generation
        1. exec, eval, and compile
        2. Abstract syntax tree (AST)
          1. Import hooks
        3. Projects that use code generation patterns
          1. Falcon's compiled router
          2. Hy
    3. Summary
  14. Choosing Good Names
    1. Technical requirements
    2. PEP 8 and naming best practices
      1. Why and when to follow PEP 8?
      2. Beyond PEP 8 – Team-specific style guidelines
    3. Naming styles
      1. Variables
        1. Constants
        2. Naming and usage
        3. Public and private variables
        4. Functions and methods
        5. The private controversy
        6. Special methods
        7. Arguments
        8. Properties
        9. Classes
        10. Modules and packages
    4. The naming guide
      1. Using the has/is prefixes for Boolean elements
      2. Using plurals for variables that are collections
      3. Using explicit names for dictionaries
      4. Avoid generic names and redundancy
      5. Avoiding existing names
    5. Best practices for arguments
      1. Building arguments by iterative design
      2. Trusting the arguments and your tests
      3. Using *args and **kwargs magic arguments carefully
    6. Class names
    7. Module and package names
    8. Useful tools
      1. Pylint
      2. pycodestyle and flake8
    9. Summary
  15. Writing a Package
    1. Technical requirements
    2. Creating a package
      1. The confusing state of Python packaging tools
        1. The current landscape of Python packaging thanks to PyPA
        2. Tool recommendations
      2. Project configuration
        1. setup.py
        2. setup.cfg
        3. MANIFEST.in
        4. Most important metadata
        5. Trove classifiers
        6. Common patterns
          1. Automated inclusion of version string from package
          2. README file
          3. Managing dependencies
      3. The custom setup command
      4. Working with packages during development
        1. setup.py install
        2. Uninstalling packages
        3. setup.py develop or pip -e
    3. Namespace packages
      1. Why is it useful?
        1. PEP 420 - implicit namespace packages
        2. Namespace packages in previous Python versions
    4. Uploading a package
      1. PyPI - Python Package Index
        1. Uploading to PyPI - or other package index
        2. .pypirc
      2. Source packages versus built packages
        1. sdist
        2. bdist and wheels
    5. Standalone executables
      1. When standalone executables useful?
      2. Popular tools
        1. PyInstaller
        2. cx_Freeze
        3. py2exe and py2app
      3. Security of Python code in executable packages
        1. Making decompilation harder
    6. Summary
  16. Deploying the Code
    1. Technical requirements
    2. The Twelve-Factor App
    3. Various approaches to deployment automation
      1. Using Fabric for deployment automation
    4. Your own package index or index mirror
      1. PyPI mirroring
      2. Bundling additional resources with your Python package
    5. Common conventions and practices
      1. The filesystem hierarchy
      2. Isolation
      3. Using process supervision tools
      4. Application code running in user space
      5. Using reverse HTTP proxies
      6. Reloading processes gracefully
    6. Code instrumentation and monitoring
      1. Logging errors – Sentry/Raven
      2. Monitoring system and application metrics
      3. Dealing with application logs
        1. Basic low-level log practices
        2. Tools for log processing
    7. Summary
  17. Python Extensions in Other Languages
    1. Technical requirements
    2. Differentiating between the C and C++ languages
      1. Loading extensions in C or C++
    3. The need to use extensions
      1. Improving the performance in critical code sections
      2. Integrating existing code written in different languages
      3. Integrating third-party dynamic libraries
      4. Creating custom datatypes
    4. Writing extensions
      1. Pure C extensions
        1. A closer look at Python/C API
        2. Calling and binding conventions
        3. Exception handling
        4. Releasing GIL
        5. Reference counting
      2. Writing extensions with Cython
        1. Cython as a source-to-source compiler
        2. Cython as a language
    5. Challenges with using extensions
      1. Additional complexity
      2. Debugging
    6. Interfacing with dynamic libraries without extensions
      1. The ctypes module
        1. Loading libraries
        2. Calling C functions using ctypes
        3. Passing Python functions as C callbacks
      2. CFFI
    7. Summary
  18. Section 3: Quality over Quantity
  19. Managing Code
    1. Technical requirements
    2. Working with a version control system
      1. Centralized systems
      2. Distributed systems
        1. Distributed strategies
      3. Centralized or distributed?
      4. Use Git if you can
      5. GitFlow and GitHub Flow
    3. Setting up continuous development processes
      1. Continuous integration
        1. Testing every commit
        2. Merge testing through CI
        3. Matrix testing
      2. Continuous delivery
      3. Continuous deployment
      4. Popular tools for continuous integration
        1. Jenkins
        2. Buildbot
        3. Travis CI
        4. GitLab CI
      5. Choosing the right tool and common pitfalls
        1. Problem 1 – Complex build strategies
        2. Problem 2 – Long building time
        3. Problem 3 – External job definitions
        4. Problem 4 – Lack of isolation
    4. Summary
  20. Documenting Your Project
    1. Technical requirements
    2. The seven rules of technical writing
      1. Write in two steps
      2. Target the readership
      3. Use a simple style
      4. Limit the scope of information
      5. Use realistic code examples
      6. Use a light but sufficient approach
      7. Use templates
    3. Documentation as code
      1. Using Python docstrings
      2. Popular markup languages and styles for documentation
    4. Popular documentation generators for Python libraries
      1. Sphinx
        1. Working with the index pages
        2. Registering module helpers
        3. Adding index markers
        4. Cross-references
      2. MkDocs
      3. Documentation building and continuous integration
    5. Documenting web APIs
      1. Documentation as API prototype with API Blueprint
      2. Self-documenting APIs with Swagger/OpenAPI
    6. Building a well-organized documentation system
      1. Building documentation portfolio
        1. Design
        2. Usage
          1. Recipe
          2. Tutorial
          3. Module helper
        3. Operations
      2. Your very own documentation portfolio
      3. Building a documentation landscape
        1. Producer's layout
        2. Consumer's layout
    7. Summary
  21. Test-Driven Development
    1. Technical requirements
    2. I don't test
      1. Three simple steps of test-driven development
        1. Preventing software regression
        2. Improving code quality
        3. Providing the best developer documentation
        4. Producing robust code faster
      2. What kind of tests?
        1. Unit tests
        2. Acceptance tests
        3. Functional tests
        4. Integration tests
        5. Load and performance testing
        6. Code quality testing
      3. Python standard test tools
        1. unittest
        2. doctest
    3. I do test
      1. unittest pitfalls
      2. unittest alternatives
        1. nose
          1. Test runner
          2. Writing tests
          3. Writing test fixtures
          4. Integration with setuptools and plugin system
          5. Wrap-up
        2. py.test
          1. Writing test fixtures
          2. Disabling test functions and classes
          3. Automated distributed tests
          4. Wrap-up
      3. Testing coverage
      4. Fakes and mocks
        1. Building a fake
        2. Using mocks
      5. Testing environment and dependency compatibility
        1. Dependency matrix testing
      6. Document-driven development
        1. Writing a story
    4. Summary
  22. Section 4: Need for Speed
  23. Optimization - Principles and Profiling Techniques
    1. Technical requirements
    2. The three rules of optimization
      1. Making it work first
      2. Working from the user's point of view
      3. Keeping the code readable and maintainable
    3. Optimization strategy
      1. Looking for another culprit
      2. Scaling the hardware
      3. Writing a speed test
    4. Finding bottlenecks
      1. Profiling CPU usage
        1. Macro-profiling
        2. Micro-profiling
      2. Profiling memory usage
        1. How Python deals with memory
        2. Profiling memory
          1. objgraph
        3. C code memory leaks
      3. Profiling network usage
        1. Tracing network transactions
    5. Summary
  24. Optimization - Some Powerful Techniques
    1. Technical requirements
    2. Defining complexity
      1. Cyclomatic complexity
      2. The big O notation
    3. Reducing complexity by choosing proper data structures
      1. Searching in a list
        1. Using sets
    4. Using collections
      1. deque
      2. defaultdict
      3. namedtuple
    5. Using architectural trade-offs
      1. Using heuristics and approximation algorithms
      2. Using task queues and delayed processing
      3. Using probabilistic data structures
    6. Caching
      1. Deterministic caching
      2. Non-deterministic caching
      3. Cache services
        1. Memcached
    7. Summary
  25. Concurrency
    1. Technical requirements
    2. Why concurrency?
    3. Multithreading
      1. What is multithreading?
      2. How Python deals with threads
      3. When should we use threading?
        1. Building responsive interfaces
        2. Delegating work
        3. Multiuser applications
        4. An example of a threaded application
          1. Using one thread per item
          2. Using a thread pool
          3. Using two-way queues
          4. Dealing with errors and rate limiting
    4. Multiprocessing
      1. The built-in multiprocessing module
        1. Using process pools
        2. Using multiprocessing.dummy as the multithreading interface
    5. Asynchronous programming
      1. Cooperative multitasking and asynchronous I/O
      2. Python async and await keywords
      3. asyncio in older versions of Python
      4. A practical example of asynchronous programming
      5. Integrating non-asynchronous code with async using futures
        1. Executors and futures
        2. Using executors in an event loop
    6. Summary
  26. Section 5: Technical Architecture
  27. Event-Driven and Signal Programming
    1. Technical requirements
    2. What exactly is event-driven programming?
      1. Event-driven != asynchronous
      2. Event-driven programming in GUIs
      3. Event-driven communication
    3. Various styles of event-driven programming
      1. Callback-based style
      2. Subject-based style
      3. Topic-based style
    4. Event-driven architectures
      1. Event and message queues
    5. Summary
  28. Useful Design Patterns
    1. Technical requirements
    2. Creational patterns
      1. Singleton
    3. Structural patterns
      1. Adapter
        1. Interfaces
          1. Using zope.interface
          2. Using function annotations and abstract base classes
          3. Using collections.abc
      2. Proxy
      3. Facade
    4. Behavioral patterns
      1. Observer
      2. Visitor
      3. Template
    5. Summary
  29. reStructuredText Primer
    1. reStructuredText
      1. Section structure
      2. Lists
      3. Inline markup
      4. Literal block
      5. Links
  30. Other Books You May Enjoy
    1. Leave a review - let other readers know what you think

Product information

  • Title: Expert Python Programming - Third Edition
  • Author(s): Michal Jaworski, Tarek Ziade
  • Release date: April 2019
  • Publisher(s): Packt Publishing
  • ISBN: 9781789808896