Mastering Google App Engine

Mastering Google App Engine

by Mohsin Shafique Hijazee
Released October 2015
Publisher(s): Packt Publishing
ISBN: 9781784396671

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Build robust and highly scalable web applications with Google App Engine

About This Book

  • Get an in-depth look at how Google App Engine works under the hood
  • Design and model your application around Google's highly scalable distributed NoSQL datastore to unlock its full potential
  • A comprehensive guide to ensure your mastery of Google App Engine

In Detail

Developing web applications that serve millions of users is no easy task, as it involves a number of configurations and administrative tasks for the underlying software and hardware stack. This whole configuration requires not only expertise, but also a fair amount of time as well. Time that could have been spent on actual application functionality.

Google App Engine allows you develop highly scalable web applications or backends for mobile applications without worrying about the system administration plumbing or hardware provisioning issues. Just focus writing on your business logic, the meat of the application, and let Google's powerful infrastructure scale it to thousands of requests per second and millions of users without any effort on your part.

This book takes you from explaining how scalable applications work to designing and developing robust scalable web applications of your own, utilizing services available on Google App Engine.

Starting with a walkthrough of scalability is and how scalable web applications work, this book introduces you to the environment under which your applications exist on Google App Engine. Next, you will learn about Google's datastore, which is a massively scalable distributed NoSQL solution built on top of BigTable. You will examine the BigTable concepts and operations in detail and reveal how it is used to build Google datastore. Armed with this knowledge, you will then advance towards how to best model your data and query that along with transactions. To augment the powerful distributed dataset, you will deep dive into search functionality offered on Google App Engine. With the search and storage sorted out, you will get a look into performing long running tasks in the background using Google App Engine task queues along with sending and receiving emails. You will also examine the memcache to boost web application performance, image processing for common image manipulation tasks. You will then explore uploading, storing, and serving large files using Blobstore and Cloud storage.

Finally, you will be presented with the deployment and monitoring of your applications in production along with a detailed look at dividing applications into different working modules.

What You Will Learn

  • Scale and develop your applications with Google App Engine's runtime environment
  • Get to grips with request handling mechanism and write request handlers
  • Deep dive into Google's distributed NoSQL and highly scalable datastore and design your application around it
  • Implement powerful search with scalable datastore
  • Perform long-running tasks in the background using task queues
  • Write compartmentalized apps using multi tenancy, memcache, and other Google App Engine runtime services
  • Handle web requests using the CGI, WSGI, and multi-threaded configurations
  • Deploy, tweak, and manage apps in production on Google App Engine

Who This Book Is For

If you have been developing web applications in Python or any other dynamic language but have always wondered how to write highly scalable web applications without getting into system administration and other plumbing, then this is the book for you. No experience in writing scalable applications is required.

Style and approach

This book is an in-depth guide where you will examine the problems in the context of highly scalable web applications. This book will take you through the libraries, services, and required configuration and finally puts everything together into a small web application that showcases all the capabilities of Google App Engine.

Table of contents

  1. Mastering Google App Engine
    1. Table of Contents
    2. Mastering Google App Engine
    3. Credits
    4. About the Author
    5. Acknowledgments
    6. About the Reviewers
    7. www.PacktPub.com
      1. Support files, eBooks, discount offers, and more
        1. Why subscribe?
        2. Free access for Packt account holders
    8. Preface
      1. What this book covers
      2. What you need for this book
      3. Who this book is for
      4. Conventions
      5. Reader feedback
      6. Customer support
        1. Downloading the example code
        2. Errata
        3. Piracy
        4. Questions
    9. 1. Understanding the Runtime Environment
      1. The overall architecture
        1. The challenge of scale
        2. How to scale with the scale?
        3. Scaling in practice
        4. Infrastructure as a Service
        5. Platform as a Service
          1. Containers
        6. How does App Engine scales?
      2. Available runtimes
        1. Python
        2. The Java runtime environment
        3. Go
        4. PHP
      3. The structure of an application
      4. The available services
        1. Datastore
        2. Google Cloud SQL
        3. The Blobstore
        4. Memcache
        5. Scheduled Tasks
        6. Queues Tasks
        7. MapReduce
        8. Mail
        9. XMPP
        10. Channels
        11. Users
        12. OAuth
      5. Writing and deploying a simple application
        1. Installing an SDK on Linux
        2. Installing an SDK on Mac
        3. Installing an SDK on Windows
        4. Writing a simple app
        5. Deploying
      6. Summary
    10. 2. Handling Web Requests
      1. Request handling
        1. The CGI program
          1. Streams and environment variables
          2. CGI and Google App Engine
        2. WSGI
          1. Problems with CGI
          2. Solutions
          3. What WSGI looks like
        3. WSGI – Multithreading considerations
          1. WSGI in Google App Engine
      2. Request handling in App Engine
      3. Rendering templates
      4. Serving static resources
        1. Cache, headers, and mime types
          1. Serving files
      5. Using web frameworks
        1. Built-in frameworks
        2. Using external frameworks
          1. Using Bottle
      6. Summary
    11. 3. Understanding the Datastore
      1. The BigTable
        1. The data model
      2. How is data stored?
        1. The physical storage
          1. Some limitations
        2. Random writes and deletion
        3. Operations on BigTable
          1. Reading
          2. Writing
          3. Deleting
          4. Updating
          5. Scanning a range
          6. Selecting a key
          7. BigTable – a hands-on approach
        4. Scaling BigTable to BigData
        5. The datastore thyself
          1. Supporting queries
            1. Data as stored in BigTable
          2. The implementation details
      3. Summary
    12. 4. Modeling Your Data
      1. The data modeling language
        1. Keys and internal storage
          1. The application ID
          2. Namespaces
          3. The Kind
          4. The ID
            1. Pre-allocating numeric IDs
            2. The string ID
        2. The key
      2. Modeling your data
        1. The first approach – storing a reference as a property
        2. The second approach – a category within a key
        3. Properties
          1. The required option
          2. The default option
          3. The repeated option
          4. The choices options
          5. The indexed option
          6. The validator option
        4. The available properties
        5. Structured Properties
        6. The computed properties
      3. The model
        1. The constructor
        2. Class methods
          1. The allocate_ids() method
          2. The get_by_id() method
          3. The get_or_insert() method
          4. The query() method
        3. The instance methods
          1. The populate() method
          2. The put() method
          3. The to_dict() method
        4. Asynchronous versions
        5. Model hooks
      4. Summary
    13. 5. Queries, Indexes, and Transactions
      1. Querying your data
      2. Queries under the hood
        1. Single-property queries
          1. Examples of single-property queries
        2. Multiple property indexes
        3. Working with indexes
      3. The query API
        1. The Query object
          1. App
          2. Namespace
          3. Kind
          4. The ancestor
          5. The projection
          6. Filters
          7. The orders
          8. Further query options
            1. The keys_only function
            2. The projection
            3. The offset
            4. The limit
            5. The batch_size entities
            6. The prefetch_size entities
            7. The produce_cursors entities
            8. The start_cursor entities
            9. The end_cursor entities
        2. Filtering entities
          1. Filtering repeated properties
        3. Filtering structured properties
          1. The AND and OR operations
        4. Iterating over the results
          1. Conclusions
      4. Transactions
      5. Summary
    14. 6. Integrating Search
      1. Background
      2. The underlying principle
      3. Indexing your data
        1. Sample data
        2. Indexing thyself
        3. Documents
        4. Fields
          1. The text fields
        5. Placing the document in an index
        6. Getting a document
        7. Updating documents
        8. Deleting documents
        9. Indexing the documents
      4. Queries
        1. Simple queries
        2. Multiple value queries
        3. Logical operations
        4. Being specific with fields
          1. Operators on NumberField
          2. Operators on DateField
          3. Operations on AtomField
          4. Operations on TextField and HTMLField
          5. Operations on GeoField
        5. Putting it all together
        6. Selecting fields and calculated fields
      5. Sorting
      6. Pagination
        1. Offset-based pagination
        2. Cursor-based pagination
      7. Facets
        1. Indexing facets
        2. Fetching facets
          1. Asking facets via automatic discovery
          2. Asking specific facets
          3. Asking facets with specific values
          4. Asking facets in specific ranges
        3. Filtering by facets
      8. Summary
    15. 7. Using Task Queues
      1. The need to queue things
      2. The queue
        1. Defining queues
        2. Adding to a queue
        3. Processing tasks
        4. Putting it all together
        5. Using a deferred library
      3. Pull queues
      4. Summary
    16. 8. Reaching out, Sending E-mails
      1. About e-mails
      2. Sending e-mails
        1. The object-oriented API
        2. E-mail on the development console
        3. Headers
      3. Receiving e-mails
        1. Handling bounce notifications
      4. Putting it all together
      5. Summary
    17. 9. Working with the Google App Engine Services
      1. Memcache
        1. The Memcache operations
        2. Memcache in Google App Engine
        3. The Memcache client
          1. The object-oriented client
      2. Multi-tenancy
        1. Automatically setting the namespace
        2. The API-specific notes
          1. The Datastore
          2. Memcache
          3. Task queues
          4. Search
          5. Blobstore
      3. Blobs
        1. Uploads
          1. Getting BlobInfo
          2. More BlobInfo methods
        2. Serving
        3. Reading
      4. Users
        1. Storing users in datastore
      5. Images
      6. Putting it all together
      7. Summary
    18. 10. Application Deployment
      1. Deployment configurations
        1. Deployment revisited
        2. Versions
        3. The instance classes
          1. Instance addressability
      2. Scaling types
        1. Manual scaling
        2. Basic scaling
        3. Automatic scaling
        4. Modules
          1. Accessing the modules
      3. The dispatch.yaml file
      4. Scheduled tasks
        1. The Scheduled tasks format
        2. Protecting cron handling URLs
        3. Logs
        4. The Remote API
      5. AppStats
      6. Summary
    19. Index

Product information

  • Title: Mastering Google App Engine
  • Author(s): Mohsin Shafique Hijazee
  • Release date: October 2015
  • Publisher(s): Packt Publishing
  • ISBN: 9781784396671