Go: Building Web Applications

Table of contents

1. Go: Building Web Applications
   1. Table of Contents
   2. Go: Building Web Applications
   3. Go: Building Web Applications
   4. Credits
   5. Preface
      1. What this learning path covers
      2. What you need for this learning path
      3. Who this learning path is for
      4. Reader feedback
      5. Customer support
         1. Downloading the example code
         2. Errata
         3. Piracy
         4. Questions
   6. 1. Module 1
      1. 1. Introducing and Setting Up Go
         1. Installing Go
         2. Structuring a project
            1. Code conventions
         3. Importing packages
            1. Handling private repositories
            2. Dealing with versioning
         4. Introducing the net package
         5. Hello, Web
         6. Summary
      2. 2. Serving and Routing
         1. Serving files directly
         2. Basic routing
         3. Using more complex routing with Gorilla
         4. Redirecting requests
         5. Serving basic errors
         6. Summary
      3. 3. Connecting to Data
         1. Connecting to a database
            1. Creating a MySQL database
         2. Using GUID for prettier URLs
         3. Handling 404s
         4. Summary
      4. 4. Using Templates
         1. Introducing templates, context, and visibility
         2. HTML templates and text templates
         3. Displaying variables and security
         4. Using logic and control structures
         5. Summary
      5. 5. Frontend Integration with RESTful APIs
         1. Setting up the basic API endpoint
         2. RESTful architecture and best practices
         3. Creating our first API endpoint
         4. Implementing security
         5. Creating data with POST
         6. Modifying data with PUT
         7. Summary
      6. 6. Sessions and Cookies
         1. Setting cookies
         2. Capturing user information
            1. Creating users
            2. Enabling sessions
            3. Letting users register
            4. Letting users log in
         3. Initiating a server-side session
            1. Creating a store
            2. Utilizing flash messages
         4. Summary
      7. 7. Microservices and Communication
         1. Introducing the microservice approach
         2. Pros and cons of utilizing microservices
         3. Understanding the heart of microservices
         4. Communicating between microservices
         5. Putting a message on the wire
         6. Reading from another service
         7. Summary
      8. 8. Logging and Testing
         1. Introducing logging in Go
         2. Logging to IO
            1. Multiple loggers
         3. Formatting your output
         4. Using panics and fatal errors
         5. Introducing testing in Go
         6. Summary
      9. 9. Security
         1. HTTPS everywhere – implementing TLS
         2. Preventing SQL injection
         3. Protecting against XSS
         4. Preventing cross-site request forgery (CSRF)
         5. Securing cookies
         6. Using the secure middleware
         7. Summary
      10. 10. Caching, Proxies and Improved Performance
          1. Identifying bottlenecks
          2. Implementing reverse proxies
          3. Implementing caching strategies
             1. Using Least Recently Used
             2. Caching by file
             3. Caching in memory
          4. Implementing HTTP/2
          5. Summary
   7. 2. Module 2
      1. 1. Chat Application with Web Sockets
         1. A simple web server
            1. Templates
               1. Doing things once
               2. Using your own handlers
            2. Properly building and executing Go programs
         2. Modeling a chat room and clients on the server
            1. Modeling the client
            2. Modeling a room
            3. Concurrency programming using idiomatic Go
            4. Turning a room into an HTTP handler
            5. Use helper functions to remove complexity
            6. Creating and using rooms
         3. Building an HTML and JavaScript chat client
            1. Getting more out of templates
         4. Tracing code to get a look under the hood
            1. Writing a package using TDD
               1. Interfaces
               2. Unit tests
               3. Red-green testing
               4. Implementing the interface
               5. Unexported types being returned to users
            2. Using our new trace package
            3. Making tracing optional
            4. Clean package APIs
         5. Summary
      2. 2. Adding Authentication
         1. Handlers all the way down
         2. Making a pretty social sign-in page
         3. Endpoints with dynamic paths
         4. OAuth2
            1. Open source OAuth2 packages
         5. Tell the authentication providers about your app
         6. Implementing external logging in
            1. Logging in
            2. Handling the response from the provider
            3. Presenting the user data
            4. Augmenting messages with additional data
         7. Summary
      3. 3. Three Ways to Implement Profile Pictures
         1. Avatars from the authentication server
            1. Getting the avatar URL
            2. Transmitting the avatar URL
            3. Adding the avatar to the user interface
            4. Logging out
            5. Making things prettier
         2. Implementing Gravatar
            1. Abstracting the avatar URL process
               1. The authentication service and avatar's implementation
               2. Using an implementation
               3. Gravatar implementation
         3. Uploading an avatar picture
            1. User identification
            2. An upload form
            3. Handling the upload
            4. Serving the images
            5. The Avatar implementation for local files
               1. Supporting different file types
            6. Refactoring and optimizing our code
               1. Replacing concrete types with interfaces
               2. Changing interfaces in a test-driven way
               3. Fixing existing implementations
               4. Global variables versus fields
               5. Implementing our new design
               6. Tidying up and testing
         4. Combining all three implementations
         5. Summary
      4. 4. Command-line Tools to Find Domain Names
         1. Pipe design for command-line tools
         2. Five simple programs
            1. Sprinkle
               1. Exercise – configurable transformations
            2. Domainify
               1. Exercise – making top-level domains configurable
            3. Coolify
            4. Synonyms
               1. Using environment variables for configuration
               2. Consuming a web API
               3. Getting domain suggestions
            5. Available
         3. Composing all five programs
            1. One program to rule them all
         4. Summary
      5. 5. Building Distributed Systems and Working with Flexible Data
         1. System design
            1. Database design
         2. Installing the environment
            1. NSQ
               1. NSQ driver for Go
            2. MongoDB
               1. MongoDB driver for Go
            3. Starting the environment
         3. Votes from Twitter
            1. Authorization with Twitter
               1. Extracting the connection
               2. Reading environment variables
            2. Reading from MongoDB
            3. Reading from Twitter
               1. Signal channels
            4. Publishing to NSQ
            5. Gracefully starting and stopping
            6. Testing
         4. Counting votes
            1. Connecting to the database
            2. Consuming messages in NSQ
            3. Keeping the database updated
            4. Responding to Ctrl + C
         5. Running our solution
         6. Summary
      6. 6. Exposing Data and Functionality through a RESTful Data Web Service API
         1. RESTful API design
         2. Sharing data between handlers
         3. Wrapping handler functions
            1. API key
            2. Database session
            3. Per request variables
            4. Cross-browser resource sharing
         4. Responding
         5. Understanding the request
         6. A simple main function to serve our API
            1. Using handler function wrappers
         7. Handling endpoints
            1. Using tags to add metadata to structs
            2. Many operations with a single handler
               1. Reading polls
               2. Creating a poll
               3. Deleting a poll
               4. CORS support
            3. Testing our API using curl
         8. A web client that consumes the API
            1. An index page showing a list of polls
            2. A page to create a new poll
            3. A page to show details of the poll
         9. Running the solution
         10. Summary
      7. 7. Random Recommendations Web Service
         1. Project overview
            1. Project design specifics
         2. Representing data in code
            1. Public views of Go structs
         3. Generating random recommendations
            1. Google Places API key
            2. Enumerators in Go
               1. Test-driven enumerator
            3. Querying the Google Places API
            4. Building recommendations
            5. Handlers that use query parameters
            6. CORS
            7. Testing our API
               1. Web application
         4. Summary
      8. 8. Filesystem Backup
         1. Solution design
            1. Project structure
         2. Backup package
            1. Obvious interfaces?
            2. Implementing ZIP
            3. Has the filesystem changed?
            4. Checking for changes and initiating a backup
               1. Hardcoding is OK for a short while
         3. The user command-line tool
            1. Persisting small data
            2. Parsing arguments
               1. Listing the paths
                  1. String representations for your own types
               2. Adding paths
               3. Removing paths
            3. Using our new tool
         4. The daemon backup tool
            1. Duplicated structures
            2. Caching data
            3. Infinite loops
            4. Updating filedb records
         5. Testing our solution
         6. Summary
   8. 3. Module 3
      1. 1. An Introduction to Concurrency in Go
         1. Introducing goroutines
            1. A patient goroutine
         2. Implementing the defer control mechanism
            1. Using Go's scheduler
            2. Using system variables
         3. Understanding goroutines versus coroutines
         4. Implementing channels
            1. Channel-based sorting at the letter capitalization factory
            2. Cleaning up our goroutines
               1. Buffered or unbuffered channels
            3. Using the select statement
         5. Closures and goroutines
         6. Building a web spider using goroutines and channels
         7. Summary
      2. 2. Understanding the Concurrency Model
         1. Understanding the working of goroutines
         2. Synchronous versus asynchronous goroutines
            1. Designing the web server plan
         3. Visualizing concurrency
         4. RSS in action
            1. An RSS reader with self diagnostics
            2. Imposing a timeout
         5. A little bit about CSP
            1. The dining philosophers problem
         6. Go and the actor model
         7. Object orientation
            1. Demonstrating simple polymorphism in Go
         8. Using concurrency
         9. Managing threads
         10. Using sync and mutexes to lock data
         11. Summary
      3. 3. Developing a Concurrent Strategy
         1. Applying efficiency in complex concurrency
         2. Identifying race conditions with race detection
            1. Using mutual exclusions
            2. Exploring timeouts
               1. Importance of consistency
         3. Synchronizing our concurrent operations
         4. The project – multiuser appointment calendar
            1. Visualizing a concurrent pattern
            2. Developing our server requirements
               1. Web server
                  1. The Gorilla toolkit
               2. Using templates
               3. Time
            3. Endpoints
            4. Custom structs
         5. A multiuser Appointments Calendar
         6. A note on style
         7. A note on immutability
         8. Summary
      4. 4. Data Integrity in an Application
         1. Getting deeper with mutexes and sync
         2. The cost of goroutines
         3. Working with files
         4. Getting low – implementing C
            1. Touching memory in cgo
            2. The structure of cgo
            3. The other way around
               1. Getting even lower – assembly in Go
         5. Distributed Go
         6. Some common consistency models
            1. Distributed shared memory
            2. First-in-first-out – PRAM
            3. Looking at the master-slave model
            4. The producer-consumer problem
            5. Looking at the leader-follower model
            6. Atomic consistency / mutual exclusion
            7. Release consistency
         7. Using memcached
            1. Circuit
         8. Summary
      5. 5. Locks, Blocks, and Better Channels
         1. Understanding blocking methods in Go
            1. Blocking method 1 – a listening, waiting channel
               1. Sending more data types via channels
                  1. Creating a function channel
                  2. Using an interface channel
                  3. Using structs, interfaces, and more complex channels
               2. The net package – a chat server with interfaced channels
                  1. Handling direct messages
            2. Examining our client
            3. Blocking method 2 – the select statement in a loop
         2. Cleaning up goroutines
            1. Blocking method 3 – network connections and reads
         3. Creating channels of channels
         4. Pprof – yet another awesome tool
         5. Handling deadlocks and errors
         6. Summary
      6. 6. C10K – A Non-blocking Web Server in Go
         1. Attacking the C10K problem
            1. Failing of servers at 10,000 concurrent connections
            2. Using concurrency to attack C10K
            3. Taking another approach
         2. Building our C10K web server
            1. Benchmarking against a blocking web server
            2. Handling requests
            3. Routing requests
         3. Serving pages
            1. Parsing our template
            2. External dependencies
               1. Connecting to MySQL
         4. Multithreading and leveraging multiple cores
         5. Exploring our web server
            1. Timing out and moving on
         6. Summary
      7. 7. Performance and Scalability
         1. High performance in Go
            1. Getting deeper into pprof
            2. Parallelism's and concurrency's impact on I/O pprof
         2. Using the App Engine
         3. Distributed Go
            1. Types of topologies
               1. Type 1 – star
               2. Type 2 – mesh
               3. The Publish and Subscribe model
               4. Serialized data
               5. Remote code execution
               6. Other topologies
               7. Message Passing Interface
         4. Some helpful libraries
            1. Nitro profiler
            2. Heka
            3. GoFlow
         5. Memory preservation
            1. Garbage collection in Go
         6. Summary
      8. 8. Concurrent Application Architecture
         1. Designing our concurrent application
         2. Identifying our requirements
         3. Using NoSQL as a data store in Go
            1. MongoDB
            2. Redis
            3. Tiedot
            4. CouchDB
            5. Cassandra
            6. Couchbase
            7. Setting up our data store
         4. Monitoring filesystem changes
         5. Managing logfiles
         6. Handling configuration files
         7. Detecting file changes
            1. Sending changes to clients
            2. Checking records against Couchbase
         8. Backing up our files
         9. Designing our web interface
         10. Reverting a file's history – command line
             1. Using Go in daemons and as a service
         11. Checking the health of our server
         12. Summary
      9. 9. Logging and Testing Concurrency in Go
         1. Handling errors and logging
            1. Breaking out goroutine logs
            2. Using the LiteIDE for richer and easier debugging
            3. Sending errors to screen
            4. Logging errors to file
            5. Logging errors to memory
         2. Using the log4go package for robust logging
            1. Panicking
            2. Recovering
               1. Logging our panics
            3. Catching stack traces with concurrent code
         3. Using the runtime package for granular stack traces
         4. Summary
      10. 10. Advanced Concurrency and Best Practices
          1. Going beyond the basics with channels
          2. Building workers
          3. Implementing nil channel blocks
             1. Using nil channels
          4. Implementing more granular control over goroutines with tomb
          5. Timing out with channels
          6. Building a load balancer with concurrent patterns
          7. Choosing unidirectional and bidirectional channels
             1. Using receive-only or send-only channels
          8. Using an indeterminate channel type
          9. Using Go with unit testing
             1. GoCheck
             2. Ginkgo and Gomega
          10. Using Google App Engine
          11. Utilizing best practices
              1. Structuring your code
              2. Documenting your code
              3. Making your code available via go get
              4. Keeping concurrency out of your packages
          12. Summary
   9. A. Bibliography
   10. Index