Books & Videos

sed & awk, 2nd Edition

UNIX Power Tools

By Dale Dougherty, Arnold Robbins

Publisher: O'Reilly Media

Release Date: November 2010

Pages: 432

Read on Safari with a 10-day trial

Start your free trial now Buy on Amazon

Where’s the cart? Now you can get everything on Safari. To purchase books, visit Amazon or your favorite retailer. Questions? See our FAQ or contact customer service:

1-800-889-8969 / 707-827-7019
support@oreilly.com

Download Example Code

View/Submit Errata

sed & awk describes two text processing programs that are mainstays of the UNIX programmer's toolbox. sed is a "stream editor" for editing streams of text that might be too large to edit as a single file, or that might be generated on the fly as part of a larger data processing step. The most common operation done with sed is substitution, replacing one block of text with another. awk is a complete programming language. Unlike many conventional languages, awk is "data driven" -- you specify what kind of data you are interested in and the operations to be performed when that data is found. awk does many things for you, including automatically opening and closing data files, reading records, breaking the records up into fields, and counting the records. While awk provides the features of most conventional programming languages, it also includes some unconventional features, such as extended regular expression matching and associative arrays. sed & awk describes both programs in detail and includes a chapter of example sed and awk scripts. This edition covers features of sed and awk that are mandated by the POSIX standard. This most notably affects awk, where POSIX standardized a new variable, CONVFMT, and new functions, toupper() and tolower(). The CONVFMT variable specifies the conversion format to use when converting numbers to strings (awk used to use OFMT for this purpose). The toupper() and tolower() functions each take a (presumably mixed case) string argument and return a new version of the string with all letters translated to the corresponding case. In addition, this edition covers GNU sed, newly available since the first edition. It also updates the first edition coverage of Bell Labs nawk and GNU awk (gawk), covers mawk, an additional freely available implementation of awk, and briefly discusses three commercial versions of awk, MKS awk, Thompson Automation awk (tawk), and Videosoft (VSAwk).

Chapter 1 Power Tools for Editing
1. May You Solve Interesting Problems
2. A Stream Editor
3. A Pattern-Matching Programming Language
4. Four Hurdles to Mastering sed and awk
Chapter 2 Understanding Basic Operations
1. Awk, by Sed and Grep, out of Ed
2. Command-Line Syntax
3. Using sed
4. Using awk
5. Using sed and awk Together
Chapter 3 Understanding Regular Expression Syntax
1. That's an Expression
2. A Line-Up of Characters
3. I Never Metacharacter I Didn't Like
Chapter 4 Writing sed Scripts
1. Applying Commands in a Script
2. A Global Perspective on Addressing
3. Testing and Saving Output
4. Four Types of sed Scripts
5. Getting to the PromiSed Land
Chapter 5 Basic sed Commands
1. About the Syntax of sed Commands
2. Comment
3. Substitution
4. Delete
5. Append, Insert, and Change
6. List
7. Transform
8. Print
9. Print Line Number
10. Next
11. Reading and Writing Files
12. Quit
Chapter 6 Advanced sed Commands
1. Multiline Pattern Space
2. A Case for Study
3. Hold That Line
4. Advanced Flow Control Commands
5. To Join a Phrase
Chapter 7 Writing Scripts for awk
1. Playing the Game
2. Hello, World
3. Awk's Programming Model
4. Pattern Matching
5. Records and Fields
6. Expressions
7. System Variables
8. Relational and Boolean Operators
9. Formatted Printing
10. Passing Parameters Into a Script
11. Information Retrieval
Chapter 8 Conditionals, Loops, and Arrays
1. Conditional Statements
2. Looping
3. Other Statements That Affect Flow Control
4. Arrays
5. An Acronym Processor
6. System Variables That Are Arrays
Chapter 9 Functions
1. Arithmetic Functions
2. String Functions
3. Writing Your Own Functions
Chapter 10 The Bottom Drawer
1. The getline Function
2. The close( ) Function
3. The system( ) Function
4. A Menu-Based Command Generator
5. Directing Output to Files and Pipes
6. Generating Columnar Reports
7. Debugging
8. Limitations
9. Invoking awk Using the #! Syntax
Chapter 11 A Flock of awks
1. Original awk
2. Freely Available awks
3. Commercial awks
4. Epilogue
Chapter 12 Full-Featured Applications
1. An Interactive Spelling Checker
2. Generating a Formatted Index
3. Spare Details of the masterindex Program
Chapter 13 A Miscellany of Scripts
1. uutot.awk—Report UUCP Statistics
2. phonebill—Track Phone Usage
3. combine—Extract Multipart uuencoded Binaries
4. mailavg—Check Size of Mailboxes
5. adj—Adjust Lines for Text Files
6. readsource—Format Program Source Files for troff
7. gent—Get a termcap Entry
8. plpr—lpr Preprocessor
9. transpose—Perform a Matrix Transposition
10. m1—Simple Macro Processor

Appendix A Quick Reference for sed
1. Command-Line Syntax
2. Syntax of sed Commands
3. Command Summary for sed
Appendix B Quick Reference for awk
1. Command-Line Syntax
2. Language Summary for awk
3. Command Summary for awk
Appendix C Supplement for Chapter 12
1. Full Listing of spellcheck.awk
2. Listing of masterindex Shell Script
3. Documentation for masterindex
4. Background Details
5. Coding Index Entries
6. Output Format
7. Compiling a Master Index
Colophon

Books & Videos

sed & awk, 2nd Edition

UNIX Power Tools

Read on Safari with a 10-day trial

Table of Contents

Chapter 1 Power Tools for Editing

May You Solve Interesting Problems

A Stream Editor

A Pattern-Matching Programming Language

Four Hurdles to Mastering sed and awk

Chapter 2 Understanding Basic Operations

Awk, by Sed and Grep, out of Ed

Command-Line Syntax

Using sed

Using awk

Using sed and awk Together

Chapter 3 Understanding Regular Expression Syntax

That's an Expression

A Line-Up of Characters

I Never Metacharacter I Didn't Like

Chapter 4 Writing sed Scripts

Applying Commands in a Script

A Global Perspective on Addressing

Testing and Saving Output

Four Types of sed Scripts

Getting to the PromiSed Land

Chapter 5 Basic sed Commands

About the Syntax of sed Commands

Comment

Substitution

Delete

Append, Insert, and Change

List

Transform

Print

Print Line Number

Next

Reading and Writing Files

Quit

Chapter 6 Advanced sed Commands

Multiline Pattern Space

A Case for Study

Hold That Line

Advanced Flow Control Commands

To Join a Phrase

Chapter 7 Writing Scripts for awk

Playing the Game

Hello, World

Awk's Programming Model

Pattern Matching

Records and Fields

Expressions

System Variables

Relational and Boolean Operators

Formatted Printing

Passing Parameters Into a Script

Information Retrieval

Chapter 8 Conditionals, Loops, and Arrays

Conditional Statements

Looping

Other Statements That Affect Flow Control

Arrays

An Acronym Processor

System Variables That Are Arrays

Chapter 9 Functions

Arithmetic Functions

String Functions

Writing Your Own Functions

Chapter 10 The Bottom Drawer

The getline Function

The close( ) Function

The system( ) Function

A Menu-Based Command Generator

Directing Output to Files and Pipes

Generating Columnar Reports

Debugging

Limitations

Invoking awk Using the #! Syntax

Chapter 11 A Flock of awks

Original awk