Perl & LWP
Fetching Web Pages, Parsing HTML, Writing Spiders & More
By Sean M. Burke
Publisher: O'Reilly Media
Final Release Date: June 2002
Pages: 262

Perl soared to popularity as a language for creating and managing web content, but with LWP (Library for WWW in Perl), Perl is equally adept at consuming information on the Web. LWP is a suite of modules for fetching and processing web pages.The Web is a vast data source that contains everything from stock prices to movie credits, and with LWP all that data is just a few lines of code away. Anything you do on the Web, whether it's buying or selling, reading or writing, uploading or downloading, news to e-commerce, can be controlled with Perl and LWP. You can automate Web-based purchase orders as easily as you can set up a program to download MP3 files from a web site.Perl & LWP covers:

  • Understanding LWP and its design
  • Fetching and analyzing URLs
  • Extracting information from HTML using regular expressions and tokens
  • Working with the structure of HTML documents using trees
  • Setting and inspecting HTTP headers and response codes
  • Managing cookies
  • Accessing information that requires authentication
  • Extracting links
  • Cooperating with proxy caches
  • Writing web spiders (also known as robots) in a safe fashion
Perl & LWP includes many step-by-step examples that show how to apply the various techniques. Programs to extract information from the web sites of BBC News, Altavista,, and the Weather Underground, to name just a few, are explained in detail, so that you understand how and why they work.Perl programmers who want to automate and mine the web can pick up this book and be immediately productive. Written by a contributor to LWP, and with a foreword by one of LWP's creators, Perl & LWP is the authoritative guide to this powerful and popular toolkit.
Table of Contents
Product Details
Recommended for You
Customer Reviews


by PowerReviews
oreillyPerl & LWP

(based on 2 reviews)

Ratings Distribution

  • 5 Stars



  • 4 Stars



  • 3 Stars



  • 2 Stars



  • 1 Stars



Reviewed by 2 customers

Displaying reviews 1-2

Back to top

(2 of 2 customers found this review helpful)


Top marks for a useful book

By Edmonton Linux User Group

from Edmonton

Comments about oreilly Perl & LWP:

To date, I haven't had much need to pull data from web pages, so I set up a simple interactive telnet session in Perl to grab what I needed. Not elegant, but it worked. Unfortunately, it was rather hit and miss, mostly because I didn't understand the protocol as well as I should have.

Enter "Perl & LWP," a book describing the "Library for WWW in Perl." Not only did the book have the expected "here's how the library works," but also an explanation of the HTTP protocol as it applies to what you are doing (or trying to do). This is particularly good for people like me who will learn the minimum necessary to get a job done, and revisit things later to learn more thoroughly, so I improve on what I've already done, an also to be able to apply what I've learned elsewhere. Some useful utilites were also provided.

Top marks for a useful book - script your way through static and dynamic web sites, deal with cookies, proxy caches, and authentication. The web was intended to be interactive, and the author shows us how to scriptify it.

Reviewed by ELUG (

(5 of 6 customers found this review helpful)


Perl & LWP Review

By avirup

from Undisclosed

Comments about oreilly Perl & LWP:

Nice & concise book.

Only backdraw is that most of the examples in this book donot work.

(as websites used have either moved or have changed their format) .

It would have been helpful if the examples had run.


Displaying reviews 1-2

Back to top

Buy 2 Get 1 Free Free Shipping Guarantee
Buying Options
Immediate Access - Go Digital what's this?
Ebook:  $31.99
Formats:  APK, DAISY, ePub, Mobi, PDF
Print & Ebook:  $43.99
Print:  $39.99