Extracting Web Page Content
Most programming languages provide functions for extracting text from HTML pages.
The instructor personally prefers to use a Unix tool, lynx, which is a text browser for the World Wide Web.
It is a Web client for users running cursor-addressable, character-cell display devices.
It will display HTML documents containing links to files residing on the local system.
The syntax of using the lynx is given below:
lynx [options] [path or URL]
where some of the options are
-dump, which dumps the formatted output of the default document or one specified on the command line to standard output,
-help, which displays a complete list of current options,
-listonly, which shows only the list of links (for -dump),
-nolist, which disables the link list feature in dumps, and
-source, which works the same as dump but outputs HTML source instead of formatted text.
lynx can be used to access information on the World Wide Web, or to build information systems intended primarily for local access.
For example, lynx has been used to build several Campus Wide Information Systems (CWIS).
In addition, lynx can be used to build systems isolated within a single LAN.