•  Table of Contents
•  Index
•  Reviews
•  Reader Reviews
•  Errata
UNIX Power Tools, 3rd Edition
By Shelley Powers, Jerry Peek, Tim O'Reilly, Mike Loukides,  et. al. 
   
Publisher : O'Reilly
Pub Date : October 2002
ISBN : 0-596-00330-7
Pages : 1136

The latest edition of this best-selling favorite is loaded with vital information on Linux, Darwin, and BSD. Unix Power Tools 3rd Edition now offers more coverage of bash, zsh, and other new shells, along with discussions about modern utilities and applications. Several sections focus on security and Internet access. There is a new chapter on access to Unix from Windows, and expanded coverage of software installation and packaging, as well as basic information on Perl and Python.

777

  Copyright
    How to Use This Book
    Preface
      A Book for Browsing
      Like an Almanac
      Like a News Magazine
      Like a Hypertext Database
      Programs on the Web
      About Unix Versions
      Cross-References
      What's New in the Third Edition
      Typefaces and Other Conventions
      The Authors
      The Fine Print
      Request for Comments
      Acknowledgments for the First Edition
      Acknowledgments for the Second Edition
      Acknowledgments for the Third Edition
   
    Part I:  Basic Unix Environment
      Chapter 1.  Introduction
      Section 1.1.  What's Special About Unix?
      Section 1.2.  Power Grows on You
      Section 1.3.  The Core of Unix
      Section 1.4.  Communication with Unix
      Section 1.5.  Programs Are Designed to Work Together
      Section 1.6.  There Are Many Shells
      Section 1.7.  Which Shell Am I Running?
      Section 1.8.  Anyone Can Program the Shell
      Section 1.9.  Internal and External Commands
      Section 1.10.  The Kernel and Daemons
      Section 1.11.  Filenames
      Section 1.12.  Filename Extensions
      Section 1.13.  Wildcards
      Section 1.14.  The Tree Structure of the Filesystem
      Section 1.15.  Your Home Directory
      Section 1.16.  Making Pathnames
      Section 1.17.  File Access Permissions
      Section 1.18.  The Superuser (Root)
      Section 1.19.  When Is a File Not a File?
      Section 1.20.  Scripting
      Section 1.21.  Unix Networking and Communications
      Section 1.22.  The X Window System
   
      Chapter 2.  Getting Help
      Section 2.1.  The man Command
      Section 2.2.  whatis: One-Line Command Summaries
      Section 2.3.  whereis: Finding Where a Command Is Located
      Section 2.4.  Searching Online Manual Pages
      Section 2.5.  How Unix Systems Remember Their Names
      Section 2.6.  Which Version Am I Using?
      Section 2.7.  What tty Am I On?
      Section 2.8.  Who's On?
      Section 2.9.  The info Command
   
   
    Part II:  Customizing Your Environment
      Chapter 3.  Setting Up Your Unix Shell
      Section 3.1.  What Happens When You Log In
      Section 3.2.  The Mac OS X Terminal Application
      Section 3.3.  Shell Setup Files — Which, Where, and Why
      Section 3.4.  Login Shells, Interactive Shells
      Section 3.5.  What Goes in Shell Setup Files?
      Section 3.6.  Tip for Changing Account Setup: Keep a Shell Ready
      Section 3.7.  Use Absolute Pathnames in Shell Setup Files
      Section 3.8.  Setup Files Aren't Read When You Want?
      Section 3.9.  Gotchas in set prompt Test
      Section 3.10.  Automatic Setups for Different Terminals
      Section 3.11.  Terminal Setup: Testing TERM
      Section 3.12.  Terminal Setup: Testing Remote Hostname and X Display
      Section 3.13.  Terminal Setup: Testing Port
      Section 3.14.  Terminal Setup: Testing Environment Variables
      Section 3.15.  Terminal Setup: Searching Terminal Table
      Section 3.16.  Terminal Setup: Testing Window Size
      Section 3.17.  Terminal Setup: Setting and Testing Window Name
      Section 3.18.  A .cshrc.$HOST File for Per Host Setup
      Section 3.19.  Making a "Login" Shell
      Section 3.20.  RC Files
      Section 3.21.  Make Your Own Manpages Without Learning troff
      Section 3.22.  Writing a Simple Manpage with the -man Macros
   
      Chapter 4.  Interacting with Your Environment
      Section 4.1.  Basics of Setting the Prompt
      Section 4.2.  Static Prompts
      Section 4.3.  Dynamic Prompts
      Section 4.4.  Simulating Dynamic Prompts
      Section 4.5.  C-Shell Prompt Causes Problems in vi, rsh, etc.
      Section 4.6.  Faster Prompt Setting with Built-ins
      Section 4.7.  Multiline Shell Prompts
      Section 4.8.  Session Info in Window Title or Status Line
      Section 4.9.  A "Menu Prompt" for Naive Users
      Section 4.10.  Highlighting and Color in Shell Prompts
      Section 4.11.  Right-Side Prompts
      Section 4.12.  Show Subshell Level with $SHLVL
      Section 4.13.  What Good Is a Blank Shell Prompt?
      Section 4.14.  dirs in Your Prompt: Better Than $cwd
      Section 4.15.  External Commands Send Signals to Set Variables
      Section 4.16.  Preprompt, Pre-execution, and Periodic Commands
      Section 4.17.  Running Commands When You Log Out
      Section 4.18.  Running Commands at Bourne/Korn Shell Logout
      Section 4.19.  Stop Accidental Bourne-Shell Logouts
   
      Chapter 5.  Getting the Most out of Terminals, xterm, and X Windows
      Section 5.1.  There's a Lot to Know About Terminals
      Section 5.2.  The Idea of a Terminal Database
      Section 5.3.  Setting the Terminal Type When You Log In
      Section 5.4.  Querying Your Terminal Type: qterm
      Section 5.5.  Querying Your xterm Size: resize
      Section 5.6.  Checklist: Terminal Hangs When I Log In
      Section 5.7.  Find Out Terminal Settings with stty
      Section 5.8.  Setting Your Erase, Kill, and Interrupt Characters
      Section 5.9.  Working with xterm and Friends
      Section 5.10.  Login xterms and rxvts
      Section 5.11.  Working with Scrollbars
      Section 5.12.  How Many Lines to Save?
      Section 5.13.  Simple Copy and Paste in xterm
      Section 5.14.  Defining What Makes Up a Word for Selection Purposes
      Section 5.15.  Setting the Titlebar and Icon Text
      Section 5.16.  The Simple Way to Pick a Font
      Section 5.17.  The xterm Menus
      Section 5.18.  Changing Fonts Dynamically
      Section 5.19.  Working with xclipboard
      Section 5.20.  Problems with Large Selections
      Section 5.21.  Tips for Copy and Paste Between Windows
      Section 5.22.  Running a Single Command with xterm -e
      Section 5.23.  Don't Quote Arguments to xterm -e
   
      Chapter 6.  Your X Environment
      Section 6.1.  Defining Keys and Button Presses with xmodmap
      Section 6.2.  Using xev to Learn Keysym Mappings
      Section 6.3.  X Resource Syntax
      Section 6.4.  X Event Translations
      Section 6.5.  Setting X Resources: Overview
      Section 6.6.  Setting Resources with the -xrm Option
      Section 6.7.  How -name Affects Resources
      Section 6.8.  Setting Resources with xrdb
      Section 6.9.  Listing the Current Resources for a Client: appres
      Section 6.10.  Starting Remote X Clients
   
   
    Part III:  Working with Files and Directories
      Chapter 7.  Directory Organization
      Section 7.1.  What? Me, Organized?
      Section 7.2.  Many Homes
      Section 7.3.  Access to Directories
      Section 7.4.  A bin Directory for Your Programs and Scripts
      Section 7.5.  Private (Personal) Directories
      Section 7.6.  Naming Files
      Section 7.7.  Make More Directories!
      Section 7.8.  Making Directories Made Easier
   
      Chapter 8.  Directories and Files
      Section 8.1.  Everything but the find Command
      Section 8.2.  The Three Unix File Times
      Section 8.3.  Finding Oldest or Newest Files with ls -t and ls -u
      Section 8.4.  List All Subdirectories with ls -R
      Section 8.5.  The ls -d Option
      Section 8.6.  Color ls
      Section 8.7.  Some GNU ls Features
      Section 8.8.  A csh Alias to List Recently Changed Files
      Section 8.9.  Showing Hidden Files with ls -A and -a
      Section 8.10.  Useful ls Aliases
      Section 8.11.  Can't Access a File? Look for Spaces in the Name
      Section 8.12.  Showing Nonprintable Characters in Filenames
      Section 8.13.  Counting Files by Types
      Section 8.14.  Listing Files by Age and Size
      Section 8.15.  newer: Print the Name of the Newest File
      Section 8.16.  oldlinks: Find Unconnected Symbolic Links
      Section 8.17.  Picking a Unique Filename Automatically
   
      Chapter 9.  Finding Files with find
      Section 9.1.  How to Use find
      Section 9.2.  Delving Through a Deep Directory Tree
      Section 9.3.  Don't Forget -print
      Section 9.4.  Looking for Files with Particular Names
      Section 9.5.  Searching for Old Files
      Section 9.6.  Be an Expert on find Search Operators
      Section 9.7.  The Times That find Finds
      Section 9.8.  Exact File-Time Comparisons
      Section 9.9.  Running Commands on What You Find
      Section 9.10.  Using -exec to Create Custom Tests
      Section 9.11.  Custom -exec Tests Applied
      Section 9.12.  Finding Many Things with One Command
      Section 9.13.  Searching for Files by Type
      Section 9.14.  Searching for Files by Size
      Section 9.15.  Searching for Files by Permission
      Section 9.16.  Searching by Owner and Group
      Section 9.17.  Duplicating a Directory Tree
      Section 9.18.  Using "Fast find" Databases
      Section 9.19.  Wildcards with "Fast find" Database
      Section 9.20.  Finding Files (Much) Faster with a find Database
      Section 9.21.  grepping a Directory Tree
      Section 9.22.  lookfor: Which File Has That Word?
      Section 9.23.  Using Shell Arrays to Browse Directories
      Section 9.24.  Finding the (Hard) Links to a File
      Section 9.25.  Finding Files with -prune
      Section 9.26.  Quick finds in the Current Directory
      Section 9.27.  Skipping Parts of a Tree in find
      Section 9.28.  Keeping find from Searching Networked Filesystem
   
      Chapter 10.  Linking, Renaming, and Copying Files
      Section 10.1.  What's So Complicated About Copying Files
      Section 10.2.  What's Really in a Directory?
      Section 10.3.  Files with Two or More Names
      Section 10.4.  More About Links
      Section 10.5.  Creating and Removing Links
      Section 10.6.  Stale Symbolic Links
      Section 10.7.  Linking Directories
      Section 10.8.  Showing the Actual Filenames for Symbolic Links
      Section 10.9.  Renaming, Copying, or Comparing a Set of Files
      Section 10.10.  Renaming a List of Files Interactively
      Section 10.11.  One More Way to Do It
      Section 10.12.  Copying Directory Trees with cp -r
      Section 10.13.  Copying Directory Trees with tar and Pipes
   
      Chapter 11.  Comparing Files
      Section 11.1.  Checking Differences with diff
      Section 11.2.  Comparing Three Different Versions with diff3
      Section 11.3.  Context diffs
      Section 11.4.  Side-by-Side diffs: sdiff
      Section 11.5.  Choosing Sides with sdiff
      Section 11.6.  Problems with diff and Tabstops
      Section 11.7.  cmp and diff
      Section 11.8.  Comparing Two Files with comm
      Section 11.9.  More Friendly comm Output
      Section 11.10.  make Isn't Just for Programmers!
      Section 11.11.  Even More Uses for make
   
      Chapter 12.  Showing What's in a File
      Section 12.1.  Cracking the Nut
      Section 12.2.  What Good Is a cat?
      Section 12.3.  "less" is More
      Section 12.4.  Show Nonprinting Characters with cat -v or od -c
      Section 12.5.  What's in That Whitespace?
      Section 12.6.  Finding File Types
      Section 12.7.  Squash Extra Blank Lines
      Section 12.8.  How to Look at the End of a File: tail
      Section 12.9.  Finer Control on tail
      Section 12.10.  How to Look at Files as They Grow
      Section 12.11.  GNU tail File Following
      Section 12.12.  Printing the Top of a File
      Section 12.13.  Numbering Lines
   
      Chapter 13.  Searching Through Files
      Section 13.1.  Different Versions of grep
      Section 13.2.  Searching for Text with grep
      Section 13.3.  Finding Text That Doesn't Match
      Section 13.4.  Extended Searching for Text with egrep
      Section 13.5.  grepping for a List of Patterns
      Section 13.6.  Approximate grep: agrep
      Section 13.7.  Search RCS Files with rcsgrep
      Section 13.8.  GNU Context greps
      Section 13.9.  A Multiline Context grep Using sed
      Section 13.10.  Compound Searches
      Section 13.11.  Narrowing a Search Quickly
      Section 13.12.  Faking Case-Insensitive Searches
      Section 13.13.  Finding a Character in a Column
      Section 13.14.  Fast Searches and Spelling Checks with "look"
      Section 13.15.  Finding Words Inside Binary Files
      Section 13.16.  A Highlighting grep
   
      Chapter 14.  Removing Files
      Section 14.1.  The Cycle of Creation and Destruction
      Section 14.2.  How Unix Keeps Track of Files: Inodes
      Section 14.3.  rm and Its Dangers
      Section 14.4.  Tricks for Making rm Safer
      Section 14.5.  Answer "Yes" or "No" Forever with yes
      Section 14.6.  Remove Some, Leave Some
      Section 14.7.  A Faster Way to Remove Files Interactively
      Section 14.8.  Safer File Deletion in Some Directories
      Section 14.9.  Safe Delete: Pros and Cons
      Section 14.10.  Deletion with Prejudice: rm -f
      Section 14.11.  Deleting Files with Odd Names
      Section 14.12.  Using Wildcards to Delete Files with Strange Names
      Section 14.13.  Handling a Filename Starting with a Dash (-)
      Section 14.14.  Using unlink to Remove a File with a Strange Name
      Section 14.15.  Removing a Strange File by its i-number
      Section 14.16.  Problems Deleting Directories
      Section 14.17.  Deleting Stale Files
      Section 14.18.  Removing Every File but One
      Section 14.19.  Using find to Clear Out Unneeded Files
   
      Chapter 15.  Optimizing Disk Space
      Section 15.1.  Disk Space Is Cheap
      Section 15.2.  Instead of Removing a File, Empty It
      Section 15.3.  Save Space with "Bit Bucket" Log Files and Mailboxes
      Section 15.4.  Save Space with a Link
      Section 15.5.  Limiting File Sizes
      Section 15.6.  Compressing Files to Save Space
      Section 15.7.  Save Space: tar and compress a Directory Tree
      Section 15.8.  How Much Disk Space?
      Section 15.9.  Compressing a Directory Tree: Fine-Tuning
      Section 15.10.  Save Space in Executable Files with strip
      Section 15.11.  Disk Quotas
   
   
    Part IV:  Basic Editing
      Chapter 16.  Spell Checking, Word Counting, and Textual Analysis
      Section 16.1.  The Unix spell Command
      Section 16.2.  Check Spelling Interactively with ispell
      Section 16.3.  How Do I Spell That Word?
      Section 16.4.  Inside spell
      Section 16.5.  Adding Words to ispell's Dictionary
      Section 16.6.  Counting Lines, Words, and Characters: wc
      Section 16.7.  Find a a Doubled Word
      Section 16.8.  Looking for Closure
      Section 16.9.  Just the Words, Please
   
      Chapter 17.  vi Tips and Tricks
      Section 17.1.  The vi Editor: Why So Much Material?
      Section 17.2.  What We Cover
      Section 17.3.  Editing Multiple Files with vi
      Section 17.4.  Edits Between Files
      Section 17.5.  Local Settings for vi
      Section 17.6.  Using Buffers to Move or Copy Text
      Section 17.7.  Get Back What You Deleted with Numbered Buffers
      Section 17.8.  Using Search Patterns and Global Commands
      Section 17.9.  Confirming Substitutions in vi
      Section 17.10.  Keep Your Original File, Write to a New File
      Section 17.11.  Saving Part of a File
      Section 17.12.  Appending to an Existing File
      Section 17.13.  Moving Blocks of Text by Patterns
      Section 17.14.  Useful Global Commands (with Pattern Matches)
      Section 17.15.  Counting Occurrences; Stopping Search Wraps
      Section 17.16.  Capitalizing Every Word on a Line
      Section 17.17.  Per-File Setups in Separate Files
      Section 17.18.  Filtering Text Through a Unix Command
      Section 17.19.  vi File Recovery Versus Networked Filesystems
      Section 17.20.  Be Careful with vi -r Recovered Buffers
      Section 17.21.  Shell Escapes: Running One UnixCommand While Using Another
      Section 17.22.  vi Compound Searches
      Section 17.23.  vi Word Abbreviation
      Section 17.24.  Using vi Abbreviations as Commands (Cut and Paste Between vi's)
      Section 17.25.  Fixing Typos with vi Abbreviations
      Section 17.26.  vi Line Commands Versus Character Commands
      Section 17.27.  Out of Temporary Space? Use Another Directory
      Section 17.28.  Neatening Lines
      Section 17.29.  Finding Your Place with Undo
      Section 17.30.  Setting Up vi with the .exrc File
   
      Chapter 18.  Creating Custom Commands in vi
      Section 18.1.  Why Type More Than You Have To?
      Section 18.2.  Save Time and Typing with the vi map Commands
      Section 18.3.  What You Lose When You Use map!
      Section 18.4.  vi @-Functions
      Section 18.5.  Keymaps for Pasting into a Window Running vi
      Section 18.6.  Protecting Keys from Interpretation by ex
      Section 18.7.  Maps for Repeated Edits
      Section 18.8.  More Examples of Mapping Keys in vi
      Section 18.9.  Repeating a vi Keymap
      Section 18.10.  Typing in Uppercase Without CAPS LOCK
      Section 18.11.  Text-Input Mode Cursor Motion with No Arrow Keys
      Section 18.12.  Don't Lose Important Functions with vi Maps: Use noremap
      Section 18.13.  vi Macro for Splitting Long Lines
      Section 18.14.  File-Backup Macros
   
      Chapter 19.  GNU Emacs
      Section 19.1.  Emacs: The Other Editor
      Section 19.2.  Emacs Features: A Laundry List
      Section 19.3.  Customizations and How to Avoid Them
      Section 19.4.  Backup and Auto-Save Files
      Section 19.5.  Putting Emacs in Overwrite Mode
      Section 19.6.  Command Completion
      Section 19.7.  Mike's Favorite Timesavers
      Section 19.8.  Rational Searches
      Section 19.9.  Unset PWD Before Using Emacs
      Section 19.10.  Inserting Binary Characters into Files
      Section 19.11.  Using Word-Abbreviation Mode
      Section 19.12.  Directories for Emacs Hacks
      Section 19.13.  An Absurd Amusement
   
      Chapter 20.  Batch Editing
      Section 20.1.  Why Line Editors Aren't Dinosaurs
      Section 20.2.  Writing Editing Scripts
      Section 20.3.  Line Addressing
      Section 20.4.  Useful ex Commands
      Section 20.5.  Running Editing Scripts Within vi
      Section 20.6.  Change Many Files by Editing Just One
      Section 20.7.  ed/ex Batch Edits: A Typical Example
      Section 20.8.  Batch Editing Gotcha: Editors Fail on Big Files
      Section 20.9.  patch: Generalized Updating of Files That Differ
      Section 20.10.  Quick Reference: awk
      Section 20.11.  Versions of awk
   
      Chapter 21.  You Can't Quite Call This Editing
      Section 21.1.  And Why Not?
      Section 21.2.  Neatening Text with fmt
      Section 21.3.  Alternatives to fmt
      Section 21.4.  Clean Up Program Comment Blocks
      Section 21.5.  Remove Mail/News Headers with behead
      Section 21.6.  Low-Level File Butchery with dd
      Section 21.7.  offset: Indent Text
      Section 21.8.  Centering Lines in a File
      Section 21.9.  Splitting Files at Fixed Points: split
      Section 21.10.  Splitting Files by Context: csplit
      Section 21.11.  Hacking on Characters with tr
      Section 21.12.  Encoding "Binary" Files into ASCII
      Section 21.13.  Text Conversion with dd
      Section 21.14.  Cutting Columns or Fields
      Section 21.15.  Making Text in Columns with pr
      Section 21.16.  Make Columns Automatically with column
      Section 21.17.  Straightening Jagged Columns
      Section 21.18.  Pasting Things in Columns
      Section 21.19.  Joining Lines with join
      Section 21.20.  What Is (or Isn't) Unique?
      Section 21.21.  Rotating Text
   
      Chapter 22.  Sorting
      Section 22.1.  Putting Things in Order
      Section 22.2.  Sort Fields: How sort Sorts
      Section 22.3.  Changing the sort Field Delimiter
      Section 22.4.  Confusion with Whitespace Field Delimiters
      Section 22.5.  Alphabetic and Numeric Sorting
      Section 22.6.  Miscellaneous sort Hints
      Section 22.7.  lensort: Sort Lines by Length
      Section 22.8.  Sorting a List of People by Last Name
   
   
    Part V:  Processes and the Kernel
      Chapter 23.  Job Control
      Section 23.1.  Job Control in a Nutshell
      Section 23.2.  Job Control Basics
      Section 23.3.  Using jobs Effectively
      Section 23.4.  Some Gotchas with Job Control
      Section 23.5.  The "Current Job" Isn't Always What You Expect
      Section 23.6.  Job Control and autowrite: Real Timesavers!
      Section 23.7.  System Overloaded? Try Stopping Some Jobs
      Section 23.8.  Notification When Jobs Change State
      Section 23.9.  Stop Background Output with stty tostop
      Section 23.10.  nohup
      Section 23.11.  Disowning Processes
      Section 23.12.  Linux Virtual Consoles
      Section 23.13.  Stopping Remote Login Sessions
   
      Chapter 24.  Starting, Stopping, and Killing Processes
      Section 24.1.  What's in This Chapter
      Section 24.2.  fork and exec
      Section 24.3.  Managing Processes: Overall Concepts
      Section 24.4.  Subshells
      Section 24.5.  The ps Command
      Section 24.6.  The Controlling Terminal
      Section 24.7.  Tracking Down Processes
      Section 24.8.  Why ps Prints Some Commands in Parentheses
      Section 24.9.  The /proc Filesystem
      Section 24.10.  What Are Signals?
      Section 24.11.  Killing Foreground Jobs
      Section 24.12.  Destroying Processes with kill
      Section 24.13.  Printer Queue Watcher: A Restartable Daemon Shell Script
      Section 24.14.  Killing All Your Processes
      Section 24.15.  Killing Processes by Name?
      Section 24.16.  Kill Processes Interactively
      Section 24.17.  Processes Out of Control? Just STOP Them
      Section 24.18.  Cleaning Up an Unkillable Process
      Section 24.19.  Why You Can't Kill a Zombie
      Section 24.20.  The Process Chain to Your Window
      Section 24.21.  Terminal Windows Without Shells
      Section 24.22.  Close a Window by Killing Its Process(es)
   
      Chapter 25.  Delayed Execution
      Section 25.1.  Building Software Robots the Easy Way
      Section 25.2.  Periodic Program Execution: The cron Facility
      Section 25.3.  Adding crontab Entries
      Section 25.4.  Including Standard Input Within a cron Entry
      Section 25.5.  The at Command
      Section 25.6.  Making Your at Jobs Quiet
      Section 25.7.  Checking and Removing Jobs
      Section 25.8.  Avoiding Other at and cron Jobs
      Section 25.9.  Waiting a Little While: sleep
   
      Chapter 26.  System Performance and Profiling
      Section 26.1.  Timing Is Everything
      Section 26.2.  Timing Programs
      Section 26.3.  What Commands Are Running and How Long Do They Take?
      Section 26.4.  Checking System Load: uptime
      Section 26.5.  Know When to Be "nice" to Other Users...and When Not To
      Section 26.6.  A nice Gotcha
      Section 26.7.  Changing a Running Job's Niceness
   
   
    Part VI:  Scripting
      Chapter 27.  Shell Interpretation
      Section 27.1.  What the Shell Does
      Section 27.2.  How the Shell Executes Other Commands
      Section 27.3.  What's a Shell, Anyway?
      Section 27.4.  Command Evaluation and Accidentally Overwriting Files
      Section 27.5.  Output Command-Line Arguments One by One
      Section 27.6.  Controlling Shell Command Searches
      Section 27.7.  Wildcards Inside Aliases
      Section 27.8.  eval: When You Need Another Chance
      Section 27.9.  Which One Will bash Use?
      Section 27.10.  Which One Will the C Shell Use?
      Section 27.11.  Is It "2>&1 file" or "> file 2>&1"? Why?
      Section 27.12.  Bourne Shell Quoting
      Section 27.13.  Differences Between Bourne and C Shell Quoting
      Section 27.14.  Quoting Special Characters in Filenames
      Section 27.15.  Verbose and Echo Settings Show Quoting
      Section 27.16.  Here Documents
      Section 27.17.  "Special" Characters and Operators
      Section 27.18.  How Many Backslashes?
   
      Chapter 28.  Saving Time on the Command Line
      Section 28.1.  What's Special About the Unix Command Line
      Section 28.2.  Reprinting Your Command Line with CTRL-r
      Section 28.3.  Use Wildcards to Create Files?
      Section 28.4.  Build Strings with { }
      Section 28.5.  String Editing (Colon) Operators
      Section 28.6.  Automatic Completion
      Section 28.7.  Don't Match Useless Files in Filename Completion
      Section 28.8.  Repeating Commands
      Section 28.9.  Repeating and Varying Commands
      Section 28.10.  Repeating a Command with Copy-and-Paste
      Section 28.11.  Repeating a Time-Varying Command
      Section 28.12.  Multiline Commands, Secondary Prompts
      Section 28.13.  Here Document Example #1: Unformatted Form Letters
      Section 28.14.  Command Substitution
      Section 28.15.  Handling Lots of Text with Temporary Files
      Section 28.16.  Separating Commands with Semicolons
      Section 28.17.  Dealing with Too Many Arguments
      Section 28.18.  Expect
   
      Chapter 29.  Custom Commands
      Section 29.1.  Creating Custom Commands
      Section 29.2.  Introduction to Shell Aliases
      Section 29.3.  C-Shell Aliases with Command-Line Arguments
      Section 29.4.  Setting and Unsetting Bourne-Type Aliases
      Section 29.5.  Korn-Shell Aliases
      Section 29.6.  zsh Aliases
      Section 29.7.  Sourceable Scripts
      Section 29.8.  Avoiding C-Shell Alias Loops
      Section 29.9.  How to Put if-then-else in a C-Shell Alias
      Section 29.10.  Fix Quoting in csh Aliases with makealias and quote
      Section 29.11.  Shell Function Basics
      Section 29.12.  Shell Function Specifics
      Section 29.13.  Propagating Shell Functions
      Section 29.14.  Simulated Bourne Shell Functions and Aliases
   
      Chapter 30.  The Use of History
      Section 30.1.  The Lessons of History
      Section 30.2.  History in a Nutshell
      Section 30.3.  My Favorite Is !$
      Section 30.4.  My Favorite Is !:n*
      Section 30.5.  My Favorite Is ^^
      Section 30.6.  Using !$ for Safety with Wildcards
      Section 30.7.  History by Number
      Section 30.8.  History Substitutions
      Section 30.9.  Repeating a Cycle of Commands
      Section 30.10.  Running a Series of Commands on a File
      Section 30.11.  Check Your History First with :p
      Section 30.12.  Picking Up Where You Left Off
      Section 30.13.  Pass History to Another Shell
      Section 30.14.  Shell Command-Line Editing
      Section 30.15.  Changing History Characters with histchars
      Section 30.16.  Instead of Changing History Characters
   
      Chapter 31.  Moving Around in a Hurry
      Section 31.1.  Getting Around the Filesystem
      Section 31.2.  Using Relative and Absolute Pathnames
      Section 31.3.  What Good Is a Current Directory?
      Section 31.4.  How Does Unix Find Your Current Directory?
      Section 31.5.  Saving Time When You Change Directories: cdpath
      Section 31.6.  Loop Control: break and continue
      Section 31.7.  The Shells' pushd and popd Commands
      Section 31.8.  Nice Aliases for pushd
      Section 31.9.  Quick cds with Aliases
      Section 31.10.  cd by Directory Initials
      Section 31.11.  Finding (Anyone's) Home Directory, Quickly
      Section 31.12.  Marking Your Place with a Shell Variable
      Section 31.13.  Automatic Setup When You Enter/Exit a Directory
   
      Chapter 32.  Regular Expressions (Pattern Matching)
      Section 32.1.  That's an Expression
      Section 32.2.  Don't Confuse Regular Expressions with Wildcards
      Section 32.3.  Understanding Expressions
      Section 32.4.  Using Metacharacters in Regular Expressions
      Section 32.5.  Regular Expressions: The Anchor Characters ^ and $
      Section 32.6.  Regular Expressions: Matching a Character with a Character Set
      Section 32.7.  Regular Expressions: Match Any Character with . (Dot)
      Section 32.8.  Regular Expressions: Specifying a Range of Characters with [...]
      Section 32.9.  Regular Expressions: Exceptions in a Character Set
      Section 32.10.  Regular Expressions: Repeating Character Sets with *
      Section 32.11.  Regular Expressions: Matching a Specific Number of Sets with \ { and \ }
      Section 32.12.  Regular Expressions: Matching Words with \ < and \ >
      Section 32.13.  Regular Expressions: Remembering Patterns with \ (, \ ), and \1
      Section 32.14.  Regular Expressions: Potential Problems
      Section 32.15.  Extended Regular Expressions
      Section 32.16.  Getting Regular Expressions Right
      Section 32.17.  Just What Does a Regular Expression Match?
      Section 32.18.  Limiting the Extent of a Match
      Section 32.19.  I Never Meta Character I Didn't Like
      Section 32.20.  Valid Metacharacters for Different Unix Programs
      Section 32.21.  Pattern Matching Quick Reference with Examples
   
      Chapter 33.  Wildcards
      Section 33.1.  File-Naming Wildcards
      Section 33.2.  Filename Wildcards in a Nutshell
      Section 33.3.  Who Handles Wildcards?
      Section 33.4.  What if a Wildcard Doesn't Match?
      Section 33.5.  Maybe You Shouldn't Use Wildcards in Pathnames
      Section 33.6.  Getting a List of Matching Files with grep -l
      Section 33.7.  Getting a List of Nonmatching Files
      Section 33.8.  nom: List Files That Don't Match a Wildcard
   
      Chapter 34.  The sed Stream Editor
      Section 34.1.  sed Sermon^H^H^H^H^H^HSummary
      Section 34.2.  Two Things You Must Know About sed
      Section 34.3.  Invoking sed
      Section 34.4.  Testing and Using a sed Script: checksed, runsed
      Section 34.5.  sed Addressing Basics
      Section 34.6.  Order of Commands in a Script
      Section 34.7.  One Thing at a Time
      Section 34.8.  Delimiting a Regular Expression
      Section 34.9.  Newlines in a sed Replacement
      Section 34.10.  Referencing the Search String in a Replacement
      Section 34.11.  Referencing Portions of a Search String
      Section 34.12.  Search and Replacement: One Match Among Many
      Section 34.13.  Transformations on Text
      Section 34.14.  Hold Space: The Set-Aside Buffer
      Section 34.15.  Transforming Part of a Line
      Section 34.16.  Making Edits Across Line Boundaries
      Section 34.17.  The Deliberate Scrivener
      Section 34.18.  Searching for Patterns Split Across Lines
      Section 34.19.  Multiline Delete
      Section 34.20.  Making Edits Everywhere Except...
      Section 34.21.  The sed Test Command
      Section 34.22.  Uses of the sed Quit Command
      Section 34.23.  Dangers of the sed Quit Command
      Section 34.24.  sed Newlines, Quoting, and Backslashes in a Shell Script
   
      Chapter 35.  Shell Programming for the Uninitiated
      Section 35.1.  Writing a Simple Shell Program
      Section 35.2.  Everyone Should Learn Some Shell Programming
      Section 35.3.  What Environment Variables Are Good For
      Section 35.4.  Parent-Child Relationships
      Section 35.5.  Predefined Environment Variables
      Section 35.6.  The PATH Environment Variable
      Section 35.7.  PATH and path
      Section 35.8.  The DISPLAY Environment Variable
      Section 35.9.  Shell Variables
      Section 35.10.  Test String Values with Bourne-Shell case
      Section 35.11.  Pattern Matching in case Statements
      Section 35.12.  Exit Status of Unix Processes
      Section 35.13.  Test Exit Status with the if Statement
      Section 35.14.  Testing Your Success
      Section 35.15.  Loops That Test Exit Status
      Section 35.16.  Set Exit Status of a Shell (Script)
      Section 35.17.  Trapping Exits Caused by Interrupts
      Section 35.18.  read: Reading from the Keyboard
      Section 35.19.  Shell Script "Wrappers" for awk, sed, etc.
      Section 35.20.  Handling Command-Line Arguments in Shell Scripts
      Section 35.21.  Handling Command-Line Arguments with a for Loop
      Section 35.22.  Handling Arguments with while and shift
      Section 35.23.  Loop Control: break and continue
      Section 35.24.  Standard Command-Line Parsing
      Section 35.25.  The Bourne Shell set Command
      Section 35.26.  test: Testing Files and Strings
      Section 35.27.  Picking a Name for a New Command
      Section 35.28.  Finding a Program Name and Giving Your Program Multiple Names
      Section 35.29.  Reading Files with the . and source Commands
      Section 35.30.  Using Shell Functions in Shell Scripts
   
      Chapter 36.  Shell Programming for the Initiated
      Section 36.1.  Beyond the Basics
      Section 36.2.  The Story of : # #!
      Section 36.3.  Don't Need a Shell for Your Script? Don't Use One
      Section 36.4.  Making #! Search the PATH
      Section 36.5.  The exec Command
      Section 36.6.  The Unappreciated Bourne Shell ":" Operator
      Section 36.7.  Parameter Substitution
      Section 36.8.  Save Disk Space and Programming: Multiple Names for a Program
      Section 36.9.  Finding the Last Command-Line Argument
      Section 36.10.  How to Unset All Command-Line Parameters
      Section 36.11.  Standard Input to a for Loop
      Section 36.12.  Making a for Loop with Multiple Variables
      Section 36.13.  Using basename and dirname
      Section 36.14.  A while Loop with Several Loop Control Commands
      Section 36.15.  Overview: Open Files and File Descriptors
      Section 36.16.  n>&m: Swap Standard Output and Standard Error
      Section 36.17.  A Shell Can Read a Script from Its Standard Input, but...
      Section 36.18.  Shell Scripts On-the-Fly from Standard Input
      Section 36.19.  Quoted hereis Document Terminators: sh Versus csh
      Section 36.20.  Turn Off echo for "Secret" Answers
      Section 36.21.  Quick Reference: expr
      Section 36.22.  Testing Characters in a String with expr
      Section 36.23.  Grabbing Parts of a String
      Section 36.24.  Nested Command Substitution
      Section 36.25.  Testing Two Strings with One case Statement
      Section 36.26.  Outputting Text to an X Window
      Section 36.27.  Shell Lockfile
   
      Chapter 37.  Shell Script Debugging and Gotchas
      Section 37.1.  Tips for Debugging Shell Scripts
      Section 37.2.  Bourne Shell Debugger Shows a Shell Variable
      Section 37.3.  Stop Syntax Errors in Numeric Tests
      Section 37.4.  Stop Syntax Errors in String Tests
      Section 37.5.  Quoting and Command-Line Parameters
      Section 37.6.  How Unix Keeps Time
      Section 37.7.  Copy What You Do with script
      Section 37.8.  Cleaning script Files
      Section 37.9.  Making an Arbitrary-Size File for Testing
   
   
    Part VII:  Extending and Managing Your Environment
      Chapter 38.  Backing Up Files
      Section 38.1.  What Is This "Backup" Thing?
      Section 38.2.  tar in a Nutshell
      Section 38.3.  Make Your Own Backups
      Section 38.4.  More Ways to Back Up
      Section 38.5.  How to Make Backups to a Local Device
      Section 38.6.  Restoring Files from Tape with tar
      Section 38.7.  Using tar to a Remote Tape Drive
      Section 38.8.  Using GNU tar with a Remote Tape Drive
      Section 38.9.  On-Demand Incremental Backups of a Project
      Section 38.10.  Using Wildcards with tar
      Section 38.11.  Avoid Absolute Paths with tar
      Section 38.12.  Getting tar's Arguments in the Right Order
      Section 38.13.  The cpio Tape Archiver
      Section 38.14.  Industrial Strength Backups
   
      Chapter 39.  Creating and Reading Archives
      Section 39.1.  Packing Up and Moving
      Section 39.2.  Using tar to Create and Unpack Archives
      Section 39.3.  GNU tar Sampler
      Section 39.4.  Managing and Sharing Files with RCS and CVS
      Section 39.5.  RCS Basics
      Section 39.6.  List RCS Revision Numbers with rcsrevs
      Section 39.7.  CVS Basics
      Section 39.8.  More CVS
   
      Chapter 40.  Software Installation
      Section 40.1.  /usr/bin and Other Software Directories
      Section 40.2.  The Challenges of Software Installation on Unix
      Section 40.3.  Which make?
      Section 40.4.  Simplifying the make Process
      Section 40.5.  Using Debian's dselect
      Section 40.6.  Installing Software with Debian's Apt-Get
      Section 40.7.  Interruptable gets with wget
      Section 40.8.  The curl Application and One-Step GNU-Darwin Auto-Installer for OS X
      Section 40.9.  Installation with FreeBSD Ports
      Section 40.10.  Installing with FreeBSD Packages
      Section 40.11.  Finding and Installing RPM Packaged Software
   
      Chapter 41.  Perl
      Section 41.1.  High-Octane Shell Scripting
      Section 41.2.  Checking your Perl Installation
      Section 41.3.  Compiling Perl from Scratch
      Section 41.4.  Perl Boot Camp, Part 1: Typical Script Anatomy
      Section 41.5.  Perl Boot Camp, Part 2: Variables and Data Types
      Section 41.6.  Perl Boot Camp, Part 3: Branching and Looping
      Section 41.7.  Perl Boot Camp, Part 4: Pattern Matching
      Section 41.8.  Perl Boot Camp, Part 5: Perl Knows Unix
      Section 41.9.  Perl Boot Camp, Part 6: Modules
      Section 41.10.  Perl Boot Camp, Part 7: perldoc
      Section 41.11.  CPAN
      Section 41.12.  Make Custom grep Commands (etc.) with Perl
      Section 41.13.  Perl and the Internet
   
      Chapter 42.  Python
      Section 42.1.  What Is Python?
      Section 42.2.  Installation and Distutils
      Section 42.3.  Python Basics
      Section 42.4.  Python and the Web
      Section 42.5.  urllib
      Section 42.6.  urllib2
      Section 42.7.  htmllib and HTMLParser
      Section 42.8.  cgi
      Section 42.9.  mod_python
      Section 42.10.  What About Perl?
   
   
    Part VIII:  Communication and Connectivity
      Chapter 43.  Redirecting Input and Output
      Section 43.1.  Using Standard Input and Output
      Section 43.2.  One Argument with a cat Isn't Enough
      Section 43.3.  Send (Only) Standard Error Down a Pipe
      Section 43.4.  Problems Piping to a Pager
      Section 43.5.  Redirection in C Shell: Capture Errors, Too?
      Section 43.6.  Safe I/O Redirection with noclobber
      Section 43.7.  The ( ) Subshell Operators
      Section 43.8.  Send Output Two or More Places
      Section 43.9.  How to tee Several Commands into One Place
      Section 43.10.  Redirecting Output to More Than One Place
      Section 43.11.  Named Pipes: FIFOs
      Section 43.12.  What Can You Do with an Empty File?
   
      Chapter 44.  Devices
      Section 44.1.  Quick Introduction to Hardware
      Section 44.2.  Reading Kernel Boot Output
      Section 44.3.  Basic Kernel Configuration
      Section 44.4.  Disk Partitioning
      Section 44.5.  Filesystem Types and /etc/fstab
      Section 44.6.  Mounting and Unmounting Removable Filesystems
      Section 44.7.  Loopback Mounts
      Section 44.8.  Network Devices — ifconfig
      Section 44.9.  Mounting Network Filesystems — NFS, SMBFS
      Section 44.10.  Win Is a Modem Not a Modem?
      Section 44.11.  Setting Up a Dialup PPP Session
      Section 44.12.  USB Configuration
      Section 44.13.  Dealing with Sound Cards and Other Annoying Hardware
      Section 44.14.  Decapitating Your Machine — Serial Consoles
   
      Chapter 45.  Printing
      Section 45.1.  Introduction to Printing
      Section 45.2.  Introduction to Printing on Unix
      Section 45.3.  Printer Control with lpc
      Section 45.4.  Using Different Printers
      Section 45.5.  Using Symbolic Links for Spooling
      Section 45.6.  Formatting Plain Text: pr
      Section 45.7.  Formatting Plain Text: enscript
      Section 45.8.  Printing Over a Network
      Section 45.9.  Printing Over Samba
      Section 45.10.  Introduction to Typesetting
      Section 45.11.  A Bit of Unix Typesetting History
      Section 45.12.  Typesetting Manpages: nroff
      Section 45.13.  Formatting Markup Languages — troff, LATEX, HTML, and So On
      Section 45.14.  Printing Languages — PostScript, PCL, DVI, PDF
      Section 45.15.  Converting Text Files into a Printing Language
      Section 45.16.  Converting Typeset Files into a Printing Language
      Section 45.17.  Converting Source Files Automagically Within the Spooler
      Section 45.18.  The Common Unix Printing System (CUPS)
      Section 45.19.  The Portable Bitmap Package
   
      Chapter 46.  Connectivity
      Section 46.1.  TCP/IP — IP Addresses and Ports
      Section 46.2.  /etc/services Is Your Friend
      Section 46.3.  Status and Troubleshooting
      Section 46.4.  Where, Oh Where Did That Packet Go?
      Section 46.5.  The Director of Operations: inetd
      Section 46.6.  Secure Shell (SSH)
      Section 46.7.  Configuring an Anonymous FTP Server
      Section 46.8.  Mail — SMTP, POP, and IMAP
      Section 46.9.  Domain Name Service (DNS)
      Section 46.10.  Dynamic Host Configuration Protocol (DHCP)
      Section 46.11.  Gateways and NAT
      Section 46.12.  Firewalls
      Section 46.13.  Gatewaying from a Personal LAN over a Modem
   
      Chapter 47.  Connecting to MS Windows
      Section 47.1.  Building Bridges
      Section 47.2.  Installing and Configuring Samba
      Section 47.3.  Securing Samba
      Section 47.4.  SWAT and GUI SMB Browsers
      Section 47.5.  Printing with Samba
      Section 47.6.  Connecting to SMB Shares from Unix
      Section 47.7.  Sharing Desktops with VNC
      Section 47.8.  Of Emulators and APIs
      Section 47.9.  Citrix: Making Windows Multiuser
   
   
    Part IX:  Security
      Chapter 48.  Security Basics
      Section 48.1.  Understanding Points of Vulnerability
      Section 48.2.  CERT Security Checklists
      Section 48.3.  Keeping Up with Security Alerts
      Section 48.4.  What We Mean by Buffer Overflow
      Section 48.5.  What We Mean by DoS
      Section 48.6.  Beware of Sluggish Performance
      Section 48.7.  Intruder Detection
      Section 48.8.  Importance of MOTD
      Section 48.9.  The Linux proc Filesystem
      Section 48.10.  Disabling inetd
      Section 48.11.  Disallow rlogin and rsh
      Section 48.12.  TCP Wrappers
   
      Chapter 49.  Root, Group, and User Management
      Section 49.1.  Unix User/Group Infrastructure
      Section 49.2.  When Does a User Become a User
      Section 49.3.  Forgetting the root Password
      Section 49.4.  Setting an Exact umask
      Section 49.5.  Group Permissions in a Directory with the setgid Bit
      Section 49.6.  Groups and Group Ownership
      Section 49.7.  Add Users to a Group to Deny Permissions
      Section 49.8.  Care and Feeding of SUID and SGID Scripts
      Section 49.9.  Substitute Identity with su
      Section 49.10.  Never Log In as root
      Section 49.11.  Providing Superpowers with sudo
      Section 49.12.  Enabling Root in Darwin
      Section 49.13.  Disable logins
   
      Chapter 50.  File Security, Ownership, and Sharing
      Section 50.1.  Introduction to File Ownership and Security
      Section 50.2.  Tutorial on File and Directory Permissions
      Section 50.3.  Who Will Own a New File?
      Section 50.4.  Protecting Files with the Sticky Bit
      Section 50.5.  Using chmod to Change File Permission
      Section 50.6.  The Handy chmod = Operator
      Section 50.7.  Protect Important Files: Make Them Unwritable
      Section 50.8.  cx, cw, c-w: Quick File Permission Changes
      Section 50.9.  A Loophole: Modifying Files Without Write Access
      Section 50.10.  A Directory That People Can Access but Can't List
      Section 50.11.  Juggling Permissions
      Section 50.12.  File Verification with md5sum
      Section 50.13.  Shell Scripts Must Be Readable and (Usually) Executable
      Section 50.14.  Why Can't You Change File Ownership?
      Section 50.15.  How to Change File Ownership Without chown
   
      Chapter 51.  SSH
      Section 51.1.  Enabling Remote Access on Mac OS X
      Section 51.2.  Protecting Access Through SSH
      Section 51.3.  Free SSH with OpenSSH
      Section 51.4.  SSH Problems and Solutions
      Section 51.5.  General and Authentication Problems
      Section 51.6.  Key and Agent Problems
      Section 51.7.  Server and Client Problems
   
   
    Glossary
 

Copyright

Copyright © 2003, 1997, 1993 O'Reilly & Associates, Inc. All rights reserved.

Printed in the United States of America.

Published by O'Reilly & Associates, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472.

O'Reilly & Associates books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://safari.oreilly.com). For more information contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com.

Nutshell Handbook, the Nutshell Handbook logo, and the O'Reilly logo are registered trademarks of O'Reilly & Associates, Inc. Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and O'Reilly & Associates, Inc. was aware of a trademark claim, the designations have been printed in caps or initial caps. The association between the image of a hand drill and the topic of Unix is a trademark of O'Reilly & Associates, Inc.

While every precaution has been taken in the preparation of this book, the publisher and the author assume no responsibility for errors or omissions, or for damages resulting from the use of the information contained herein.

How to Use This Book

This section refers to conventions used in the print book and explains how they were modified for the Safari version. The numbers in the following images correspond to the list below.

figs/sampage1safari.gif

  1. Summary Boxes. You'll see gray shaded summary boxes all through the book. (On Safari, the Summary Boxes are bordered sidebars with the title "Summary Box.") They summarize a topic and point you to articles with examples and further explanation.

  2. Article/Section Number. The first two digits indicate in which chapter the article resides; the last two digits indicate the number of the article within that chapter. The article number is used to refer to this article in all cross-references throughout the book. (On Safari, Article numbers correspond to Section numbers.)

  3. Cross-Reference in a Sentence. To find out more about the topic displayed in gray type (On Safari, this text is displayed in boldface.), see the article referenced by the number in parentheses immediately following the term.

  4. Cross-Reference in a Code Example. When a cross-reference occurs in an example, the cross-referenced text and related article number appear in the left margin. (On Safari, these cross-references appear above the code example.)

figs/sampage2safari.gif

  1. Globe If you don't want to type this script into a file yourself, or if we're talking about a C program that isn't shown, you can download it from the book's web site. See the Preface for full details on the content available for download. (Online version available at http://examples.oreilly.com/upt3)

  2. Screw. Be careful with this feature, or you might get screwed.

figs/sampage3safari.gif

  1. Pushpin. A note to keep in mind, or a helpful tip.

  2. Bomb. A bomb icon in the margin is a cross-reference to another article that explains the possible trouble you might encounter using the tip or script in the current article. (You can think of the bomb as a cross-referenced screw.) (On Safari, the Bomb appears above the paragraph it refers to.)

  3. Author's Initials. The author's full name is listed in the Preface.

Preface

A Book for Browsing

Technical books can be boring. But this is not an ordinary technical book! This book is like an almanac, a news magazine, and a hypertext database all rolled into one. Instead of trying to put the topics in perfect order — and expecting you to start at the beginning, then read through to the end — we hope that you'll browse. Start anywhere. Read what you want. (That's not quite true. First, you should read this Preface and the pages before it titled How to Use This Book. They will help you get the most out of your time with this book. Next, you may want to skim through the Unix fundamentals in Chapter 1. Then read what you want.)

Like an Almanac

The book is full of practical information. The main purpose isn't to teach you concepts (though they're in here). We've picked a lot of common problems, and we'll show you how to solve them.

Even though it's not designed to be read in strict order, the book is organized into chapters with related subject matter. If you want to find a specific subject, the table of contents is still a good place to start. In addition, several of the chapters contain shaded boxes. These are like small tables of contents on a particular subject, which might be even more limited than the scope of the chapter itself. Use the Index when you're trying to find a specific piece of information instead of a general group of articles about a topic.

Like a News Magazine

This book has short articles. Most show a problem and a solution — in one page or less. The articles are numbered within each chapter. Not all articles are "how-to" tips. Some articles have background information and concepts.

Like a Hypertext Database

Each article doesn't define all the concepts and words used. Instead, it gives you "links" that let you get more information if you need it. It's easy to get more information when you need it, but just skip the link if you don't. Unix Power Tools uses two kinds of links: those in a sentence and those in the margin. For examples, see the pages before this Preface titled How to Use This Book.

Programs on the Web

figs/www.gif

The book describes scripts and freely available programs that are available on the web site. An article about a program or file that's on the web site will have a globe icon next to it, like this. To get one of these programs, use our visit the web site:

http://www.oreilly.com/catalog/upt3/

About Unix Versions

There are lots of similarities between different versions of Unix. But it's almost impossible to write a book that covers every detail of every version correctly. Where we know there might be big differences or problems, we'll print a note in the text. Other places, we're forced to use "weasel words" like "Some versions of XXX will do...," without telling you exactly which versions. When you see those weasel words, what can you do?

Cross-References

If a cross-reference is to a single word — for example, a command name like this: tar — the cross reference is probably to an article that introduces that command. Cross references to phrases — like this: from a parent process to child process — are to an article that explains more about the concept or problem printed in gray.

Cross references don't necessarily give a complete list of all articles about a topic. We've tried to pick one or a few articles that give the best information. For a more complete list, use the Index.

What's New in the Third Edition

There have been some big changes in Unix since we wrote the first edition in the early 1990s, and there's been a surprising number of changes since the second edition, released in the late 1990s. Well over half of the articles have been revised, and we've expanded our coverage of the so-called small Unix flavors: Linux, FreeBSD, Mac OS X's Darwin, and so on.

A major change to this edition was the addition of several new topics relevant to today's connected world, including protecting your machine from attack and several articles related to Internet protocols. We've also added chapters with coverage of two of the more popular languages used in Unix: Perl and Python.

Typefaces and Other Conventions

Italic

Is used for the names of all Unix utilities, switches, directories, and filenames and to emphasize new terms and concepts when they are first introduced. It's also used in programs and examples to explain what's happening or what's been left out at the . . . marks.

Bold

Is used occasionally within text to make words easy to find — just like movie stars' names in the People section of your local newspaper.

Constant width

Is used for sample code fragments and examples. A reference in text to a word or item used in an example or code fragment is also shown in constant width font.

Constant width bold

Is used in examples to show commands or text that would be typed in literally by the user.

Constant width italic, bold italic

Are used in code fragments and examples to show variables for which a context-specific substitution should be made. (The variable filename, for example, would be replaced by some actual filename.)

function(n)

Is a reference to a manual page in Section n of the Unix programmer's manual. For example, getopt(3) refers to a page called getopt in Section 3.

%

Is the C-shell prompt.

$

Is the Bourne-shell prompt.

:-)

Is a "smiley face" that means "don't take this seriously." The idea started on Usenet and spread.

& . . .

Stands for text (usually computer output) that's been omitted for clarity or to save space.

CTRL

Starts a control character. To create CTRL-d, for example, hold down the "control" key and press the "d" key. Control characters are not case sensitive; "d" refers to both the upper- and lowercase letter. The notation ^D also means CTRL-d. Also, you'll sometimes see the key sequence in bold (for example, CTRL-d is used when we want to make it clear exactly what you should type.

Λ?

Is used in some examples to represent a space charaΛ?cter.

TAB

Is used in some examples to represent a TAB character.

The Authors

This book is the effort of several authors who have contributed to one edition or another since the first edition was released. Much of the material for the first and second edition came from three authors: Jerry Peek, Tim O'Reilly, and Mike Loukides. Their work is still present, though edited for current times. This third edition brought in four new authors, who edited the previous material, in addition to contributing new articles: Shelley Powers, Steven Champeon, Deborah Hooker, and Joe Johnston.

In addition, we also had several other authors contribute to all three editions — either people who originally posted a good tip to Usenet, authors of Nutshell Handbooks who let us take material from their books, or authors of software packages who let us take a few paragraphs from README files or other documentation.

Here's a list of authors and their initials:

AD
Angus Duggan
JIK
Jonathan I. Kamens
AF
AEleen Frisch
JM
Jeff Moskow
AN
Adrian Nye
JP
Jerry Peek
BA
Brandon S. Allbery
JJ
Joe Johnston
BB
Bruce Barnett
JS
John Strang
BR
Bill Rosenblatt
LK
Lar Kaufman
CT
Chris Torek
LL
Linda Lamb
DC
Debra Cameron
LM
Linda Mui
DD
Dale Dougherty
LW
Larry Wall
DG
Daniel Gilly
MAL
Maarten Litmaath
DH
Dave Hitz
ML
Mike Loukides
DJPH
Deborah Hooker
MS
Mike Stansbery
DL
Don Libes
RS
Randal Schwartz
DR
Daniel Romike
SP
Shelley Powers
DS
Daniel Smith
SG
Simson Garfinkel
EK
Eileen Kramer
SC
Steve Champeon
EP
Eric Pearce
SW
Sun Wu
GS
Gene Spafford
TC
Tom Christiansen
GU
Greg Ubben
TOR
Tim O'Reilly
HS
Henry Spencer
UM
Udi Manber

The Fine Print

Where we show an article from an author on Usenet, that person may not have thought of the idea originally, but may just be passing on something he or she learned. We attribute everything we can.

Request for Comments

Please tell us about any errors you find in this book or ways you think it could be improved. Our U.S. mail address, phone numbers, and electronic mail address are as follows:

O'Reilly & Associates, Inc.
1005 Gravenstein Highway North
Sebastopol, CA 95472
(800) 998-9938 (in the United States or Canada)
(707) 829-0515 (international/local)
(707) 829-0104 (fax)
bookquestions@oreilly.com (email)

Acknowledgments for the First Edition

This book wouldn't exist without Ron Petrusha. As the technical book buyer at Golden-Lee, a major book distributor, he discovered us soon after we started publishing Nutshell Handbooks in the mid-1980s. He was one of our early boosters, and we owed him one. So when he became an editor at Bantam (whose computer-book publishing operations were later acquired by Random House), we took him seriously when he started asking if there was anything we could do together.

At first nothing seemed to fit, since by that time we were doing pretty well as a publisher. We needed to find something that we could do together that might sell better than something that either company might do alone. Eventually, Ron suggested that we copublish a Unix book for Bantam's "Power Tools" series. This made sense for both of us. It gave Bantam access to our Unix expertise and reputation, and it gave us a chance to learn from Bantam about the mass market bookstore trade, as well as build on their successful "Power Tools" series.

But what would the book contain? There were two features of Bantam's original DOS Power Tools that we decided to emulate: its in-depth treatment of under-documented system features and its large collection of freely available scripts and utilities. However, we didn't want to write yet another book that duplicated the format of many others on the market, in which chapters on each of the major Unix tools follow one another in predictable succession. Our goal was certainly to provide essential technical information on Unix utilities, but more importantly, to show how the utilities can be combined and used to solve common (and uncommon) problems.

Similarly, because we were weary of the multitude of endless tutorial books about Unix utilities, we wanted to keep the tone brisk and to the point. The solution I came up with, a kind of "hypertext in print," actually owes a lot to Dale Dougherty. Dale has been working for several years on hypertext and online information delivery, and I was trying to get him to work with me on this project. So I tried to imagine the kind of book that he might like to create. (We have a kind of friendly rivalry, in which we try to leapfrog each other with ideas for new and better books!) Dale's involvement never went far beyond the early brainstorming stage, but the book still bears his indirect stamp. In some of the first books he wrote for me, he introduced the idea that sidebars — asides that illuminate and expand on the topic under discussion — could be used effectively in a technical book. Well, Dale, here's a book that's nothing but sidebars!

Dale, Mike Loukides, and I worked out the basic outline for the book in a week or two of brainstorming and mail exchanges. We thought we could throw it together pretty quickly by mining many of our existing books for the tips and tricks buried in them. Unfortunately, none of us was ever able to find enough time, and the book looked to be dying a slow death. (Mike was the only one who got any writing done.) Steve Talbott rescued the project by insisting that it was just too good an idea to let go; he recruited Jerry Peek, who had just joined the company as a writer and Unix consultant/tools developer for our production department.

Production lost the resulting tug of war, and Jerry plunged in. Jerry has forgotten more Unix tips and tricks than Mike, Dale, or I ever knew; he fleshed out our outline and spent a solid year writing and collecting the bulk of the book. I sat back in amazement and delight as Jerry made my ideas take shape. Finally, though, Jerry had had enough. The book was just too big, and he'd never signed on to do it all alone! (It was about 1,000 pages at that point, and only half done.) Jerry, Mike, and I spent a week locked up in our conference room, refining the outline, writing and cutting articles, and generally trying to make Jerry feel a little less like Sisyphus.

From that point on, Jerry continued to carry the ball, but not quite alone, with Mike and I playing "tag team," writing and editing to fill in gaps. I'm especially grateful to Mike for pitching in, since he had many other books to edit and this was supposed to be "my" project. I am continually amazed by the breadth of Mike's knowledge and his knack for putting important concepts in perspective.

Toward the end of the project, Linda Mui finished up another book she was working on and joined the project, documenting many of the freely available utilities that we'd planned to include but hadn't gotten around to writing up. Linda, you really saved us at the end!

Thanks also to all the other authors, who allowed us to use (and sometimes abuse!) their material. In particular, we're grateful to Bruce Barnett, who let us use so much of what he's written, even though we haven't yet published his book, and Chris Torek, who let us use many of the gems he's posted to the Net over the years. (Chris didn't keep copies of most of these articles; they were saved and sent in by Usenet readers, including Dan Duval, Kurt J. Lidl, and Jarkko Hietaniemi.)

Jonathan Kamens and Tom Christiansen not only contributed articles but read parts of the book with learned and critical eyes. They saved us from many a "power goof." If we'd been able to give them enough time to read the whole thing, we wouldn't have to issue the standard disclaimer that any errors that remain are our own. H. Milton Peek provided technical review and proofreading. Four sharp-eyed Usenet readers helped with debugging: Casper Dik of the University of Amsterdam, Byron Ratzikis of Network Appliance Corporation, Dave Barr of the Population Research Institute, and Duncan Sinclair.

In addition to all the acknowledged contributors, there are many unacknowledged ones — people who have posted questions or answers to the Net over the years and who have helped to build the rich texture of the Unix culture that we've tried to reflect in this book. Jerry also singles out one major contributor to his own mastery of Unix. He says: "Daniel Romike of Tektronix, Inc. (who wrote Section 28.5 and Section 30.8 in the early 1980s, by the way) led the first Unix workshop I attended. He took the time to answer a ton of questions as I taught myself Unix in the early 1980s. I'm sure some of the insights and neat tricks that I thought I've figured out myself actually came from Dan instead."

James Revell and Bryan Buus scoured "the Net" for useful and interesting free software that we weren't aware of. Bryan also compiled most of the software he collected so we could try it out and gradually winnow down the list.

Thanks also to all of the authors of the software packages we wrote about and included on the CD! Without their efforts, we wouldn't have had anything to write about; without their generosity in making their software free in the first place, we wouldn't be able to distribute hundreds of megabytes of software for the price of a book.

Jeff Moskow of Ready-to-Run Software solved the problem we had been putting off to the end: that of packaging up all the software for the original disk, porting it to the major Unix platforms, and making it easy to install. This was a much bigger job than we'd anticipated, and we could never have done it without Jeff and the RTR staff. We might have been able to distribute source code and binaries for a few platforms, but without their porting expertise, we could never have ported all these programs to every supported platform. Eric Pearce worked with RTR to pre-master the software for CD-ROM duplication, wrote the installation instructions, and made sure that everything came together at the end! (Eric, thanks for pitching in at the last minute. You were right that there were a lot of details that might fall through the cracks.)

Edie Freedman worked with us to design the format of the book — quite an achievement considering everything we wanted the format to do! She met the challenge of presenting thousands of inline cross-references without distracting the reader or creating a visual monstrosity. What she created is as attractive as it is useful — a real breakthrough in technical book design, and one that we plan to use again and again!

Lenny Muellner was given the frightful task of implementing all of our ideas in troff — no mean feat, and one that added to his store of grey hair.

Eileen Kramer was the copyeditor, proofreader, and critic who made sure that everything came together. For a thousand-plus page book with multiple authors, it's hard to imagine just how much work that was.

Ellie Cutler wrote the index; Chris Reilley created the illustrations. Additional administrative support was provided by Bonnie Hyland, Donna Woonteiler, and Jane Appleyard.

—Tim O'Reilly

Acknowledgments for the Second Edition

After teaching myself about Unix for the past 15 years, I'm off to graduate school in Computer Science. Frank Willison, O'Reilly's Editor-in-Chief, fit this project into the summer between leaving my position at ORA and starting school. Frank didn't just give me something to do in the summer: the royalties should help to pay for my coursework. (So, buy this book and support a student! ;-)) Gigi Estabrook edited this edition and fielded my zillions of questions along the way. Many thanks to Gigi, Frank, and ORA's Production staff. Clairemarie Fisher O'Leary and Nancy Wolfe Kotary shared the jobs of production editor and project manager. Madeleine Newell and Kismet McDonough-Chan provided production support. Sheryl Avruch, Nicole Gipson Arigo, and Danny Marcus provided quality control checks. Lenny Muellner provided extensive troff assistance and technical support. Chris Reilley created the technical illustrations.

When time was short, I got expert advice from Arnold Robbins, the maintainer of the GNU gawk utility, and coauthor of O'Reilly's sed & awk, Second Edition. He reviewed parts of the book and gave me thorough comments.

I'd also like to thank all the readers who took a moment to send us comments and corrections. I read every message, and the ideas in them made a big difference in this second edition. Three peoples' comments were extensive enough to mention specially. Ted Timar spotted problems that showed his deep knowledge of Unix. I'm glad he still found the book useful enough to read it — and to spot goofs in some of our hairier tips. Andrew T. Young sent two long email messages: one a few years ago and another after I contacted him. He caught plenty of techno-goofs and also sent fixes for them. Andy doesn't know just Unix: his background in English helped to sharpen a few rough spots in our folksy writing style. Finally, Greg Ubben sent a 15-page (!) email message that took me most of a week to work through. When I tracked him down, three years after writing his message, he was even more helpful. Greg wrote enough to make into a small book — and, in fact, agreed to write a few new articles, too. He's an expert in sed and regular expressions (and Unix) who taught me a lot in our month of email messages back and forth. I deeply appreciate all that he's given to this book's readers.

—Jerry Peek, jpeek@jpeek.com

Acknowledgments for the Third Edition

Though much of this book is new material or has been newly updated for changes in Unix, there is a core that remains from previous editions. The fact that this material has continued to be fresh, useful, and relevant through the years is a testament to the abilities — technical and writing — of the original authors. This includes Tim O'Reilly and Jerry Peek, among others previously mentioned, who contributed to past additions. We, the authors of this current edition, thank you. We had a number of terrific reviewers comment on this version of the text. We appreciate the work of Dave Carrano, Chris DiBona, Schuyler Erle, Jeff Kawski, Werner Klauser, Adam Langley, Arnold Robbins, Jaron Rubenstein, Kevin Schmidt, Jay Sekora, Joe Sloan, Nat Torkington, and Jay Ts.

In addition, I would like to thank those who contribute their time and efforts on Unix systems, particularly the open source versions of Unix such as FreeBSD, Linux, and now Darwin.

—Shelley Powers

I'd just like to thank you all for inviting me to contribute to a book that helped me learn Unix a long time ago. It's nice to be able to give something back, given how much the book helped me back in 1994 when I was just another Unix newbie.

—Steven Champeon

Thank you, Amy and Joel, for the input and review and just for putting up with me through it, and Jasper, for being my strength when I needed it.

—Deborah Hooker