Shell Scripting

Introduction to Shell Scripting

``Here's a hint. When you think your code to execute a shell function is just not working, never, repeat NEVER send it "/etc/reboot" just to see what happens.''
-- Elliott Evans

Suppose you often type the command

    find . -name file -print

and you'd rather type a simple command, say

    sfind file

This is the type of situation for which a simple shell script is useful.

Observations

Some information about shell scripts:

Shell scripts are simple text files created with an editor such as vi.
Shell scripts are marked as executeable eg: >chmod a+x sfind They should be located in your search path and ~/bin should be in your search path, and you'll need to run rehash if you use tcsh ( the default shell on Turing.)
* If you don't want to chmod the script, you can invoke the shell directly by typing sh (or whatever shell you are writing for) and the scriptname.
Arguments are passed from the command line and referenced.

#!/bin/sh

All Bourne Shell scripts should begin with the sequence

    #!/bin/sh

This is usually known as a "she-bang" notation. In this case, the "#" character isn't interpreted as a comment but instead the "#!" indicates that sh will be used to run the script. ( you may also want to use other more powerful shells such as bash, but remember that sh is the standard shell that you can be certain any system will have. You should under no circumstances use a csh variant, no matter how tempting. If you're curious why, look at Csh Programming Considered Harmful.)

All shell scripts should include a search path specification:

PATH=/usr/ucb:/usr/bin:/bin; export PATH

A PATH specification is recommended -- often times a script will fail for some people because they have a different or incomplete search path.

The Bourne Shell does not export environment variables to children unless explicitly instructed to do so by using the export command. A good script, just like any other program, should verify that the arguments supplied (if any) are correct.


    if [ $# -ne 3 ]; then		# If there aren't 3 arguments
         echo 1>&2 Usage: $0 19 Oct 91  # Prints "Usage: $0 19 Oct 91"
         exit 127			# exits with a non-zero error.  
	 				# I like 127 because its ASCII for esc,
					# but any posint works fine.
    fi

This script requires three arguments and gripes accordingly.

So, what now?
First, all Unix utilities should return an exit status.

    					# is the year out of range for me?
					# -lt = less than
					# -gt = greater than
					# 
    if [ $year -lt 1901  -o  $year -gt 2099 ]; then
         echo 1>&2 Year \"$year\" out of range
         exit 127
    fi

    etc...

    exit 0				# All done, exit ok

A non-zero exit status indicates an error condition of some sort while a zero exit status indicates things worked as expected.

The conditional construct is:

    if command; then
         command
    fi					# fi is the equivalent of an "end if"

For example,

    if tty -s; then			# If keyboard input is being accepted.
         echo Enter text and end with \^D
    fi

Standard input, output, and error are file descriptors 0, 1, and 2. Each has a particular role and should be used accordingly:

    					# is the year out of range for me?

    if [ $year -lt 1901  -o  $year -gt 2099 ]; then
         echo 1>&2 Year \"$year\" out of my range
         exit 127
    fi

    etc...

    # ok, you have the number of days since Jan 1, ...

    case `expr $days % 7` in
    
    0)
    	 echo Mon;;
    1)
         echo Tue;;

    etc...

Error messages should appear on stderr not on stdout! Output should appear on stdout. As for input/output dialogue:

    # A tiny piece of code from a script that demonstrates i/o	
    # give the fellow a chance to quit

    if tty -s ; then
         echo This will remove all files
         echo $Ok to procede? $c;  
	 read ans			# getting input
         case "$ans" in			# a switch statenment to process input
              n*|N*)
    echo File purge abandoned;		
    exit 0   ;;
         esac
         RM="rm -rfi"
    else
         RM="rm -rf"
    fi

Note: this code behaves differently if there's a user to communicate with (ie. if the standard input is a tty rather than a pipe, or file, or etc. See tty(1)).

Useful stuff to help you code.

For loop iteration

Substitute values for variables and perform tasks:

    for variable in word ...
    do
         command
    done				# "done" is used instead of "end"

# Backing up log files

    for i in `cat $LOGS`		# for all the log files
    do
            mv $i $i.$TODAY		# creates a copy of the current file 
	    				# in the format i.date
    done

    for variable in word ...; do command; done

Case

Switch to statements depending on pattern match

    case word in
    [ pattern [ | pattern ... ] )
         command ;; ] ...
    esac

For example:


# Returns the age of someone born in 1901 given any year ( doesn't check for input < 1901 )

    case "$year" in			# declares a switch on year

    [0-9][0-9])				# if a legal 2-digit year was inputted
            year=19${year}		# concat 19 with year
            years=`expr $year - 1901`	# gets the age
            ;;
    [0-9][0-9][0-9][0-9])		# if a legal 4-digit year was inputted
            years=`expr $year - 1901`	# gets the age
            ;;
    *)					# if the input is something not covered
    					# 2 cases
            echo 1>&2 Year \"$year\" out of range ...
            exit 127			# exits
            ;;
    esac

Conditional Execution

Test exit status of command and branch

    if command
    then
         command
    [ else
         command ]
    fi

Alternatively you may see:

    if command; then command; [ else command; ] fi

While/Until Iteration

Repeat task while command returns good exit status.

    {while | until} command
    do
         command
    done

For example:

    # for each argument mentioned, purge that directory

    while [ $# -ge 1 ]; do		# While a 1st argument exists
            rm -rf $1			# Purge that directory
            shift			# Get next argument
    done

Alternatively you may see:

    while command; do command; done

Variables

Variables are sequences of letters, digits, or underscores beginning with a letter or underscore. To get the contents of a variable you must prepend the name with a $.

Numeric variables (eg. like $1, etc.) are positional vari ables for argument communication.

Variable Assignment

Assign a value to a variable by variable=value. For example:

    PATH=/usr/ucb:/usr/bin:/bin; export PATH

    TODAY=`(set \`date\`; echo $1)`	# set is explained below

Exporting Variables

Variables are not exported to children unless explicitly marked. (that is, a variable must be exported to the program to be used by the program)

    # We MUST have a DISPLAY environment variable

    if [ "$DISPLAY" = "" ]; then	# If DISPLAY isn't already set	
      if tty -s ; then
        echo "DISPLAY (`hostname`:0.0)? \c";
        read DISPLAY			# Gets display
      fi
      if [ "$DISPLAY" = "" ]; then	# if DISPLAY is still not set
        DISPLAY=`hostname`:0.0		# Sets DISPLAY manually
      fi
      export DISPLAY			# exports DISPLAY
    fi

Likewise, for variables like the PRINTER which you want hon ored by lpr(1). From a user's .profile:

    PRINTER=PostScript; export PRINTER	# Lets your program use the PRINTER 
    					# variable.

Note: that the Cshell exports all environment variables.

Referencing Variables

Use $variable (or, if necessary, ${variable}) to reference the value.

    # Sets the user's path to their own /bin

    if [ "$USER" != "root" ]; then	
            PATH=$HOME/bin:$PATH	# Sets PATH to ~/bin
    else				# If they're root..	
            PATH=/bin:/usr/bin:$PATH	# Sets PATH to /bin and /usr/bin
    fi

The braces are required for concatenation, e.g.:

$p_01

The value of the variable p_01.

${p}_01

The value of the variable p with "_01" pasted onto the end.

Conditional Reference
```
${variable-word}
```
If the variable has been set, use it's value, else use word (which would be set to some default value).
```
POSTSCRIPT=${POSTSCRIPT-PostScript};	# POSTSCRIPT is either Postscript or its value
					# value if one has been set.
export POSTSCRIPT

${variable:-word}
```
If the variable has been set and is not null, use it's value, else use word.
These are useful constructions for honoring the user envi ronment. Ie. the user of the script can override variable assignments. Cf. programs like lpr(1) honor the PRINTER environment variable, you can do the same trick with your shell scripts.
```
${variable:?word}
```
If variable is set use it's value, else print out word and exit. Useful for bailing out.

Arguments

Command line arguments to shell scripts are positional vari ables:

$0, $1, ... 				# The  command and arguments. With $0 the command 
					# and the rest the arguments.

$#					# The number of arguments.

$*, $@					# All the arguments as a blank  separated  string.
					# Watch out for "$*" vs. "$@".

And, some commands:

shift				# Shift  the  postional  variables down one and
					# decrement number of arguments.

set [arg] [arg] ...		# Set the positional variables to the argument 
					# list.

Command line parsing uses shift:

    # parse argument list

    while [ $# -ge 1 ]; do
            case $1 in				# do stuff..
         process arguments...
            esac
            shift				# Again shift is being 
    done					# used to get the next arguement

A use of the set command:

    # figure out what day it is

    TODAY=`(set \`date\`; echo $1)` 		# the result of the date
						# command is the 1st argument,
						# given to the variable TODAY .

Special Variables

$$

Current process id. This is very useful for constructing temporary files.

         tmp=/tmp/cal0$$	# constructs a temporary file using the PID

$?

The exit status of the last command.

         $command
         # Run target file if no errors and ...

         if [ $? -eq 0 ]	# If exit status was zero (i.e.
         then
  etc...
         fi

Quotes/Special Characters
Special characters to terminate words:
```
      ; & ( ) | ^ < > newline space tab
```
These are for command sequences, background jobs, etc. To quote any of these use a backslash (\) or bracket with quote marks ("" or '').
Single Quotes
Within single quotes all characters are quoted -- including the backslash. The result is one word.
```
         grep :${gid}: /etc/group | awk -F: '{print $1}'
```
Double Quotes
Within double quotes you have variable subsitution (ie. the dollar sign is interpreted) but no file name generation (ie. * and ? are quoted). The result is one word.
```
         if [ ! "${parent}" ]; then
           parent=${people}/${group}/${user}
         fi
```
Back Quotes
Back quotes mean run the command and substitute the output.
```
         if [ "`echo -n`" = "-n" ]; then
          n=""
          c="\c"
         else
          n="-n"
          c=""
         fi
```
and
```
         TODAY=`(set \`date\`; echo $1)`
```

Functions

Functions are a powerful feature that aren't used often enough. Syntax is

    name ()
    {
         commands
    }

For example:


    # Purge a directory

    _purge()
    {
            # there had better be a directory

            if [ ! -d $1 ]; then
     echo $1: No such directory 1>&2
     return
            fi

         etc...
    }

Within a function the positional parmeters $0, $1, etc. are the arguments to the function (not the arguments to the script).

Within a function use return instead of exit.

Functions are good for encapsulations. You can pipe, redi rect input, etc. to functions. For example:

    # deal with a file, add people one at a time

    do_file()
    {
            while parse_one

            etc...
    }

    etc...

    # take standard input (or a specified file) and do it.

    if [ "$1" != "" ]; then
            cat $1 | do_file
    else
            do_file
    fi

One of the more common things you'll need to do is parse strings. Some tricks


    TIME=`date | cut -c12-19`

    TIME=`date | sed 's/.* .* .* \(.*\) .* .*/\1/'`

    TIME=`date | awk '{print $4}'`

    TIME=`set \`date\`; echo $4`

    TIME=`date | (read u v w x y z; echo $x)`

With some care, redefining the input field separators can help.

# Note: this is a complete working happy script :)

    #!/bin/sh
    # convert IP number to in-addr.arpa name

    name()
    {    set `IFS=".";echo $1`
         echo $4.$3.$2.$1.in-addr.arpa
    }

    if [ $# -ne 1 ]; then
         echo 1>&2 Usage: bynum IP-address
         exit 127
    fi

    add=`name $1`

    nslookup < < EOF | grep "$add" | sed 's/.*= //'
    set type=any
    $add
    EOF

Testing a script
The most powerful command for the shell programmer is test(1). ( refer to the man page for further information ).
```
    if test expression; then

         etc...
```
and (note the matching bracket argument)
```
    if [ expression ]; then

         etc...
```
Useful expressions are:
```
test { -w, -r, -x, -s, ... } filename
```
is file writeable, readable, executeable, empty, etc?
```
test n1 { -eq, -ne, -gt, ... } n2
```
are numbers equal, not equal, greater than, etc.?
```
test s1 { =, != } s2
```
Are strings the same or different?
```
test cond1 { -o, -a } cond2
```
Binary or; binary and; use ! for unary negation.
For example
```
    if [ $year -lt 1901  -o  $year -gt 2099 ]; then
         echo 1>&2 Year \"$year\" out of range
         exit 127
    fi
```
Learn this command inside out! It does a lot for you.
Debugging
The shell has a number of flags that make debugging easier:
sh -n command
Read the shell script but don't execute the commands. IE. check syntax.
sh -x command
Display commands and arguments as they're executed. In a lot of my shell scripts you'll see
```
    # Uncomment the next line for testing
    # set -x
```
Based on An Introduction to Shell Programing by:
Reg Quinton

Computing and Communications Services

The University of Western Ontario

London, Ontario N6A 5B7

Canada

Links:
Copyright (c) HMC Computer Science Department. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with the no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is included in the section entitled ``GNU Free Documentation License.''

HMC Computer Science Department
Contact Information

Last Modified Tuesday, 11-Oct-2005 17:08:29 PDT

Introduction to Shell Scripting

``Here's a hint. When you think your code to execute a shell function is just not working, never, repeat NEVER send it "/etc/reboot" just to see what happens.'' -- Elliott Evans

Observations

#!/bin/sh

Useful stuff to help you code.

Links:

``Here's a hint. When you think your code to execute a shell function is just not working, never, repeat NEVER send it "/etc/reboot" just to see what happens.''
-- Elliott Evans