Reference Cards to Shell Script(Over View): PART 6

Reference Cards:

Table 1. Special Shell Variables:

Variable	Meaning
$0	Filename of script
$1	Positional parameter #1
$2 - $9	Positional parameters #2 - #9
${10}	Positional parameter #10
$#	Number of positional parameters
"$*"	All the positional parameters (as a single word) *
"$@"	All the positional parameters (as separate strings)
${#*}	Number of positional parameters
${#@}	Number of positional parameters
$?	Return value
$$	Process ID (PID) of script
$-	Flags passed to script (using set)
$_	Last argument of previous command
$!	Process ID (PID) of last job run in background

* Must be quoted, otherwise it defaults to $@.

Table 2. TEST Operators: Binary Comparison:

Operator	Meaning	-----	Operator	Meaning
Arithmetic Comparison			String Comparison
-eq	Equal to		=	Equal to
			==	Equal to
-ne	Not equal to		!=	Not equal to
-lt	Less than		\<	Less than (ASCII) *
-le	Less than or equal to
-gt	Greater than		\>	Greater than (ASCII) *
-ge	Greater than or equal to
			-z	String is empty
			-n	String is not empty
Arithmetic Comparison	within double parentheses (( ... ))
>	Greater than
>=	Greater than or equal to
<	Less than
<=	Less than or equal to

* If within a double-bracket [[ ... ]] test construct, then no escape \ is needed.

Table 3. TEST Operators: Files:

Operator	Tests Whether	-----	Operator	Tests Whether
-e	File exists		-s	File is not zero size
-f	File is a regular file
-d	File is a directory		-r	File has read permission
-h	File is a symbolic link		-w	File has write permission
-L	File is a symbolic link		-x	File has execute permission
-b	File is a block device
-c	File is a character device		-g	sgid flag set
-p	File is a pipe		-u	suid flag set
-S	File is a socket		-k	"sticky bit" set
-t	File is associated with a terminal
-N	File modified since it was last read		F1 -nt F2	File F1 is newer than F2 *
-O	You own the file		F1 -ot F2	File F1 is older than F2 *
-G	Group id of file same as yours		F1 -ef F2	Files F1 and F2 are hard links to the same file *
!	NOT (inverts sense of above tests)

* Binary operator (requires two operands).

Table 4. Parameter Substitution and Expansion:

Expression	Meaning
${var}	Value of var (same as $var)
${var-DEFAULT}	If var not set, evaluate expression as $DEFAULT *
${var:-DEFAULT}	If var not set or is empty, evaluate expression as $DEFAULT *
${var=DEFAULT}	If var not set, evaluate expression as $DEFAULT *
${var:=DEFAULT}	If var not set, evaluate expression as $DEFAULT *
${var+OTHER}	If var set, evaluate expression as $OTHER, otherwise as null string
${var:+OTHER}	If var set, evaluate expression as $OTHER, otherwise as null string
${var?ERR_MSG}	If var not set, print $ERR_MSG and abort script with an exit status of 1.*
${var:?ERR_MSG}	If var not set, print $ERR_MSG and abort script with an exit status of 1.*
${!varprefix*}	Matches all previously declared variables beginning with varprefix
${!varprefix@}	Matches all previously declared variables beginning with varprefix

* If var is set, evaluate the expression as $var with no side-effects.

Table 5. String Operations:

Expression	Meaning
${#string}	Length of $string
${string:position}	Extract substring from $string at $position
${string:position:length}	Extract $length characters substring from $string at $position [zero-indexed, first character is at position 0]
${string#substring}	Strip shortest match of $substring from front of $string
${string##substring}	Strip longest match of $substring from front of $string
${string%substring}	Strip shortest match of $substring from back of $string
${string%%substring}	Strip longest match of $substring from back of $string
${string/substring/replacement}	Replace first match of $substring with $replacement
${string//substring/replacement}	Replace all matches of $substring with $replacement
${string/#substring/replacement}	If $substring matches front end of $string, substitute $replacement for $substring
${string/%substring/replacement}	If $substring matches back end of $string, substitute $replacement for $substring
expr match "$string" '$substring'	Length of matching $substring* at beginning of $string
expr "$string" : '$substring'	Length of matching $substring* at beginning of $string
expr index "$string" $substring	Numerical position in $string of first character in $substring* that matches [0 if no match, first character counts as position 1]
expr substr $string $position $length	Extract $length characters from $string starting at $position [0 if no match, first character counts as position 1]
expr match "$string" '\($substring\)'	Extract $substring*, searching from beginning of $string
expr "$string" : '\($substring\)'	Extract $substring* , searching from beginning of $string
expr match "$string" '.*\($substring\)'	Extract $substring*, searching from end of $string
expr "$string" : '.*\($substring\)'	Extract $substring*, searching from end of $string

* Where $substring is a Regular Expression.

Table 6. Miscellaneous Constructs:

Expression	Interpretation
Brackets
if [ CONDITION ]	Test construct
if [[ CONDITION ]]	Extended test construct
Array[1]=element1	Array initialization
[a-z]	Range of characters within a Regular Expression
Curly Brackets
${variable}	Parameter substitution
${!variable}	Indirect variable reference
{ command1; command2; . . . commandN; }	Block of code
{string1,string2,string3,...}	Brace expansion
{a..z}	Extended brace expansion
{}	Text replacement, after find and xargs
Parentheses
( command1; command2 )	Command group executed within a subshell
Array=(element1 element2 element3)	Array initialization
result=$(COMMAND)	Command substitution, new style
>(COMMAND)	Process substitution
<(COMMAND)	Process substitution
Double Parentheses
(( var = 78 ))	Integer arithmetic
var=$(( 20 + 5 ))	Integer arithmetic, with variable assignment
(( var++ ))	C-style variable increment
(( var-- ))	C-style variable decrement
(( var0 = var1<98?9:21 ))	C-style ternary operation
Quoting
"$variable"	"Weak" quoting
'string'	'Strong' quoting
Back Quotes
result=`COMMAND`	Command substitution, classic style

This is a very brief introduction to the sed and awk text processing utilities.

sed: a non-interactive text file editor
awk: a field-oriented pattern processing language with a C-style syntax

Operator Precedence
Table 7. Operator Precedence

Operator	Meaning	Comments
		HIGHEST PRECEDENCE
var++ var--	post-increment, post-decrement	C-style operators
++var --var	pre-increment, pre-decrement
! ~	negation	logical / bitwise, inverts sense of following operator
**	exponentiation	arithmetic operation
* / %	multiplication, division, modulo	arithmetic operation
+ -	addition, subtraction	arithmetic operation
<< >>	left, right shift	bitwise
-z -n	unary comparison	string is/is-not null
-e -f -t -x, etc.	unary comparison	file-test
< -lt > -gt <= -le >= -ge	compound comparison	string and integer
-nt -ot -ef	compound comparison	file-test
== -eq != -ne	equality / inequality	test operators, string and integer
&	AND	bitwise
^	XOR	exclusive OR, bitwise
|	OR	bitwise
&& -a	AND	logical, compound comparison
|| -o	OR	logical, compound comparison
?:	trinary operator	C-style
=	assignment	(do not confuse with equality test)
*= /= %= += -= <<= >>= &=	combination assignment	times-equal, divide-equal, mod-equal, etc.
,	comma	links a sequence of operations
		LOWEST PRECEDENCE

Table 8. Job identifiers

Notation	Meaning
%N	Job number [N]
%S	Invocation (command-line) of job begins with string S
%?S	Invocation (command-line) of job contains within it string S
%%	"current" job (last job stopped in foreground or started in background)
%+	"current" job (last job stopped in foreground or started in background)
%-	Last job
$!	Last background process

Exit Codes With Special Meanings

Table . Reserved Exit Codes:

Exit Code Number	Meaning	Example	Comments
1	Catchall for general errors	let "var1 = 1/0"	Miscellaneous errors, such as "divide by zero" and other impermissible operations
2	Misuse of shell builtins (according to Bash documentation)	empty_function() {}	Missing keyword or command
126	Command invoked cannot execute	/dev/null	Permission problem or command is not an executable
127	"command not found"	illegal_command	Possible problem with $PATH or a typo
128	Invalid argument to exit	exit 3.14159	exit takes only integer args in the range 0 - 255 (see first footnote)
128+n	Fatal error signal "n"	kill -9 $PPID of script	$? returns 137 (128 + 9)
130	Script terminated by Control-C	Ctl-C	Control-C is fatal error signal 2, (130 = 128 + 2, see above)
255*	Exit status out of range	exit -1	exit takes only integer args in the range 0 - 255

Notes:

1.	Out of range exit values can result in unexpected exit codes. An exit value greater than 255 returns an exit code modulo 256. For example, exit 3809 gives an exit code of 225 (3809 % 256 = 225).
2.	An update of /usr/include/sysexits.h allocates previously unused exit codes from 64 - 78. It may be anticipated that the range of unallotted exit codes will be further restricted in the future.

A Detailed Introduction to I/O and I/O Redirection

A command expects the first three file descriptors to be available. 

The first, fd 0 (standard input, stdin), is for reading.
The other two (fd 1, stdout and fd 2,stderr) are for writing.

There is a stdin, stdout, and a stderr associated with each command. ls 2>&1 means temporarily connecting the stderr of the ls command to the same"resource" as the shell's stdout.


[sankar@new-host ~]$ cat /etc/passwd >&-
cat: standard output: Bad file descriptor

Standard Command-Line Options:

The two most widely-accepted options are:

-h or (--help) ==> Help: Give usage message and exit.

-v or (--version) ==> Version: Show program version and exit.

-r or -R (--recursive) ==> Recursive: Operate recursively (down directory tree).

-v or (--verbose) ==> Verbose: output additional information to stdout or stderr.

Important Files

startup files

These files contain the aliases and environmental variables made available to Bash running as a user shell and to all Bash scripts invoked after system initialization.

/etc/profile

Systemwide defaults, mostly setting the environment (all Bourne-type shells, not just Bash)

/etc/bashrc

systemwide functions and aliases for Bash

$HOME/.bash_profile

user-specific Bash environmental default settings, found in each user's home directory (the local counterpart to /etc/profile)

$HOME/.bashrc

user-specific Bash init file, found in each user's home directory (the local counterpart to /etc/bashrc). Only interactive shells and user scripts read this file. 

logout file

$HOME/.bash_logout

user-specific instruction file, found in each user's home directory. Upon exit from a login (Bash) shell, the commands in this file execute.

data files

/etc/passwd

A listing of all the user accounts on the system, their identities, their home directories, the groups they belong to, and their default shell. Note that the user passwords are not stored in this file, 

But in /etc/shadow in encrypted form.

system configuration files

/etc/sysconfig/hwconf

Listing and description of attached hardware devices. This information is in text form and can be extracted and parsed.

bash$ grep -A 5 AUDIO /etc/sysconfig/hwconf       
 class: AUDIO
 bus: PCI
 detached: 0
 driver: snd-intel8x0
 desc: "Intel Corporation 82801CA/CAM AC'97 Audio Controller"
 vendorId: 8086

Important System Directories

Sysadmins and anyone else writing administrative scripts should be intimately familiar with the following system directories.

• /bin
  Binaries (executables). Basic system programs and utilities (such as bash).
  
• /usr/bin 
  More system binaries.
  
• /usr/local/bin
  Miscellaneous binaries local to the particular machine.
  
• /sbin
  System binaries. Basic system administrative programs and utilities (such as fsck).
  
• /usr/sbin
  More system administrative programs and utilities.
  
• /etc
  Et cetera. Systemwide configuration scripts.
  Of particular interest are the /etc/fstab (filesystem table), /etc/mtab (mounted filesystem table), and the /etc/inittab files.
  
• /etc/rc.d
  Boot scripts, on Red Hat and derivative distributions of Linux.
  
• /usr/share/doc
  Documentation for installed packages.
  
• /usr/man
  The systemwide manpages.
  
• /dev
  Device directory. Entries (but not mount points) for physical and virtual devices.
  
• /proc
  Process directory. Contains information and statistics about running processes and kernel parameters.
  
• /sys
  Systemwide device directory. Contains information and statistics about device and device names. This is newly added to Linux with the 2.6.X kernels.
  
• /mnt
  Mount. Directory for mounting hard drive partitions, such as /mnt/dos, and physical devices. In newer Linux distros, the /media directory has taken over as the preferred mount point for I/O devices.
  
• /media
  In newer Linux distros, the preferred mount point for I/O devices, such as CD/DVD drives or USB flash drives.
  
• /var
  Variable (changeable) system files. This is a catchall "scratchpad" directory for data generated while a Linux/UNIX machine is running.
  
• /var/log
  Systemwide log files.
  
• /var/spool/mail
  User mail spool.
  
• /lib
  Systemwide library files.
  
• /usr/lib
  More systemwide library files.
  
• /tmp
  System temporary files.
  
• /boot
  System boot directory. The kernel, module links, system map, and boot manager reside here.
  
  
  

  
  

  History Commands

  
  

  Bash history commands:
  
  
  

  1. history
  2. fc
     
     

  
  

  bash$ history 1 mount /mnt/cdrom 2 cd /mnt/cdrom 3 ls ...

  
  Internal variables associated with Bash history commands:
  
  
  

  1. $HISTCMD
  2. $HISTCONTROL
  3. $HISTIGNORE
  4. $HISTFILE
  5. $HISTFILESIZE
  6. $HISTSIZE
  7. $HISTTIMEFORMAT (Bash, ver. 3.0 or later)
  8. !!
  9. !$
  10. !#
  11. !N
  12. !-N
  13. !STRING
  14. !?STRING?
  15. ^STRING^string^
      

  
  Unfortunately, the Bash history tools find no use in scripting.
  
  

  #!/bin/bash # history.sh # A (vain) attempt to use the 'history' command in a script. history # No output. var=$(history); echo "$var" # $var is empty. # History commands disabled within a script. bash$ ./history.sh (no output)

  
  
  Converting DOS Batch Files to Shell Scripts
  

  
  

  Table 1. Batch file keywords / variables / operators, and their shell equivalents
  
  
  

  Batch File Operator	Shell Script Equivalent	Meaning
  %	$	command-line parameter prefix
  /	-	command option flag
  \	/	directory path separator
  ==	=	(equal-to) string comparison test
  !==!	!=	(not equal-to) string comparison test
  |	|	pipe
  @	set +v	do not echo current command
  *	*	filename "wild card"
  >	>	file redirection (overwrite)
  >>	>>	file redirection (append)
  <	<	redirect stdin
  %VAR%	$VAR	environmental variable
  REM	#	comment
  NOT	!	negate following test
  NUL	/dev/null	"black hole" for burying command output
  ECHO	echo	echo (many more option in Bash)
  ECHO.	echo	echo blank line
  ECHO OFF	set +v	do not echo command(s) following
  FOR %%VAR IN (LIST) DO	for var in [list]; do	"for" loop
  :LABEL	none (unnecessary)	label
  GOTO	none (use a function)	jump to another location in the script
  PAUSE	sleep	pause or wait an interval
  CHOICE	case or select	menu choice
  IF	if	if-test
  IF EXIST FILENAME	if [ -e filename ]	test if file exists
  IF !%N==!	if [ -z "$N" ]	if replaceable parameter "N" not present
  CALL	source or . (dot operator)	"include" another script
  COMMAND /C	source or . (dot operator)	"include" another script (same as CALL)
  SET	export	set an environmental variable
  SHIFT	shift	left shift command-line argument list
  SGN	-lt or -gt	sign (of integer)
  ERRORLEVEL	$?	exit status
  CON	stdin	"console" (stdin)
  PRN	/dev/lp0	(generic) printer device
  LPT1	/dev/lp0	first printer device
  COM1	/dev/ttyS0	first serial port

  Batch files usually contain DOS commands. These must be translated into their UNIX equivalents in order to convert a batch file into a shell script.

  
  

  Table 2. DOS commands and their UNIX equivalents
  
  
  

  DOS Command	UNIX Equivalent	Effect
  ASSIGN	ln	link file or directory
  ATTRIB	chmod	change file permissions
  CD	cd	change directory
  CHDIR	cd	change directory
  CLS	clear	clear screen
  COMP	diff, comm, cmp	file compare
  COPY	cp	file copy
  Ctl-C	Ctl-C	break (signal)
  Ctl-Z	Ctl-D	EOF (end-of-file)
  DEL	rm	delete file(s)
  DELTREE	rm -rf	delete directory recursively
  DIR	ls -l	directory listing
  ERASE	rm	delete file(s)
  EXIT	exit	exit current process
  FC	comm, cmp	file compare
  FIND	grep	find strings in files
  MD	mkdir	make directory
  MKDIR	mkdir	make directory
  MORE	more	text file paging filter
  MOVE	mv	move
  PATH	$PATH	path to executables
  REN	mv	rename (move)
  RENAME	mv	rename (move)
  RD	rmdir	remove directory
  RMDIR	rmdir	remove directory
  SORT	sort	sort file
  TIME	date	display system time
  TYPE	cat	output file to stdout
  XCOPY	cp	(extended) file copy

•