LINUX & BASH

# Linux Basics

• Shell :
Commands Description
!! or !$ execute previous command
![x] run xth command
!-[x] run last xth command
which [command] print full path of (shell) commands
whereis [command] locate binary, source, and manual-page files for command
clear clear terminal screen
exit exit shell
exit [x] exit shell with exit status x
history shell history
history [x] last x shell history
sleep [x] sleep (pause) terminal for x sec
reset initialize the terminal (resets terminal mode)
tty print file name of terminal connected to standard input
• System and User :
Commands Description
su switch user - allow switch to another user
sudo superuser do - allow executing commands with elevated/root privileges
df filesystem disk space usage
du estimate file space usage of set of files
free print free and used memory in system
arch print CPU architecture
dmesg print or control kernel ring buffer (logs)
lsblk list block devices information
lspci list PCI devices (buses) information
lsusb list USB devices (buses) information
hostid print numeric identifier for current host
hostname print hostname of current system
uname print system information
uptime show uptime (how long the system has been running)
halt halts system by stopping all processes but does not power off
poweroff poweroff system
shutdown shutdown system
reboot reboot system
last reboot print system reboot history
id print user and group id
users print all logged in users on system
w print who is logged-in and what running
who print logged in user
whoami print current user
passwd change user password
ps snapshot of current processes
top view running system in real-time interactively
kill [PID] kill/terminate process
• Networking :
Commands Description
wget [URL] non-interactive network retriever, used for downloading files from the web
curl [URL] transfers data from or to a server using URLs
ping [IP] Packet Internet Groper (ping) sends ICMP ECHO_REQUEST to network hosts to test the reachability of a host
traceroute [IP] print the route packets trace to network host
whois [IP] client for the whois directory service
host [DOMAIN] DNS lookup utility converts names to IP addresses and vice versa
dig [IP] Domain Information Groper (dig) retrieves information about DNS Name Servers
ip show / manipulate routing, network devices, interfaces and tunnels (replaces deprecated: ifconfig, route, arp)
arp show / manipulate Address Resolution Protocol (ARP) cache
ifconfig Interface Configuration (ifconfig) configures network interfaces (deprecated)
route show / manipulate the IP routing table (deprecated)
netstat print network connections, routing tables, interface statistics, masquerade connections, multicast memberships, etc. (deprecated)
nslookup [IP] Name Server Lookup (nsookup) queries DNS Servers interactively (deprecated)
• File Management :
Commands Description
ls list current directory contents (except hidden)
ls [dir] list directory contents
ls -a list directory contents include hidden
ls -l list directory contents in detailed format
pwd print current/working directory (full path)
cd [dir] change directory
cd - change to last directory
cd .. change to one level above current directory
cd or cd ~ change to home directory
mkdir [dir] create directory
mkdir -p [dir1]/[dir2] create directory inside non-existing parent directory
rmdir [dir] remove empty directory
cp [source] [destin] copy files and directories from source to destination
cp -r [dir1] [dir2] copy directories & its content recursively
mv [source/old] [destin/new] rename or move file and directory
rm [file] remove files
rm -r [file] remove files/directories & its content recursively
cat [file] concatenate and print files on standard output
tac [file] concatenate and print files in reverse
head [file] print first 10 lines of file
head -[x] [file] print first x lines of file
tail [file] show last 10 lines of file
tail -[x] [file] show last x lines of file
more [file] paging through content one screenful at time (it is primitive, use less instead)
less [file] paging through content one screenful at time with enhancements
nl [file] number lines of files
touch [file] update modification and access time of file (create if not exist)
touch -c [file] update modification and access time of file but do not create new file
sort [file] sort lines of text files (default alphabetical order)
uniq [file] filter adjacent matching (duplicate) lines from files
wc [file] print line, word and byte counts of files
wc -l [file] print lines count (l-line, m-char, c-byte, w-word)
tree [dir] list contents of directories in trees format
file [file] print file type
stat [file] display file or file-system status
locate [file] search for entries (files) in locate database
ln [file] [link] create hard link to file
ln -s [file] [link] create symbolic/soft link to file

# Bash Scripts

A Bash script is a plain text file that contains a series of commands that are executed in sequence. Scripts have .sh extension (for example: script.sh). Shell scripts begin with the shebang (#!/bin/sh), which tells the system which shell interpreter to use to run the script.

Ex :
#!/bin/bash

mkdir Docs
cp doc1.txt Docs/
rm doc1.txt
clear
Executing Scripts :

Note : The script must have executable permission.

./[script]
sh [script]
bash [script]

# Output each command execution :
bash -x [script]
Ex :
./script.sh
bash script.sh

# Variables

Declaring :
greet="Hello World"
num=17
Access :
echo $greet
echo "Number is ${num}"
Read-Only Variable :
readonly COUNT=1
Declare Integer :
declare -i num=10
Remove Variable :
unset num

# Strings

Declaring :
greet="Hello World"
Access :
echo $greet
Length of string :
echo ${#greet}
String Replace :
greet="Hello World"
echo "Before: $greet"
greet=${greet/World/Universe}
echo "After: $greet"

# Arrays

Declaring Indexed Array :
declare -a fruits=("apple" "mango" "banana" "orange")
Declaring Associative Array :
declare -A india["capital"]="New Delhi"

declare -A capital=(["India"]="New Delhi", ["Nepal"]="Kathmandu", ["Bhutan"]="Thimphu")
Indexing :
fruits[0]="grapes"
Access Element :
echo ${fruits[0]}
Print All Elements :
echo ${fruits[@]}
Print Key of Associative Array :
echo ${!india[@]}
Length of Array :
echo ${#fruits[@]}
Input :
read -a fruits

# Input

Input (STDIN) :
read [var]
Prompt Input :
read -p "Username : " [var]
Secure Prompt Input :
read -sp "Password : " [var]
Input NChar :
read -n [x] [var]
Array Input :
read -a [array]

# Output

Print (STDOUT) :

The echo prints output followed by a new line character.

echo "Hello World"

msg="Hello World"
echo $msg
echo "Hii, ${msg}"
Enable Backslash Escapes :
echo -e "\n Hello World"
Special Char in Output :
echo '$Hello World'
Print (STDOUT) :

The printf prints output in the specified format; it does not append an implied newline.

printf "\n%s\n\n" "Hello World"

# Operators

• Arithmetic Operator :
Operator Operation
+ addition
- subtraction
* multiplication
/ division
% modulus
++ increment
-- decrement

Note : Arithmetic operations should be in double braces: count=$((count+1))

• Logical Operator :
Operator Operation
![expr] not
[expr] || [expr] or [expr] -o [expr] or
[expr] && [expr] or [expr] -a [expr] and
• Relational Operator :

(Numerical Comparison)

Operator Operation
[expr] -eq [expr] equal
[expr] -ne [expr] not equal
[expr] -gt [expr] greater than
[expr] -lt [expr] less than
[expr] -ge [expr] greater than or equal
[expr] -le [expr] less than or equal
Ex :
if [ 80 -gt 40 ]
then
    echo "Greater"
fi
• Relational Operator :

(String Comparison)

Operator Operation
[string] = [string] equal
[string] != [string] not equal
[string] == [string] equal for pattern matching
-n [string] true if length of string is nonzero
-z [string] true if length of string is zero
Ex :
if [ "Hello" = "Hi" ]
then
    echo "Matched"
else
    echo "Not Matched"
fi
• Lexicographic Comparison :
Operator Operation
[string] > [string] true if left string is greater than right in alphabetical order
[string] < [string] true if right string is greater than left string in alphabetical order
• File Testing Operator :
Operator Operation
-e [file] true if file exists
-f [file] true if file exists and is regular file
-d [file] true if file exists and is directory
-c [file] true if file exists and is character file
-b [file] true if file exists and is block file
-r [file] true if file exists and read permission is granted
-w [file] true if file exists and write permission is granted
-x [file] true if file exists and execute permission is granted
-s [file] true if size of file is greater than zero
Ex :
if [ -f text.txt ]
then
    echo "text.txt exists and it is regular file."
fi

Note :

  • Expressions can be grouped with ( [expression] )
  • Conditional expressions must be in square braces []
  • Space must be given between the operands and the operator.

# Escape Sequence

Escape Description
\\ backslash
\a alert (BEL)
\b backspace
\c suppress trailing newline
\f form feed
\n new line
\r carriage return
\t horizontal tab
\v vertical tab

# Variable Substitution

Var Description
${x} substitute value of variable x
${x:+y} if x is set, y is substituted for x. (value of x does not change)
${x:-y} if x is null or unset, y is substituted for x. (value of x does not change)
${x:=y} if x is null or unset, x is set to value of y.
${x:?y} if x is null or unset, y is printed to standard error.

# Command Substitution

Command substitution allows the output of a command to replace the command itself. Using a backquote is old-style; instead, the second form is recommended.

• Back-Quote :
Syntax :
`[command]`
Ex :
# Example 1:
DATE=`date`
echo "Date is $DATE"

# Example 2:
USERS=`who | wc -l`
echo "Logged in user are $USERS"
• Dollor Sign & Parenthesis :
Syntax :
$([command])
Ex :
# Example 1:
echo "Today is $(date)"

# Example 2:
for i in $(cat file.txt)
do
    echo "$i"
done

# Decision Making

• If Statement :
Syntax :
if [ condition ]
then
    #Statement
fi

# with OR
if [ condition ] || [ condition ]
then
    #Statement
fi

# with AND
if [ condition ] && [ condition ]
then
    #Statement
fi
Ex :
x="Y"
if [ "$x" = "y" ] || [ "$x" = "Y" ]
then
    echo "YES"
fi
• If-Else Statement :
Syntax :
if [ condition ]
then
    #Statement
else
    #Statement
fi
Ex :
name="admin"
if [ $name = "admin" ]
then
    echo "Hello BOSS"
else
    echo "Who Are You?"
fi
• Elif Statements :
Syntax :
If [ condition ]
then
    #Statement
elif [ condition ]
then
    #Statement
else
    #Statement
fi
Ex :
# Example 1:
x=10
y=20
if [ $x -lt $y ]
then
    echo "X is less than Y"
elif [ $x -gt $y ]
then
    echo "X is greater than Y"
elif [ $x -eq $y ]
then
    echo "X is equal to Y"
fi

# Example 2:
a=3
b=3
c=3
if [ $a -eq $b ] && [ $a -eq $c ]
then
    echo "Equilateral"
elif [ $a -eq $b ] || [ $b -eq $c ] || [ $a -eq $c ]
then
    echo "Isosceles"
else
    echo "Scalene"
fi
• Switch Statement :
Syntax :
case [var] in
case-1)
    #Statement
    ;;
case-2)
    #Statement
    ;;
case-3)
    #Statement
    ;;
*)
    #Default Statement 
    ;;
esac
Ex :
drink="tea"
case $drink in
tea|cofee|water)
    echo "Go to canteen"
    ;;
orange|apple)
    echo "Go to fruit market"
    ;;
none)
    echo "Okay"
    ;;
*)
    echo "ERROR: Invalid Selection"
    ;;
esac

# Loops

• While Loop :
Syntax :
while [ condition ]
do
    #Statement
done
Ex :
# Example 1: print "hello" 5 times
count=0
while [ $count -lt 5 ]
do
    echo "Hello World"
    count=$((count+1))
done

# OR
count=0
while [ $count -lt 5 ]
do
    echo "Hello World"
    ((count++))
done

# Example 2: print file content by line
cat file.txt | while read line
do
    echo $line
done

# Example 3: read while loop
while read line
do
    echo $line
done < file.txt

# Example 4: infinite loop
while true
do
    echo "Infinite Loop..."
done

# Example 5: infinite loop
while :
do
    echo "Infinite Loop..."
done

# Example 6: factorial - pass number as command argument
counter=$1
factorial=1
while [ $counter -gt 0 ]
do
    factorial=$(( $factorial * $counter ))
    counter=$(( $counter - 1 ))
done
echo $factorial
• For Loop :
Syntax :
for [var] in [expression]
do
    #Statement
done
2nd Syntax :
for (( expr; expr; expr ))
do
    #Statement
done
Ex :
# Example 1: input array & print
read -a array
count=1
for i in ${array[@]}
do
    echo "Element $count : $i"
    ((count++))
done

# Example 2:
for i in 1 2 3 4 5
do
    echo $i
done

# Example 3:
for i in {1..25}
do
    echo $i
done

# Example 4: increment by 2 from 0 to 10
for i in {0..10..2}
do
    echo $i
done

# OR using seq - increment by 2 from 0 to 10
for i in $(seq 0 2 10)
do
    echo $i
done

# Example 5:
for (( i=1; i<=5; i++ ))
do
    echo $i
done

# Example 6: infinite loop
for (( ; ; ))
do
    echo "Infinite Loop..."
done

# Example 7: create 10 copies of test.txt
for i in $(seq 1 10)
do
    cp test.txt "test_${i}.txt"
done

# OR single line format
for i in $(seq 1 10); do cp test.txt "test_${i}.txt"; done

# Example 8: reading file
for line in $(cat test.txt); do
    echo $line
done

# Example 9: Iterating associative array
for key in ${!array[@]}; do
    echo "$key : ${array[$key]}"
done
• Until Loop :

Executes until the condition is false.

Syntax :
until [ condition ]
do
    #Statement
done
Ex :
count=5
until [ $count -le 0 ]
do
    echo "count = $count"
    count=$((count-1))
    sleep 1
done
• Select Loop :
Syntax :
select [var] in [expressions]
do
    #Statements
done
Ex :
select drink in tea cofee water orange apple none
do
    case $drink in
    tea|cofee|water)
        echo "Go to canteen"
        ;;
    orange|apple)
        echo "Go to fruit market"
        ;;
    none)
        break
        ;;
    *)
        echo "ERROR: Invalid Selection"
        ;;
    esac
done
• Loop Controls :
1. Break Statement -
a=1
while [ $a -lt 10 ]
do
    echo $a
    if [ $a -eq 5 ]
    then
        break
    fi
    a=$(expr $a + 1)
done
2. Continue Statement -
nums="1 2 3 4 5 6 7"
for num in $nums
do
    q=$(expr $num % 2)
    if [ $q -eq 0 ]
    then
        echo "Even"
        continue
    fi
    echo $num
done

# Functions

Syntax :
function(){
    #Statements
}
Calling Function :
greet(){
    echo "Hello World"
}

greet
Function with Parameter :
greet(){
    echo "Hello $1"
}

greet World
Call within statement :
capital(){
    echo $1 | tr 'a-z' 'A-Z'
}

name="admin"
echo "Capital: $(capital $name)"
Local Variables :

Local variables are defined within a function and have local scope. They are only accessible within the function where they are defined, not outside of it. It is declared with the local keyword.

swap(){
    local t=$a
    a=$b
    b=$t
}

a=3
b=7
printf "\nA = $a\nb = $b"
swap
printf "\nA = $a\nb = $b"

# Command Line Arguments

Also called shell script parameters.

Arguments Description
$0 returns file name of script
basename $0 filename of script without leading path
$[x] arguments pass to script/function
$# total number of arguments passed
$$ PID of current shell
$! PID of last executed background process
$? exit status of last command/script executed (0 for success and 1 for failure)
$_ command which is being executed previously
$∗ returns all the arguments (in single string)
$@ same as $∗, but differ when enclosed in (") (in array)
$- print the current options flags

# Output Redirection

Commands Description
[command] > [file] redirect STDOUT to file
[command] 1> [file] redirect STDOUT to file (default file descriptor)
[command] 2> [file] redirect STDERR (standard error) to file
[command] &> [file] redirect STDOUT and STDERR to file
[command] > [file] 2>&1 redirect STDOUT and STDERR to file
[command] >> [file] redirect and append STDOUT to file
[command] > /dev/null redirect STDOUT to null (discard STDOUT)
[command] &> /dev/null redirect STDOUT and STDERR to null
echo > [file] create empty file
> [file] create empty file
cat > [file] create file (CTRL+C to exit)

NOTE : 0,1,2 are global file descriptors.

File Descriptor Value
STDIN 0
STDOUT 1
STDERR 2
• Custom File Descriptor :

Define own file descriptor and redirect to it.

  1. Define/Create output file descriptor with x and redirect to file : exec x<> [file]
  2. Redirect to it : [command] >&x
  3. Undefine/Close output file descriptor x : exec x>&-
Ex :
$ exec 3<> log.txt
$ echo "Hello" >&3 #output is redirected to log.txt
$ cat log.txt
Hello

$ exec 3>&-

# Piping

Pipelines or pipes (|) are used as a funnel to pass the output of one command to another command as an input or parameter.

Syntax :
[command] | [command]
Ex :
# Output 1 page at time:
ls -iR | less

# Display 10 rows:
ls -it | head -10

# Print line with word "Hello" in file.txt:
cat file.txt | grep Hello

# Sort output file having word "index":
grep -l index * | sort

# Bash Aliases

Redefine aliases by assigning commands to words or short-names as substitutions.

Synatx :
alias name='[command]'

Save aliases in the file .bashrc (in home directory) and execute the file with -

source .bashrc
Ex :
alias cpp='g++ $1 && ./a.out && rm a.out'

# Brace Expansion

  • echo {1,2,3} : 1 2 3
  • echo {1..5} : 1 2 3 4 5
  • echo {5..1} : 5 4 3 2 1
  • echo {1..-3} : 1 0 -1 -2 -3
  • echo {1..10..2} : 1 3 5 7 9
  • echo {A..E} : A B C D E
  • echo {A..H..2} : A C E G
  • echo a{2..5}b : a2b a3b a4b a5b
  • echo a, b, c, d{1..3}, e : a, b, c, d1, d2, d3, e
  • echo {code1,code2,code3}.c : code1.c code2.c code3.c
  • echo code{.c,.cpp,.py,.sh} : code.c code.cpp code.py code.sh

# RegEx

A regular expression (shortened as regex) is a sequence of characters that defines a search pattern for use in pattern matching to find, match, or replace text.

  • grep ^The file.txt : print line start with "The"
  • grep The^ file.txt : print line end with "The"
  • grep [Hh]ello file.txt : print line with word "Hello" or "hello"
  • grep hello|HELLO file.txt : print line with word "hello" or "HELLO"
  • ls file[1-9] : print files with name file1 to file9
  • grep '[0-9][0-9][0-9][0-9]' file.txt : match for 4 digit number from file.txt with range 0-9
  • ls *[0-9]* : print files containing numbers 0 to 9 (file1.txt test2.sh note5.pdf test_20)
  • ls test.* : print files start with "test" (test.txt test.pdf)
  • ls ab* : print files start with "ab" (ab.txt abcd abort.sh)
  • ls {*.pdf,*.doc,*.exe} : print files end with .pdf/.doc/.exe (file.txt project.doc run.exe)
  • ls a[a-d]b : print files with name bet a[a to d]b (abd acd)
  • ls test_[!3] : print files with number except 3 (test_1 test_2 test_4)

# Grep

Global regular expression print (grep) prints lines that match patterns from text or files.

Syntax :
grep -option [pattern] [file]
  • grep hello file.txt : match for word "hello" in file.txt and print matched line
  • grep -i hello file.txt : case insensitive match for "hello" in file.txt
  • grep -r hello file.txt : recursively match for "hello" in file.txt
  • grep -v hello * : match for all files & folders which doesn`t have word "hello"
  • grep -c hello file.txt : count occurance of "hello" in file.txt
  • grep -w hello file.txt : match for whole word "hello" in file.txt
  • grep -n hello file.txt : match for "hello" in file.txt and print line number with matched lines
  • grep -l hello * : match for "hello" in all files and print name of each matched files instead of matched line
  • grep -B 2 -A 3 hello file.txt : print before 2 lines and after 3 lines of matched case

# Tr

Translate, squeeze, and/or delete characters from standard input and write them to standard output.

Syntax :
tr -option [string1] [string2]
  • tr "ab" "xy" <<< "abcd" : replace occurrence of "a" with "x" and "c" with "z"
  • tr [a-z] [A-Z] <<< "abcd" : convert lower case characters to upper case
  • tr [:lower:] [:upper:] <<< "abcd" : convert lower case characters to upper case
  • tr [:space:] "\t" <<< "Hello World" : translate white-space characters to tabs
  • tr -s [a-z] <<< "Hellooo Worrld" : squeeze sequence of repetitive characters using -s option and replaces with single occurrence of character
  • tr -d "l" <<< "Hello World" : delete specified characters using -d option
  • tr -d [:digit:] <<< "Hello 123 World6789" : remove all the digits from the string
  • tr -c "a" "x" <<< "ababc" : invert the selection by using -c option (complement)
  • tr -cd [:digit:] <<< "abc123def456" : delete everything except digits
Tokens :
Token Description
CHAR1-CHAR2 all characters from CHAR1 to CHAR2 in ascending order
[CHAR*] in ARRAY2, copies of CHAR until length of ARRAY1
[CHAR*REPEAT] REPEAT copies of CHAR, REPEAT octal if starting with 0
[:alnum:] all letters and digits
[:alpha:] all letters
[:blank:] all horizontal whitespace
[:cntrl:] all control characters
[:digit:] all digits
[:graph:] all printable characters, not including space
[:lower:] all lower case letters
[:print:] all printable characters, including space
[:punct:] all punctuation characters
[:space:] all horizontal or vertical whitespace
[:upper:] all upper case letters
[:xdigit:] all hexadecimal digits
[=CHAR=] all characters which are equivalent to CHAR

# Find

It searches for files and directories in the directory hierarchy.

Syntax :
find [path] -option [file]
  • find . -name file.txt : find "file.txt" and is case sensitive
  • find . -iname file.txt : find "file.txt" and is case insensitive
  • find . -name "*.html" : find files end with ".html" and is case sensitive
  • find . -maxdepth 2 "*.cpp" : find files end with ".html" in 2 level depth only
  • find . -not "*.cpp" : find all files except those end with ".cpp"
  • find / -mtime 10 : find files that modified in last 10 days
  • find / -atime 10 : find files whose access time is within last 10 days
  • find / -cmin 10 : find files that created within last 10 days

# Eval

Eval evaluates the specified parameters as a command.

Syntax :
eval [arg...]
Ex :
list="ls | more"
    eval $list

# Expr

Expr evaluates a given expression and prints it to standard output.

Syntax :
expr [OPTION] [EXPRESSION]
Ex :
# Example 1:
expr 1 + 2
expr 5 - 3
expr 2 \* 3
expr 8 / 2

# Example 2:
read x
read y
sum=$(expr $x + $y)
echo "Sum = $sum"

# Example 3: comparing numbers with '=' (0=false & 1=true)
res=$(expr $x = $y)
echo $res

# Example 4: 1 when arg1 < arg2
res=$(expr $x \< $y)
echo $res

# Example 5: length of string
x="Hello World"
echo $(expr length $x)

# Example 6: Extract substring from string
x="Hello World"
sub=$(expr substr $x 2 3)
echo $sub

# Example 7: index of character
x="Hello World"
c=$(expr index $x)
echo $c

# File Permissions

Modifies file permissions.

Syntax :
chmod user+permission [file]
Change Permission Recursively :
chmod -R user+permission [file]
Format of Permission :
-rwx r-- r--
  • 1st bit is file type
  • 2nd 3 bits are user permission
  • 3rd 3 bits are group permission
  • 4th 3 bits are others permission
Options Description
File Type - (regular file), d (dir), c (block device), l (symbolic link)
Setting Permission + (add), - (remove), = (assign)
Permission r (read), w (write), x (execute), - (no permission)
Owner u (user), g (group), o (other), a (all three)
Ex :
chmod u+wr-r text.txt
    chmod o+wrx script.sh
• Permission using Integers :
Permission Value
r 4
w 2
x 1
- 0
Permission Value Permission Description
7 rwx read, write & execute
6 rw- read & write
5 r-x read & execute
4 r-- read-only
3 -wx write & execute
2 -w- write only
1 --x execute only
0 --- none
Note : max value is r+w+x = 4+2+1 = 7
  • wrx wrx --- = 4+2+1 4+2+1 0+0+0 = 770
  • r-x r-- r-- = 4+0+1 4+0+0 4+0+0 = 544
Ex :
chmod 777 file.sh

# Linux Filesystem Hierarchy

linux filesystem