10 Practical Example Of SORT Command in Unix/Linux

Sort command is used to sort the data available in files. We can arrange the data in specific order based on the requirement. we can apply sort command in both string as well as numeric data by using switches provided by sort command. Let see how sort works in Unix/Linux with the help of some examples. Lets start..!

1. Sort string data

Lets assume you have file with below sample data.
TRANS
FIN
MED
HR
CSR
FIN

$ sort F_Input_File.txt

Output: Will sort the file in alphabetical order and will produce below output:
CSR
FIN
FIN
HR
MED
TRANS

2. Sort numeric data

Lets assume you have file with below sample data.
10
110
09
21
34
15

$ sort F_Input_File.txt

Output: Will produced below output based on ASCCI values. But this is not what we are looking for.
09
10
110
15
21
34

This is numeric data and sort has sorted it based on ascii, to sort numeric data sort provides a septate switch -n. below will provide the required o/p.

$ sort -n F_Input_File.txt

Output:
09
10
15
21
34
110

3. Reverse the sort output

We can sort the data in reverse order as well. In previous case data is being sorted in Ascending order. Below will sort the data in Descending order

$ sort -n -r F_Input_File.txt

[or]

$ sort -r -n F_Input_File.txt

Output: Both will provide the same output. -n force for numeric sort and -r reverse the operation of sort.
110
34
21
15
10
09

By default sort sort the data in ascending order, by using -r switch we can sort in descending order ( same as oracle sql order clause)

4. Sort the data and return unique records

Lets assume you have file with below sample data.
TRANS
FIN
MED
HR
CSR
FIN

$ sort -u F_Input_File.txt

Output: Above will return below output, -u switch force sort to return the unique records.
CSR
FIN
HR
MED
TRANS

Explanation: Sort first sort the data and -u make sure that only one occurrence of record would be available in the output.

5. Don't sort the data if already sorted

Let say we have below sorted data available in input file.
CSR
FIN
HR
MED
TRANS

$ sort -c F_Input_File.txt

Output: Will not perform any action as data is already sorted

6. Redirect sort result into output file

String Sort

$ sort -o F_Output_File.txt F_Input_File.txt

Output: It will sort the data available in F_Input_File.txt to output file F_Output_File.txt 

Numeric Sort
$ sort -n -o F_Output_File.txt F_Input_File.txt
Output: Here if input file has numeric data then it will first sort the data based on numer and then redirect the output to the output file. 

Reverse Sort

$ sort -n -r -o F_Output_File.txt F_Input_File.txt

Output: Here output file will have the sorted data in reverse order, there fore in descending order.

7. Sort the Data on a specifid field in file.

This is most important part of sort as we are going to use this very often on data files. In real time scenarios, we receives data in data files where we have to sort the data on a specific column or multiple columns. Let see how to implement it.

SYNTAX:
$ sort -t<delimiter> km,n File_name

-t is used to hold the delimiter, space is default delimiter
-k stands for key and used to provide the column/field on which you want to perform your sorting.
m,n is used to provide the window (eg. from col1 to col4) on which you want to perform sorting.

Lets assume we have a pipe(|) delimited data file as shown below.

A5|CSR|N|50000
A7|FIN|N|40000
A6|TRANS|Y|30000
A4|HR|Y|20000
A2|MED|Y|20000
A3|FIN|N|10000
A1|TRANS|Y|100000

If we perform simple sort, it will return the below output:

$ sort  F_Input_File.txt

A1|TRANS|Y|100000
A2|MED|Y|20000
A3|FIN|N|10000
A4|HR|Y|20000
A5|CSR|N|50000
A6|TRANS|Y|30000
A7|FIN|Y|40000

Output: Has sort the data based on complete line not on any specific field.

8. sort based on salary(last field)

$ sort -t"|" -k4,4 F_Input_File.txt

Output: Opps, this is not what we are expecting, A1 should be the last record, but as we know the column on which we are performing sort operation is number we have to use -n switch for correct result.

A3|FIN|N|10000
A1|TRANS|Y|100000
A2|MED|Y|20000
A4|HR|Y|20000
A6|TRANS|Y|30000
A7|FIN|N|40000
A5|CSR|N|50000

$ sort -t"|" -n -k4,4 F_Input_File.txt
[or]
$ sort -t"|"  -nk4,4 F_Input_File.txt

Note: You must you -km,n syntax to perform the sort even if you are performing sort based on a single column or field because of below reasons:

1. -k2 means sort the data starting from 2nd field to last field.
2. -k2,2 means sort the data based on 2nd column only.

Output: Now we have correct output.
A3|FIN|N|10000
A2|MED|Y|20000
A4|HR|Y|20000
A6|TRANS|Y|30000
A7|FIN|N|40000
A5|CSR|N|50000
A1|TRANS|Y|100000

9. sort based on department(2nd field)

$ sort -t"|" -k2,2 F_Input_File.txt

Output:
A5|CSR|N|50000
A3|FIN|N|10000
A7|FIN|Y|40000
A4|HR|Y|20000
A2|MED|Y|20000
A1|TRANS|Y|100000
A6|TRANS|Y|30000

10. Sort the data based on the department and then list down the employee based on their salary.

$ sort -t"|" -k2,2 -k4,4 F_Input_File.txt

Output: It reads, first sort the data based on 2nd column and then on 4th column but again 4th column is numeric and we have not used the -n switch hence not received the expected output. 

A5|CSR|N|50000
A3|FIN|N|10000
A7|FIN|Y|40000
A4|HR|Y|20000
A2|MED|Y|20000
A1|TRANS|Y|100000
A6|TRANS|Y|30000

To get the correct output below need to be used.

$ sort -t"|" -k2,2 -k4,4n F_Input_File.txt

Output: This time we have correct output as per the requirement.
A5|CSR|N|50000
A3|FIN|N|10000
A7|FIN|Y|40000
A4|HR|Y|20000
A2|MED|Y|20000
A6|TRANS|Y|30000
A1|TRANS|Y|100000

Reverse the sort data.

$ sort -t"|" -r -k2,2 -k4,4n F_Input_File.txt

Output:
A6|TRANS|Y|30000
A1|TRANS|Y|100000
A2|MED|Y|20000
A4|HR|Y|20000
A3|FIN|N|10000
A7|FIN|Y|40000
A5|CSR|N|50000

Conclusion: We will come across situation or requirement where we need data in a specific order either in ascending or descending.  Sort command is very handy to arrange the data in a specified order. Keep learning, Keep practising ...!!

How To Use File Test Operator in Unix/Linux

In this article, we will see how to use file operators in scripting. We have number of test operators for file but we will see which are used extensively on day to day basis. Once you are good with the following file test operators you can explore the other operator as well using man page.

-s file is not zero size
-e file exists
-d file is a directory
-r file has read permission
-w file has write permission
-x file has execute permission

$ ls -ltr  abc.txt

-rw-rw----   1 baba     baba  0 Dec 24 05:20 abc.txt

F_File_Name="abc.txt"

How to check whether a file exists or not
if [ -e ${F_File_Name} ]
then
  echo "File Exists"
else
  echo "File Doesn't Exists"
fi

As by now we have learnt one liner if-else syntax, we can write above block as below:

$ [ -e ${F_File_Name} ] &&  echo "File Exists" || echo "File Doesn't Exists";

Output: File Exists
-e stands for exists and return TRUE if file exists.

How to check whether a file is empty/zero-size or not

if [ -s ${F_File_Name} ]
then
  echo "File Is Not Empty"
else
  echo "File Is Empty"
fi

[OR]

$ [ -s ${F_File_Name} ] &&  echo "File Is Not Empty" || echo "File Is Empty";

Output: File Exists
-s returns TRUE if file is not empty or else returns FALSE

How to check whether a file is directory or not

if [ -d ${F_File_Name} ]
then
  echo "File Is a Directory"
else
  echo "File Is Not a Directory"
fi

[OR]

$ [ -d ${F_File_Name} ] &&  echo "File Is a Directory" || echo "File Is Not a Directory";

Output: File Is Not a Directory
-s returns TRUE if file is not empty or else returns FALSE

How To Check Whether Testing User Has Read/Write/Execute Permission On File

if [ -r ${F_File_Name} ]
then
  echo "File Has Read Permission"
else
  echo "File Does Not Have Read Permission"
fi
[OR]

$ [ -r ${F_File_Name} ] &&  echo echo "File Has Read Permission" || echo "File Does Not Have Read Permission";

Output: File Has Read Permission
-r returns TRUE if file has read permission or else returns FALSE
  if [ -w ${F_File_Name} ]
then
  echo "File Has Write Permission"
else
  echo "File Does Not Have Write Permission"
fi

[OR]

$ [ -w ${F_File_Name} ] &&  echo echo "File Has Write Permission" || echo "File Does Not Have Write Permission";

Output: File Has Read Permission
-w returns TRUE if file has write permission or else returns FALSE

if [ -x ${F_File_Name} ]
then
  echo "File Has Execute Permission"
else
  echo "File Does Not Have Execute Permission"
fi

[OR]

$ [ -x ${F_File_Name} ] &&  echo echo "File Has Execute Permission" || echo "File Does Not Have Execute Permission";

Output: File Has Read Permission
-x returns TRUE if file has execute permission or else returns FALSE

Conclusion: The file test operators are very useful in script writing, you will face situations where you need to check these operators and based on the output need to perform required action. Keep practicing..!!

How To Check String Nullability In Unix/Linux

In this article we will see how to validate/check whether a variable is NULL or empty. Let see with the help of examples.Lets say we have a variable V_Error_Msg as given below; I will explain all the solution on this variable.

V_Error_Msg="ORA-00942"

Using ==/= Operator

if [ ${V_Error_Msg} = "" ];
then
  echo "V_Error_Msg is Empty/NULL"
else
  echo "V_Error_Msg is Not Empty/NULL"
fi

[OR]

if [ ${V_Error_Msg} == "" ];
then
  echo "V_Error_Msg is Empty/NULL"
else
  echo "V_Error_Msg is Not Empty/NULL"
fi

Output: Both will produce same output in some shells equal(=) to operator do the needful and in some double equal(==) do the needful.
V_Error_Msg is Not Empty/NULL

Using ! Operator

if [ ! "${V_Error_Msg}" ];
then
  echo "V_Error_Msg is Empty/NULL"
else
  echo "V_Error_Msg is Not Empty/NULL"
fi

Output:
V_Error_Msg is Not Empty/NULL
Explanation: If V_Error_Msg holds any value then if return true, means variable is not empty and if V_Error_Msg is NULL then if block return false. Here variable V_Error_Msg is not empty hence it has returned TRUE and ! of TRUE is FALSE hence "V_Error_Msg is Not Empty/NULL" has been printed on terminal.

Using -z Switch

$ [ -z "${V_Error_Msg}" ] && echo "V_Error_Msg is Empty/NULL" || echo "V_Error_Msg is Not Empty/NULL"

Output:
V_Error_Msg is Empty/NULL
Explanation: Syntax reads, if V_Error_Msg is empty then(&&) TRUE else(||) FALSE

$ [ ! -z "${V_Error_Msg}" ] && echo "V_Error_Msg is Not Empty/NULL"  || echo "V_Error_Msg is Empty/NULL"

Output:
V_Error_Msg is Empty/NULL
Explanation: Syntax reads, if V_Error_Msg is Not empty then(&&) TRUE else(||) FALSE

Using -n Switch

$ [ ! -n "${V_Error_Msg}" ] && echo "V_Error_Msg is Empty/NULL" || echo "V_Error_Msg is Not Empty/NULL"

Output:
V_Error_Msg is Empty/NULL
Explanation: Syntax reads, if V_Error_Msg is empty then(&&) TRUE else(||) FALSE

$ [ -n "${V_Error_Msg}" ] && echo "V_Error_Msg is Not Empty/NULL"  || echo "V_Error_Msg is Empty/NULL"

Output:
V_Error_Msg is Empty/NULL

Explanation: Syntax reads, if V_Error_Msg is empty then(&&) TRUE else(||) FALSE
Note: Please observe carefully, there is a space between operator and operand. If we will not provide the space it will through an error
sh: test: argument expected

Keep Learning, Keep Practising, keep Sharing.

UNIX NVL Functionality

In this article we will see, how can we implement oracle NVL like functionality in UNIX.
If Unix/Linux variable is not null/empty, display that value else display an user defined value. Let see how can we implement and use it in real time.

Scenario:
We have a metadata file which has configuration entries. the content of the file are shown below.


$ cat F_Execution_Meta_Data
EXECUTION_MODE=FULL


#!bin/sh
V_Error_Code_Tmp=`grep -w "EXECUTION_MODE" F_Execution_Meta_Data | cut -d"=" -f2`
if [ ${V_Error_Code}=="FULL" ]
then
  #--* Perform your Operation, If condition is TRUE;
else
  #--* Perform your Operation, If condition is FALSE;
fi

Above will work perfectly fine till we have value available in Meta Data, if no value specified [EXECUTION_MODE=] then grep will return NULL/empty which will result  into If condition failure. We will get below error:
Error: Null_Check.sh[2]: test: argument expected

How to handle such scenario or what should be the basic standard which we should incorporate in our code. Lets have a look multiple solutions.

Solution 1: Using -n switch

#!bin/sh
V_Error_Code=`grep -w "EXECUTION_MODE" F_Execution_Meta_Data | cut -d"=" -f2`
if [ ! -n "${V_Error_Code}" ]
then
 V_Error_Code="Z"
fi
if [ ${V_Error_Code}=="FULL" ]
then
  #--* perform your Operation, If condition is TRUE;
else
  #--* perform your Operation, If condition is FALSE;
fi

-n the length of STRING is NonZero
! -n the length of STRING is Zero

instead of writing complete if block, you can write the same in a single line.
$ [ ! -n "$V_Error_Code" ] && V_Error_Code="Z"

Explanation: Now if grep returns NULL, first if block will catch it and assign a value to the variable V_Error_Code which will then be used in second if block. It will makes sure your code would not fail due to test: argument expected error.

Solution 2: Using -z switch

#!bin/sh
V_Error_Code=`grep -w "EXECUTION_MODE" F_Execution_Meta_Data | cut -d"=" -f2`
if [  -z "${V_Error_Code}" ]
then
 V_Error_Code="Z"
fi
if [ ${V_Error_Code}=="FULL" ]
then
  #--* perform your Operation, If condition is TRUE;
else
  #--* perform your Operation, If condition is FALSE;
fi

-z the length of STRING is zero
! -z the length of STRING is Nonzero

instead of writing complete if block, you can write the same in a single line.
$ [ -z "$V_Error_Code" ] && V_Error_Code="Z"


Explanation: have same mechanism as in case of first solution, only difference is -z switch which works opposite to -n switch.

Solution 3: Using Parameter substitution ( Simple and best one)

#!bin/sh
V_Error_Code_Tmp=`grep -w "EXECUTION_MODE" F_Execution_Meta_Data | cut -d"=" -f2`
V_Error_Code=${V_Error_Code_Tmp:-Z}
if [ ${V_Error_Code}=="FULL" ]
then
  #--* perform your Operation, If condition is TRUE;
else
  #--* perform your Operation, If condition is FALSE;
fi

Explanation: Observe the syntax clearly, ${V_Error_Code_Tmp:-Z}, it reads, if value of V_Error_Code_Tmp is NULL then assign Z to the variable or if not null then assign the same value to the variable.

Conclusion: We should always use this functionality when ever we are using a variable
in if for some condition test to make sure our if block doesn't get failed. Keep learning,
keep designing & keep the coding standard high..!!

How To Redirect Duplicate Record In One File And Uniqe In Other

In this article we will see how can we redirect the duplicate & unique records from a fixed width file in two different files. The below solution can be implemented for delimited files as well. let see how can we implement it. Duplicate records based on the value available from 1st position to 7th position and redirect in a log file. 

Please go through How if-else Works Without Operators In AWK  for more understanding of this post.

Input File:F_Input_File.txt

10001A1TRANSY10000
10001A1TRANSY10000
10002A2MEDY20000
10003A3FINN10000
10004A4HRY20000
10005A5CSRN50000

$ awk '{V_Duplicate=substr($0,1,7);
  if (V_Filter[V_Duplicate]++)
  {
      print $0 > "F_Dup_Record.log"
  }
  else
  {
      print $0 >"F_Unique_Record.txt"
  }
 } ' F_Input_File.txt

[OR]

You can write above syntax as below and can reduce some line of code. But the 1st one is more descrptive and understandable.

$ awk '{if (V_Filter[substr($0,1,7)]++)
  {
      print $0 > "F_Dup_Record.log"
  }
  else
  {
      print $0 >"F_Unique_Record.txt"
  }
 } ' F_Input_File.txt

Output:
1. F_Dup_Record.log will hold all the V_Duplicate records.
2. F_Unique_Record.txt will hold all the unique records(It will also have the a copy of the V_Duplicate record).

$ cat F_Unique_Record.txt
10001A1TRANSY10000
10002A2MEDY20000
10003A3FINN10000
10004A4HRY20000
10005A5CSRN50000

$ cat F_Dup_Record.log
10001A1TRANSY10000

Explanation:

To understand, how this command is working, first we need to understand how a++ works.

a++ is postfix operation, meaning that the value of a will get changed after the evaluation of expression or in other word it says add 1 to a, returns the old value of a i.e return 0.

Assume we have 3 records in file as below:

aa
bb
aa

Let see our case, array V_Filter[V_Duplicate]++ says add 1 to the VALUE of V_Filter[V_Duplicate] and return the old value of V_Filter[V_Duplicate] which is 0 at the start.

Step 1: V_Filter[aa] = V_Filter[aa] + 1 => will return 0 as old value and assign 1 to V_Filter[aa] after the evolution; as 0 is considered false, if evaluate it to failure of condition hence passed to the unique(else) block.

Step 2: V_Filter[bb] = V_Filter[bb] + 1 => will return 0 as old value assign 1 to V_Filter[bb] ; as 0 is considered false, if evalute it to failure of condition hence passed to the unique(else) block.

Step 3: When 3rd record will be processed, scenario would be something like below:
V_Filter[aa] = V_Filter[aa] + 1 => will return old value, for V_Filter[aa] old value is 1 from the step 1, and assign new value to V_Filter[aa] which is 2 after evalution.  as 1(non-zero : Old return value) is considered true hence passed to the V_Duplicate(if) block.

Note: We need to focus on what is being returned during the postfix operation, that is old value. If you focus on old value you will understand the logic used here.

How if-else Works Without Operators In AWK

In this article we will see when if block is true or false in without operator scenario. Means none of the following is used.
-eq/-ne/-lt/-gt/=/==/!=

echo "" | awk 'BEGIN{V_Count=0}
   { if (V_Count)
     {
         print "TRUE"
     }
     else
     {
         print "FALSE"
     }
   }'

Output: In above scenario, we have initialized a variable V_Count and assigned the value as 0. if inside awk command considered 0 as FALSE hence displayed FALSE on terminal.

$ echo "" | awk 'BEGIN{V_Count=1}
   { if (V_Count)
     {
         print "TRUE"
     }
     else
     {
         print "FALSE"
     }
   }'

[OR]

$ echo "" | awk 'BEGIN{V_Count=2}
   { if (V_Count) 
     {
         print "TRUE"
     } 
     else
     {
         print "FALSE"
     }
   }'

Output: In above scenario, we have initialized a variable V_Count and assigned the value as 1 and 2 in next example. if inside awk command considered non-zero value as TRUE hence displayed TRUE on terminal for both the cases.

Note: This is very important concept of awk command and is used extensively. Keep the concept handy for complex code understanding.

How To Find Duplicate Records In Fixed Width File In UNix/Linux

In this article we will see how to find Duplicate record in a fixed width file using awk command.

Input File:F_Input_File.txt

10001A1TRANSY10000
10001A1TRANSY10000
10001A1TRANSY10000
10002A2MEDY20000
10003A3FINN10000
10004A4HRY20000
10005A5CSRN50000

Find out V_Duplicate records based on the value available from 1st position to 7th position.
$ awk '{V_Duplicate=substr($0,1,7);V_Filter[V_Duplicate]++} END{for(i in  V_Filter) {print i"|" V_Filter[i]}}' F_Input_File.txt | awk -F"|" '$2>1{print $0}';

[OR]
$ awk '{V_Filter[substr($0,1,7)]++} END{for(i in  V_Filter) {print i"|" V_Filter[i]}}' F_Input_File.txt | awk -F"|" '$2>1{print $0}';
Output:
10001A1|2

Explanation:
1. substr($0,1,7) will use complete record ($0) as string and then will slice a string from position 1 to 7.
2. Please follow the post How to Use Associative Array in AWK for further explanation on how awk array process the data.

How To Use Tail Command In Unix/Linux

In this article we will see how tail command works and its real time practical example.

File1:abc.txt
10001|A1|TRANS|Y|10000
10002|A2|MED|Y|20000
10003|A3|FIN|N|10000
10004|A4|HR|Y|20000
10005|A5|CSR|N|50000
10006|A6|TRANS|Y|30000
10007|A7|FIN|N|40000

File2: xyz.txt
20001|B1|TRANS|Y|10000
20002|B2|MED|Y|20000
20003|B3|FIN|N|10000
20004|B4|HR|Y|20000
20005|B5|CSR|N|50000

How to read last 3 line of a file
$ tail -n 3 abc.txt

[OR]

$ tail -3 abc.txt

Output: Will display the last 3 lines of the file abc.txt
10005|A5|CSR|N|50000
10006|A6|TRANS|Y|30000
10007|A7|FIN|N|40000

How to print line from a specefied line (nth)
$ tail -n +3 abc.txt

Note: observe how we have used + sign with the number in tail, while in head we have used - sign.

Output: Will display the all the lines from 3rd line to the end of file
10003|A3|FIN|N|10000
10004|A4|HR|Y|20000
10005|A5|CSR|N|50000
10006|A6|TRANS|Y|30000
10007|A7|FIN|N|40000

Default tail will display last 10 lines of a file
$ tail abc.txt

Output: Has displayed complete file as it has less than 10 lines.
10001|A1|TRANS|Y|10000
10002|A2|MED|Y|20000
10003|A3|FIN|N|10000
10004|A4|HR|Y|20000
10005|A5|CSR|N|50000
10006|A6|TRANS|Y|30000
10007|A7|FIN|N|40000


How to see lines from multiple files
$ tail abc.txt xyz.txt

Output: Will display the last ten lines of both abc.txt and xyz.txt, with a header before each that indicates the filename.

How display the first 3 lines from multiple table
$ tail -n +3 abc.txt xyz.txt

Output:Will displya the line from 3rd line to the end of file.
==> abc.txt <==
10003|A3|FIN|N|10000
10004|A4|HR|Y|20000
10005|A5|CSR|N|50000
10006|A6|TRANS|Y|30000
10007|A7|FIN|N|40000

==> xyz.txt <==
20003|B3|FIN|N|10000
20004|B4|HR|Y|20000
20005|B5|CSR|N|50000

How to see the the lines from 4th line to the end for every .txt file in the directory
$ tail -n +4 *.txt

Output: Display the line from the 4th line from every .txt file in the working directory
==> abc.txt <==
10004|A4|HR|Y|20000
10005|A5|CSR|N|50000
10006|A6|TRANS|Y|30000
10007|A7|FIN|N|40000

==> xyz.txt <==
20004|B4|HR|Y|20000
20005|B5|CSR|N|50000

View continuously increasing file in real time using tail command
$ tail -f abc.txt

Output: Will display the latest added lines in the growing file abc.txt

How To Use Head Command In Unix/Linux

In this article we will see how head command works and its real time practical example.

File1:abc.txt
10001|A1|TRANS|Y|10000
10002|A2|MED|Y|20000
10003|A3|FIN|N|10000
10004|A4|HR|Y|20000
10005|A5|CSR|N|50000
10006|A6|TRANS|Y|30000
10007|A7|FIN|N|40000

File2: xyz.txt
20001|B1|TRANS|Y|10000
20002|B2|MED|Y|20000
20003|B3|FIN|N|10000
20004|B4|HR|Y|20000
20005|B5|CSR|N|50000

How to read first 3 line of a file
$ head -n 3 abc.txt

[OR]

$ head -3 abc.txt

Output: Will display the first 3 lines of the file abc.txt
10001|A1|TRANS|Y|10000
10002|A2|MED|Y|20000
10003|A3|FIN|N|10000

How to ignore last 3 line of a file 
$ head -n -3 abc.txt

Output: Will display the all the lines except last 3 lines from the file abc.txt
10001|A1|TRANS|Y|10000
10002|A2|MED|Y|20000
10003|A3|FIN|N|10000
10004|A4|HR|Y|20000

Default head will display first 10 lines of a file.
$ head abc.txt

Output: Has displayed complete file as it has less than 10 lines.
10001|A1|TRANS|Y|10000
10002|A2|MED|Y|20000
10003|A3|FIN|N|10000
10004|A4|HR|Y|20000
10005|A5|CSR|N|50000
10006|A6|TRANS|Y|30000
10007|A7|FIN|N|40000

How to see lines from multiple files
$ head abc.txt xyz.txt

Output: Will display the first ten lines of both abc.txt and xyz.txt, with a header before each that indicates the file name.

How display the first 3 lines from multiple table
$ head -n 3 abc.txt xyz.txt

Output:Will display the first 3 line from both the file.
==> abc.txt <==
10001|A1|TRANS|Y|10000
10002|A2|MED|Y|20000
10003|A3|FIN|N|10000

==> xyz.txt <==
20001|B1|TRANS|Y|10000
20002|B2|MED|Y|20000
20003|B3|FIN|N|10000

How to see the first 4 lines of .txt file in the directory
$ head -n 4 *.txt

Output: Display the first four lines of every .txt file in the working directory
==> abc.txt <==
10001|A1|TRANS|Y|10000
10002|A2|MED|Y|20000
10003|A3|FIN|N|10000
10004|A4|HR|Y|20000

==> xyz.txt <==
20001|B1|TRANS|Y|10000
20002|B2|MED|Y|20000
20003|B3|FIN|N|10000
20004|B4|HR|Y|20000

5 Practical Uses Of While Loop

In this article we will see the basics of while loop and how to use it with logical operators, string and with files. Please go through with the post 15 Practical Uses For Loop In Unix/Linux  for more clarity with loops.

SYNTAX:

 while [ Condition ]
 do
  #--Program logic
 done

Operation on Number:
-lt less than <
-le less than or equal to <=
-gt greater than >
-ge greater than or equal to >=
-ne not equal to !=

Operation on String:
== Equals to
!= Not Equals to

1. How to iterate using while loop.
k=0
while [ $k -lt 4 ]
do
 echo $k
 k=`expr $k + 1`
done

2. How to provide AND and OR condition in while loop.
k=0
m=0
while [[ $k -lt 4 && $m -lt 3 ]]
do
    echo "Value of k is: $k"
    echo "Value of m is: $m"

    k=`expr $k + 1`
    m=`expr $m + 1`
done

Output: Will list down below output, please see double bracket in while condition. When you are using multiple condition provide doube square bracket.
Value of k is:0
Value of m is:0
Value of k is:1
Value of m is:1
Value of k is:2
Value of m is:2

[OR]

k=0
m=0
while [[ $k -lt 4 || $m -lt 3 ]]
do
    echo "Value of k is: $k"
    echo "Value of m is: $m"
    
    k=`expr $k + 1`
    m=`expr $m + 1`
done

Output: Here while has iterated until the value of k becomes 4, irrespective of value of m because of OR (||) condition.
Value of k is: 0
Value of m is: 0
Value of k is: 1
Value of m is: 1
Value of k is: 2
Value of m is: 2
Value of k is: 3
Value of m is: 3

3. How to use while loop with strings
V_User_Name="Unix"
while [ "${V_User_Name}" != "Unix" ]
do
    echo "Welcome ${V_User_Name}"
done

Output: Will not print anything and will come out.

V_User_Name="Unix"
while [ "${V_User_Name}"=="Unix" ]
do
    echo "Welcome ${V_User_Name}"
    break;
done

Output: Will print Welcome Unix.
Please see == sign carefully. there should not be any space between operands and operator.

4. How to execute while in Infinite loop.
while true:
do
 echo "Unix is Great"
done

Output: This while loop will execute indefinitely. To coming out from an infinite loop we can use break statement as used in above scenario.

5. How to read a file using while loop.
while read V_File_Line
do
 echo ${V_File_Line}
done < F_Input_File.txt

Output: Will read the file line by line and display at console. We can build our logic on V_File_Line as per our requirement.
Unix Linux Windows
All are awesome OS
Here,
1. F_Input_File.txt is the file from where we are reading the data/line.
2. read is the keyword.
3. unlike for loop, we need not require to set IFS in case of while loop. While loop internally manage to print the complete line.

15 Practical Uses For Loop In Unix/Linux

In this article we will see the basic and advance syntax of for loop and how we can use the for loop utility in shell scripting. 

1. How to loop through a series of numbers using for loop
for k in 1 2 3 4 5
do
   echo "Output Value=> $k"
done

Output: Will display the series on terminal.
Output Value=> 1
Output Value=> 2
Output Value=> 3
Output Value=> 4
Output Value=> 5

2. How to loop through a range of numbers using for loop
for k in {1..5}
do
   echo "Output Value=> $k"
done

Output: Will display the range on terminal.
Output Value=> 1
Output Value=> 2
Output Value=> 3
Output Value=> 4
Output Value=> 5

3. How to loop through sequence (seq function) value using for loop
for k in $(seq 1 5)
do
  echo "Output Value=> $k"
done

[OR]

for k in `seq 1 5`
do
  echo "Output Value=> $k"

done

Output: Sequence function will generate values from 1 to 5 and for loop will treat as a range and will display it on terminal.
Output Value=> 1
Output Value=> 2
Output Value=> 3
Output Value=> 4
Output Value=> 5

4. How to increment for loop by a interval in range 
for k in {0..10..2}
do
     echo "Output Value=> $k"
done

Output: Will display the range  of 0 to 10 with the increment of 2 every time.
Output Value=> 0
Output Value=> 2
Output Value=> 4
Output Value=> 6
Output Value=> 8
Output Value=> 10

5. How to loop through the character series using for loop.
It is not like you can only loop through Number, infect you can loop through anything.

for k in a b c d e
do
   echo "Output Value=> $k"
done

Output: Will loop through the series one by one and display on terminal.
Output Value=> a
Output Value=> b
Output Value=> c
Output Value=> d
Output Value=> e

6. How to loop through the character range
for k in {a..c}
do 
echo "Output Value=> $k" 
done

Output: Will display the all characters of range a to c.
Output Value=> a
Output Value=> b
Output Value=> c
We can use upper case range as well.

7. How to loop through the Words series
for k in unix linux windows
do
   echo "Output Value=> $k"
done

Output: Will loop through each word and display accordingly.
Output Value=> unix
Output Value=> linux
Output Value=> windows

8. How to list down all the files of directory
for k in `ls`
do
   echo "Output Value=> $k"
done

[OR]

for k in *
do
   echo "Output Value=> $k"
done

Output: * stands for everything in the directory. it will display ever thing where for loop is executed.
Output Value=> xyz.txt
Output Value=> abc.txt
Output Value=> Tgt

9. How to list down all the files of directory
for k in `ls *.txt`
do
   echo "Output Value=> $k"
done

Output:
Output Value=> abc.txt

10. How to implement an C-style loop in UNIX script
for (( k=1; k<=5; k++ ))
do
echo "Value of k is =>$k"
done

11. How to implement an infinite loop in UNIX script 
for (( ; ; ))
do
   echo "infinite loops"
done

Output: Will go in infinite loop, If you want to menu bases scripts use infinite for loop. ctr + c will get you out from infinite loop execution.

12. How to read a file using for loop
for V_Line in `cat abc.txt` 
do 
echo "Output Line=>"$V_Line 
done 

IFS=$'\n'; 
for V_Line in `cat abc.txt` 
do 
echo "Output Line=>"$V_Line 
done 

[OR]

IFS=$'\n'; 
for V_Line in $(cat abc.txt); 
do 
echo "Output Line=>"$V_Line
done

IFS= Internal Field Separator
This just tells the shell to split on newlines only, not spaces which is by default.

13. How to use a variable in Range limit of for loop.
V_Upper_Limit=5
for k in {1..${V_Upper_Limit}}
do
    echo "First Value=> $k"
done

Will not produced as required instead will display {1..5}, to over come this issue we can use below approaches.

V_Upper_Limit=5
for k in `seq 1 ${V_Upper_Limit}`
do
  echo "First Value=> $k"
done
[OR]

V_Upper_Limit=5
for k in `eval "echo {1..${V_Upper_Limit}}"`
do
    echo "First Value=> $k"
done

14. How to loop through script parameter
for k in $@
do
echo "First Script Parameter $k"
done

Output: Will display all the parameter to the a.sh. $@ holds all the parameter.

15 . How to loop through the output of any Unix/Linux command.
for k in `grep -i "error" abc.log`
do
echo "Output is=>"$k
done

Output: Will list down all the lines from abc.log where error is found but break the output line based on space not on newline. We need to set Internal Field separator IFS=$'\n'; 

IFS=$'\n'; 
for k in `grep -i "error" abc.log`
do
echo "Output is=>"$k
done

Now it will print complete line which grep produced as a search.

[OR]

for k in `find . -name "*.txt"`
do
echo "Output is=>"$k
done

Output: Will list down all the .txt file from current directory.

We can write for block in single line as below only thing is we need to put semi colon(;) after every action.

$ for k in `find . -name "*.txt"`; do echo "Output is=>"$k; done
$ for k in `grep -i "Error" abc.log`; do echo "Output is=>"$k; done

Conclusion:
Unix/Linux for loop is very powerful utility and can be used in multiple situations as explained above. It reduces the code length and provides an easy syntax. Keep practising to become a champ in for loop.