Swap or Change Appearance of Columns Using AWK

Input File: F_Data_File.txt

EMPID|EMPNAME|EMPDEPT|EMPSAL|LOCATION
10001|A1|HR|10000|USA
10002|A2|FIN|20000|USA
10003|A3|NSS|30000|IND
10004|A4|SEC|40000|USA
10005|A5|TECH|50000|IND
10006|A6|TECH|60000|IND
10007|A7|TECH|70000|IND

1. We want to swap location with Salary means now location column sould be the 4th column which is 5th in original file.

$ awk 'BEGIN{FS="|";OFS="|";}{print $1 OFS $2 OFS $3 OFS $5 OFS $4}' F_Data_File.txt
(or)
$ awk 'BEGIN{FS="|";OFS="|";}{print $1 OFS $2 OFS $3 OFS $5 OFS $4}' F_Data_File.txt > F_Temp_File.txt

We have redirected the output of the awk command into a file for permanent changes as below:

Output:

EMPID|EMPNAME|EMPDEPT|LOCATION|EMPSAL
10001|A1|HR|USA|10000
10002|A2|FIN|USA|20000
10003|A3|NSS|IND|30000
10004|A4|SEC|USA|40000
10005|A5|TECH|IND|50000
10006|A6|TECH|IND|60000
10007|A7|TECH|IND|70000

2. We want to remove salary column from the input data.

$ awk 'BEGIN{FS="|";OFS="|";}{print $1 OFS $2 OFS $3 OFS $5 OFS $4}' F_Data_File.txt
(or)
$ awk 'BEGIN{FS="|";OFS="|";}{print $1 OFS $2 OFS $3 OFS $5}' F_Data_File.txt > F_Temp_File.txt

Output:

EMPID|EMPNAME|EMPDEPT|LOCATION
10001|A1|HR|USA
10002|A2|FIN|USA
10003|A3|NSS|IND
10004|A4|SEC|USA
10005|A5|TECH|IND
10006|A6|TECH|IND
10007|A7|TECH|IND

3. Concatenate two fields, let say EMPNAME & EMPDEPT and create a new column EMPNAME-DEPT.

$ awk 'BEGIN{FS="|";OFS="|";print "EMPID|EMPNAME-DEPT|LOCATION"} NR>1{print $1 OFS $2"-"$3 OFS $4 OFS $5}' F_Data_File.txt
(or)
$awk 'BEGIN{FS="|";OFS="|";}{print $1 OFS $2"-"$3 OFS $4 OFS $5}' F_Data_File.txt > F_Temp_File.txt

As we have concatenated two columns hence data will be available in new column but header still have 5 column names. To update the header we have initialize the header in BEGIN block and eliminated the first line from the existing file.

Output:

EMPID|EMPNAME-DEPT|LOCATION
10001|A1-HR|10000|USA
10002|A2-FIN|20000|USA
10003|A3-NSS|30000|IND
10004|A4-SEC|40000|USA
10005|A5-TECH|50000|IND
10006|A6-TECH|60000|IND
10007|A7-TECH|70000|IND

AWK Basics For beginners

AWK Basic Syntax:

awk -F"<Delimiter>"  'BEGIN{INITIALIZATION}
                      {ACTION} #-- For every line in file
                      END{END BLOCK}'  <FILE_NAME>

1. BEGIN BLCOK: Block executes only once at the start of AWK command, used for initialization.
2. EXECUTION BLOCK: Block is the heart of AWK command carry all the processing logic.
3. END BLOCK: Block executes only once in the END.
4. FILE_NAME: holds the file that needs to be processed by AWK.

AWK views a text file as records and fields represent by $1 (First Field),$2 (Second Field)...so on. $0 has special meaning in AWK, it represent complete record means, it hold the current processing line from the file. Please go through earlier post on Inbuilt Variables in AWK.

Input File: F_Data_File.txt

EMPID|EMPNAME|EMPDEPT|EMPSAL|LOCATION
10001|A1|HR|10000|USA
10002|A2|FIN|20000|USA
10003|A3|NSS|30000|IND
10004|A4|SEC|40000|USA
10005|A5|TECH|50000|IND
10006|A6|TECH|60000|IND
10007|A7|TECH|70000|IND

1. Display complete file

$ awk '{print}' F_Data_File.txt
(or)
$ awk '{print $0}' F_Data_File.txt

2. Print only first field from the file

awk '{print $1}' F_Data_File.txt

Output:

EMPID|EMPNAME|EMPDEPT|EMPSAL|LOCATION
10001|A1|HR|10000|USA
10002|A2|FIN|20000|USA
10003|A3|NSS|30000|IND
10004|A4|SEC|40000|USA
10005|A5|TECH|50000|IND
10006|A6|TECH|60000|IND
10007|A7|TECH|70000|IND

Has printed complete file as we have not provided any delimiter(default delimiter is SPACE).How to give delimiter? Let see.

$ awk -F"|"  '{print $1}' F_Data_File.txt

Check -F switch, this switch is used to provide field separator. Above command will print only first field from the pipe delimited file.

Output:
EMPID
10001
10002
10003
10004
10005
10006
10007

3. print 1st & 2nd field separated by space

$ awk -F"|"  '{print $1 " " $2}' F_Data_File.txt

4. print 1st & 2nd field separated by <==>

$ awk -F"|"  '{print $1 "<==>" $2}' F_Data_File.txt

4. How to Initialize field Separator

$ awk 'BEGIN{FS="|";OFS=",";}{print $1 OFS $2 OFS $3 OFS $4 OFS $5}'  F_Data_File.txt

Here we have Initialized Input field separator (FS) and output Field Separator(OFS)
The output will be displayed as comma(,) separated.

Output:

EMPID,EMPNAME,EMPDEPT,EMPSAL,LOCATION
10001,A1,HR,10000,USA
10002,A2,FIN,20000,USA
10003,A3,NSS,30000,IND
10004,A4,SEC,40000,USA
10005,A5,TECH,50000,IND
10006,A6,TECH,60000,IND
10007,A7,TECH,70000,IND

5. Print row number in front of every record

$ awk 'BEGIN{FS="|";}{print NR ":" $0}'  F_Data_File.txt

Output:

1:EMPID|EMPNAME|EMPDEPT|EMPSAL|LOCATION
2:10001|A1|HR|10000|USA
3:10002|A2|FIN|20000|USA
4:10003|A3|NSS|30000|IND
5:10004|A4|SEC|40000|USA
6:10005|A5|TECH|50000|IND
7:10006|A6|TECH|60000|IND
8:10007|A7|TECH|70000|IND

6. Calculate Number of Field from every row

$ awk 'BEGIN{FS="|";}{print "Number of Field in Row:" NR " is=>" NF}'  F_Data_File.txt

Output:

Number of Field in Row:1 is=>5
Number of Field in Row:2 is=>5
Number of Field in Row:3 is=>5
Number of Field in Row:4 is=>5
Number of Field in Row:5 is=>5
Number of Field in Row:6 is=>5
Number of Field in Row:7 is=>5
Number of Field in Row:8 is=>5

7. Print first & Last field from the file

$ awk 'BEGIN{FS="|"}{print $1 "-->" $NF}'  F_Data_File.txt

Output:

EMPID-->LOCATION
10001-->USA
10002-->USA
10003-->IND
10004-->USA
10005-->IND
10006-->IND
10007-->IND


8. Print only First line of the file

$ awk 'BEGIN{FS="|";}NR==1{print $0}'  F_Data_File.txt

Output:

EMPID|EMPNAME|EMPDEPT|EMPSAL|LOCATION

9. Read file from 3rd row

$ awk 'BEGIN{FS="|";}NR>=3{print $0}'  F_Data_File.txt

Output:

10002|A2|FIN|20000|USA
10003|A3|NSS|30000|IND
10004|A4|SEC|40000|USA
10005|A5|TECH|50000|IND
10006|A6|TECH|60000|IND
10007|A7|TECH|70000|IND

10. Read line in multiple of 3 i.e. 3rd,5th,8th so on

$ awk 'BEGIN{FS="|";}NR%3==0{print $0}'  F_Data_File.txt

Output:

10002|A2|FIN|20000|USA
10005|A5|TECH|50000|IND

11. Print LAST line of the File

$ awk 'END{print}'  F_Data_File.txt

12. Count number of Rows in a file (wc-l F_Data_File.txt)

$ awk -F"|" '{V_Row_Cnt++}END{print V_Row_Cnt}' F_Data_File.txt
(or)
$ awk 'END { print NR }' F_Data_File.txt

Output: 8

13. Calculate total salary of the employees:

$ awk -F"|" 'NR>1{V_Sum_Sal=$4 + V_Sum_Sal }END{print V_Sum_Sal}' F_Data_File.txt

Output: 280000

NR>1 as first row is header.

14. Count empty lines from the file.

$ awk 'NF==0{print NR":"}' F_Data_File.txt
(or)
$ awk '/^$/{print NR":"}' F_Data_File.txt

Output: Will display the line number which are empty
5:
7:

NF stands for number of fields, if number of fields are 0 means row does not have any data. ^ represent start of the line and $ represent end of line, and if there is nothing in between 6 and $ means line is empty.

$ awk 'NF==0{V_Count++}{print V_Count}' F_Data_File.txt
(or)
$ awk '/^$/{V_Count++}{print V_Count}' F_Data_File.txt

Output: 2

15. Remove Empty lines from the file.

$ awk 'NF' F_Data_File.txt
$ awk 'NF > 0' F_Data_File.txt
$ awk '!NF==0{print NR":"}' F_Data_File.txt
$ awk '!/^$/{print NR":"}' F_Data_File.txt
$ awk '/./{print NR":"}' F_Data_File.txt

Output: All the above command will display non-empty lines available in the file.

How to Calculate RANK Using AWK

Input File: F_Data_File.txt

10001|A1|TRANS|Y|10000
10002|A2|MED|Y|20000
10003|A3|FIN|N|10000
10004|A4|HR|Y|20000
10005|A5|CSR|N|50000
10006|A6|TRANS|Y|30000
10007|A7|FIN|N|40000

Desired: We want to give rank based on the salary of employees, same salary employee should get same rank.

10005|A5|CSR|N|50000|1
10007|A7|FIN|N|40000|2
10006|A6|TRANS|Y|30000|3
10004|A4|HR|Y|20000|4
10002|A2|MED|Y|20000|4
10003|A3|FIN|N|10000|5
10001|A1|TRANS|Y|10000|5

Solution:

sort -nr -k5 -t"|" F_Data_File.txt |\

awk -F"|" '{

   if(V_Temp != $5)

 {k++ #--*k=k+1

 }

 {V_Temp=$5}

 }

 {print $0 "|" k

}'

(or)

sort -nr -k5 -t"|" F_Data_File.txt | \

awk -F"|" '{

if(V_Temp!=$5)

{k++ #--*k=k+1

}

{V_Temp=$5

}

}

{print $0 FS k}';

(or) #--If you want a header in output then we need to initialize the same in BEGIN.

sort -nr -k5 -t"|" F_Data_File.txt | \

awk -F"|" 'BEGIN{print "EMPID|ENAME|DEPT|FLAG|SALA|RANK"}

          {

            if(V_Temp!=$5)

            {k++ #--*k=k+1

            }

            {V_Temp=$5

            }

         }

         {print $0 FS k}';

Note: Observe how rank is being populated, once using concatenation and one using inbuilt function FS.

Explanation:

1. sort -nr -k5 -t"|", sort function sort the data based on key field 5 that is salary column.

2. Once data is sorted same is piped to AWK command for further processing.
3. When AWK process first row, it will check whether TEMP column V_Temp is equal to $5 (Salary) as at initial level TEMP column will not hold anything, this condition will be TRUE and will set the value of k as 1.
4. For 2nd row V_TEMP will have salary of previous row hence if second record also have same salary then if condition will fail and k will not increment. It will set the same value of k to the second record as well. of if has different salary then it will increment the value of k to 2 and assign it to 2nd row.
5. Same will be followed till the end.

How to Split File Dynamically (Part 2) using AWK

In previous Post, How to Split File Dynamically using AWK, We have seen how to split file dynamically using AWK. In this article we will see how to split file dynamically and keep header & footer in each created file.

Input File: F_Data_File.txt

EMPID|ENAME|DEPT|FLAG
10001|A1|TRANS|Y
10002|A2|MED|Y
10003|A3|FIN|N
10004|A4|HR|Y
10005|A5|CSR|N
10006|A6|TRANS|Y
10007|A7|FIN|N

1. Split the file dynamically and keep Header & Footer in all the newly created files.

$ V_Header="EMPID|ENAME|DEPT|FLAG"
$ V_Footer="/*EOF*/"

$ awk -F"|" -v x=$V_Header -v y=$V_Footer  '{V_FileName=$3; V_Curr_FileName =$3;
if(V_Curr_FileName != V_Prev_FileName) 
{ 
if(V_Flag==1)
{
#--* To add the footer in each file except which will be creatd in last.
print y > V_Prev_FileName; 
}
#--* To add the headed in each file.
print x > V_FileName;
}
else{
V_Flag=1;
} 
print $0 > V_FileName; 
V_Prev_FileName=$3;
}END{
#--* To add the footer in last created file.
print y > V_FileName;
}' F_Data_File.txt

Output:

Above Unix command will generate five files based on department same as in previous post but here it will add header & footer in each created file.

2. Split the file dynamically and keep only Header in all the newly created files.

$ V_Header="EMPID|ENAME|DEPT_FLAG"
$ awk -F"|" -v x=$V_Header '{V_FileName=$3; V_Curr_FileName=$3;
 if(V_Curr_FileName != V_Prev_FileName) 
 { 
            #--* To add the headed in each file. 
    print x > V_FileName;
 }
   print $0 > V_FileName; 
   V_Prev_FileName=$3;

}' F_Data_File.txt

Output:

Above Unix command will generate five files based on department same as in previous post but here it will add only header in each created file.

Explanation:

1. First department name will be stored in variables V_FileName & V_Curr_FileName
2. In If block we are comparing  V_Curr_FileName & V_Prev_FileName at very first time, V_Prev_FileName will have NULL value and hence condition will be TRUE and it will go inside IF block.
3. It will then check for V_Flag, at very first time this flag value will be NULL hence condition will be FALSE and it will come out from If block.
4. It will add header in the file whose name is stored in V_FileName.
5. it will then print entire line $0 in file and assign the file name to V_Prev_FileName which will help to redirect all the record of same department in same file.
6. Same process will be followed until last line of the file reached and END block footer will be added in last generated file as END block executes in the last.

Note: Same process will be followed for 2nd scenario as well only footer part will not be applicable.

How to Split File Dynamically using AWK

Input File: F_Data_File.txt

10001|A1|TRANS|Y
10002|A2|MED|Y
10003|A3|FIN|N
10004|A4|HR|Y
10005|A5|CSR|N
10006|A6|TRANS|Y
10007|A7|FIN|N

Desired Output: 

1. Here we want to create files based on Department, i.e. we want to have all the employee of same department in a single file named with that department only.

For example, user 10001 & 10002 should be available in a single file name in the fomr F_<DEPT_NAME>.txt hence for dept Finance file name should be F_FIN.txt

2. Generate files dynamically based on ACTIVE or Inactive  status, i.e. all the active employees in a file and all the inactive in another file. File name should follow the following naming F_<Y|N>.txt

Solution 1:

$ awk -F"|" 'NR>1{print > "F_"$3".txt"}'  F_Data_File.txt
(or)
$ awk 'NR>1{V_FileName="F_"$1".txt"; print $0 > V_FileName;}' F_Data_File.txt

NR>1 because we don't want to use header line in AWK processing.

Output:

it will create five files, each for one department and will have employee records associated to that department.

F_TRANS.txt
F_MED.txt
F_FIN.txt
F_HR.txt
F_CSR.txt

Explanation:

1. NR>1 ignore the first line which is header line of the base file.
2. Every time when AWK reads a line it re-direct it to the corresponding file internally.

Solution 2:

$ awk -F"|" 'NR>1{if($4=="Y") print > "F_"$4".txt"; else print > "F_"$4".txt" }' F_Data_File.txt
(or)
$ awk -F"|" 'NR>1{V_FileName="F_"$4".txt"; if($4=="Y") print $0 > V_FileName; else print $0> V_FileName }' F_Data_File.txt

NR>1 because we don't want to use header line in AWK processing.

Explanation:

1. NR>1 ignore the first line which is header line of the base file.
2. Here if block is used to check whether $4(FLAG) Y or N, ased on Y & N AWK redirects the processing rows into associated files.
3. Every time when AWK reads a line it re-direct it to the corresponding file internally.

How to Remove Duplicate Records in File Using AWK

In Previous post, How To Find Duplicate Records in File Using AWK, we have seen how to find duplicate records, here we will see how to remove duplicate records using AWK command.

Input File: F_Data_File.txt

EMPID|ENAME|DEPT|FLAG
10001|A1|TRANS|Y
10002|A2|MED|Y
10003|A3|FIN|N
10004|A4|HR|Y
10005|A5|CSR|N
10001|A1|TRANS|Y
10003|A3|FIN|N

$ awk '!($0 in V_Uniq_Rec) {V_Uniq_Rec[$0];print}' F_Data_File.txt

Output:
10001|A1|TRANS|Y
10002|A2|MED|Y
10003|A3|FIN|N
10004|A4|HR|Y
10005|A5|CSR|N

Explanation:

It reads, If a record ($0) is not available in array V_Uniq_Rec then return true and store it in array V_Uniq_Rec. If it is already available in array V_Uniq_Rec then it will return FALSE and will not store in array and will be eliminated. This command will not change the file data, it will display the unique records in console.

Another way of writing above AWK command is which is very common and used frequently to remove duplicate.

$ awk '!V_Uniq_Rec[$0]++' F_Data_File.txt

Output:
10001|A1|TRANS|Y
10002|A2|MED|Y
10003|A3|FIN|N
10004|A4|HR|Y
10005|A5|CSR|N

How To Find Duplicate Records in File Using AWK

Input File: F_Data_File.txt

EMPID|ENAME|DEPT|FLAG
10001|A1|TRANS|Y
10002|A2|MED|Y
10003|A3|FIN|N
10004|A4|HR|Y
10005|A5|CSR|N
10001|A1|TRANS|Y
10003|A3|FIN|N

1. Find out the duplicate record available in the file.

$ awk  '{V_Dup_Count[$0]++} END{for(k in V_Dup_Count) {print k,"Rec_Count=>",V_Dup_Count[k]} }' F_Data_File.txt | awk -F" " '$3 > 1{print $0}'

10001|A1|TRANS|Y Rec_Count=> 2
10003|A3|FIN|N Rec_Count=> 2

Explanation: 

V_Dup_Count holds the count of record associated that record which is available as INDEX to array V_Dup_Count.

V_Dup_Count[10001|A1|TRANS|Y] = 2
V_Dup_Count[10003|A3|FIN|N] = 2

Index value in red is k of for loop and 2 is V_Dup_Count[k]. Please follow post How to Use Associative Array in AWK

We have passed the output of first AWK to second AWK command which will receive 3 fields as input: 1st is complete record, 2nd is Rec_Count=> and 3rd field is count of record all are separated by space. If 3rd field value > 1 that means record is duplicate. 

2. Find out duplicate record based on key column, e.g. based on 1st column:

$ cat F_Data_File.txt | cut -d"|" -f1 | awk  '{V_Dup_Count[$0]++} END{for(k in V_Dup_Count) {print k,"Rec_Count=>",V_Dup_Count[k]} }' | awk -F" " '$3 > 1{print $0}'

(or)

awk -F"|" '{V_Dup_Count[$1]++} END{for(k in V_Dup_Count) {print k,"Rec_Count=>",V_Dup_Count[k]} }' 1.txt | awk -F" " '$3 > 1{print $0}'

10001 Rec_Count=> 2
10003 Rec_Count=> 2

Explanation: 

Same as in case 1 but here we are generating the array V_Dup_Count based on field 1 of the record.

3. Find out duplicate record based on key column, e.g. based on 1st & 2nd column:

$ cat F_Data_File.txt | cut -d"|" -f1,2 | awk  '{V_Dup_Count[$0]++} END{for(k in V_Dup_Count) {print k,"Rec_Count=>",V_Dup_Count[k]} }' | awk -F" " '$3 > 1{print $0}'

(or)

$ awk -F"|" '{V_Dup_Count[$1"|"$2]++} END{for(k in V_Dup_Count) {print k,"Rec_Count=>",V_Dup_Count[k]} }' 1.txt | awk -F" " '$3 > 1{print $0}'


10001|A1 Rec_Count=> 2
10003|A3 Rec_Count=> 2

Explanation: 

Same as in case 1 but here we are generating the array V_Dup_Count based on field 1 & 2 of the record.

How To Look-Up (Part 3) in File Data Using AWK

Main Input File: F_Data_File1.txt

EMPID|ENAME|DEPT|FLAG
10001|A1|TRANS|Y
10002|A2|MED|Y
10003|A3|FIN|N
10004|A4|HR|Y
10005|A5|CSR|N
10006|A6|TRANS|Y
10007|A7|FIN|Y
10008|A8|HR|N

Look-Up File : F_Data_File2.txt

Y|ACTIVE
N|INACTIVE

Desired Output: We want to update the Value associated with FLAG Y & N with Active or Inactive accordingly

10001|A1|TRANS|ACTIVE

10002|A2|MED|ACTIVE

10003|A3|FIN|INACTIVE

10004|A4|HR|ACTIVE

10005|A5|CSR|INACTIVE

10006|A6|TRANS|ACTIVE

10007|A7|FIN|ACTIVE

10008|A8|HR|INACTIVE

Solution:

$ awk 'BEGIN {FS=OFS="|"}FNR==NR{V_LkUp[$1]=$2;next}($4 in V_LkUp)  {print $1,$2,$3,V_LkUp[$4]}' F_Data_File2.txt F_Data_File1.txt

$ awk 'BEGIN {FS=OFS="|"}FNR==NR{V_LkUp[$1]=$2;next}($4 in V_LkUp)  {print $1,$2,$3,V_LkUp[$4]"["$4"]" }'  F_Data_File2.txt F_Data_File1.txt

Will produced below Output:

10001|A1|TRANS|ACTIVE[Y]

10002|A2|MED|ACTIVE[Y]

10003|A3|FIN|INACTIVE[N]

10004|A4|HR|ACTIVE[Y]

10005|A5|CSR|INACTIVE[N]

10006|A6|TRANS|ACTIVE[Y]

10007|A7|FIN|ACTIVE[Y]

10008|A8|HR|INACTIVE[N]

Explanation:

1. Array V_LkUp holds the LookUp table data. In loop $4, 4th field of Main file is lookuped     in array V_LkUp. Whenever Y is found it returns ACTIVE or else INACTIVE.

2. If you have not checked other post on Look-Up please go through 

• How To Look-Up in File Data Using AWK
• How To Look-Up (Part 2) in File Data Using AWK

How To Look-Up (Part 2) in File Data Using AWK

I have already explained about how to lookup in flat file using AWK in earlier post, for more detail/explain follow How To Look-Up in File Data Using AWK

Input Data : F_Data_File1.txt

EMPID
10001
10002
10003
10004
10008

Input Data : F_Data_File2.txt

EMPID|ENAME|DEPT
10001|A1|TRANS
10002|A2|MED
10003|A3|FIN
10004|A4|HR
10005|A5|CSR
10006|A6|TRANS
10007|A7|FIN
10008|A8|HR

Desired Output: We need to fetch the deails of the records which are not available in F_Data_File1.txt from file F_Data_File2.txt

EMPID|ENAME|DEPT
10005|A5|CSR
10006|A6|TRANS
10007|A7|FIN

Solution:

$ awk 'NR==FNR && FNR !=1 {A[$1];next} !($1 in A)' FS="|" F_Data_File1.txt F_Data_File2.txt

How to Use Shell Variable in AWK

The -v option of AWK can be used to pass shell variables to awk command.

If we use V_user directly inside AWK, it will not work and we will not get what we want

$ V_User="Champ"
$ awk 'BEGIN{print "Hello=>" $V_User}'

Output:Hello=>

$ V_User="Champ"
$ awk -v x=${V_User} 'BEGIN{print "Hello=>" x}'

Output: Hello=>Champ

If we want to use multiple shell variables inside AWK below code will do it

$ V_User="Champ"
$ V_Msg="Welcome"

$ awk -v x=$V_Msg -v y=$V_User 'BEGIN{ print x, y}'

Output: Welcome Champ

$ V_Msg="Hello"
$ V_User="Champ"
$ awk 'BEGIN{x='$V_Msg';y='$V_User'} END{print x,y}' /dev/null
$ echo "" | awk 'BEGIN{x='$V_Msg';y='$V_User'} END{print x,y}'

Output: Hello Champ

$ awk 'END{print x,y}' x=$V_Msg y=$V_User /dev/null
$ echo "" | awk 'END{print x,y}' x=$V_Msg y=$V_User

Output: Hello Champ

Common Functions in AWK

LENGTH: Return the length of the string

$ echo "AWK Tutorial" | awk '{print length($0)}';

Output: 12

INDEX : Returns the position of the required in string.

syntax : index(<string>,<string to be find>)

$ V_Text_String="You are learning AWK functions";

$ echo $V_Text_String | awk '{print index ($0,"AWK")}'

Output: 18

SUBSTR : returns a part of the string

syntax   : substr(<string>,<position>,<length>)

$ V_Text_String="You are learning AWK functions";

$ echo $V_Text_String | awk '{print substr($0,1)}'

Output: You are learning AWK functions

function has returned complete string starting from 1st position

$ echo $V_Text_String | awk '{print substr($0,2)}'

Output: ou are learning AWK functions

function has returned complete string starting from 2nd position

$ echo $V_Text_String | awk '{print substr($0,1,8)}'

Output: You are 

function has returned string starting from 1st position to 8th position

SPLIT : It is used to split the string

syntax: split(<string>,<array>,<delimiter>)

$ V_Text_String="Apple,Nokia,Samsung,MI";

$ echo $V_Text_String | awk '{print split($0,V_Prod,",")}'

Output: 4 ( basically returned number of field separated by  comma"," or We can say the length of the array V_Prod)

$ echo $V_Text_String | awk 'split($0,V_Prod,",") {print V_Count} END{for(k in V_Prod) print k, V_Prod[k] }'

Output:

1 Apple

2 Nokia

3 Samsung

4 MI

Note: split() function is used to create Associative Arrays.

TOLOWER : Convert string into lower case
syntax       : <tolower(<string>)

$ V_Text_String="Apple,Nokia,Samsung,MI";
$ echo $V_Text_String | awk '{print tolower($0)}'

Output: apple,nokia,samsung,mi

TOUPPER  : Convert string into upper case

syntax       : <tolower(<string>)

$ V_Text_String="Apple,Nokia,Samsung,MI";
$ echo $V_Text_String | awk '{print toupper($0)}'

Output: APPLE,NOKIA,SAMSUNG,MI