How to Split File Dynamically (Part 2) using AWK

In previous Post, How to Split File Dynamically using AWK, We have seen how to split file dynamically using AWK. In this article we will see how to split file dynamically and keep header & footer in each created file.

Input File: F_Data_File.txt

EMPID|ENAME|DEPT|FLAG
10001|A1|TRANS|Y
10002|A2|MED|Y
10003|A3|FIN|N
10004|A4|HR|Y
10005|A5|CSR|N
10006|A6|TRANS|Y
10007|A7|FIN|N

1. Split the file dynamically and keep Header & Footer in all the newly created files.

$ V_Header="EMPID|ENAME|DEPT|FLAG"
$ V_Footer="/*EOF*/"

$ awk -F"|" -v x=$V_Header -v y=$V_Footer  '{V_FileName=$3; V_Curr_FileName =$3;
if(V_Curr_FileName != V_Prev_FileName) 
{ 
if(V_Flag==1)
{
#--* To add the footer in each file except which will be creatd in last.
print y > V_Prev_FileName; 
}
#--* To add the headed in each file.
print x > V_FileName;
}
else{
V_Flag=1;
} 
print $0 > V_FileName; 
V_Prev_FileName=$3;
}END{
#--* To add the footer in last created file.
print y > V_FileName;
}' F_Data_File.txt

Output:

Above Unix command will generate five files based on department same as in previous post but here it will add header & footer in each created file.

2. Split the file dynamically and keep only Header in all the newly created files.

$ V_Header="EMPID|ENAME|DEPT_FLAG"
$ awk -F"|" -v x=$V_Header '{V_FileName=$3; V_Curr_FileName=$3;
 if(V_Curr_FileName != V_Prev_FileName) 
 { 
            #--* To add the headed in each file. 
    print x > V_FileName;
 }
   print $0 > V_FileName; 
   V_Prev_FileName=$3;

}' F_Data_File.txt

Output:

Above Unix command will generate five files based on department same as in previous post but here it will add only header in each created file.

Explanation:

1. First department name will be stored in variables V_FileName & V_Curr_FileName
2. In If block we are comparing  V_Curr_FileName & V_Prev_FileName at very first time, V_Prev_FileName will have NULL value and hence condition will be TRUE and it will go inside IF block.
3. It will then check for V_Flag, at very first time this flag value will be NULL hence condition will be FALSE and it will come out from If block.
4. It will add header in the file whose name is stored in V_FileName.
5. it will then print entire line $0 in file and assign the file name to V_Prev_FileName which will help to redirect all the record of same department in same file.
6. Same process will be followed until last line of the file reached and END block footer will be added in last generated file as END block executes in the last.

Note: Same process will be followed for 2nd scenario as well only footer part will not be applicable.