0

I have files like raw_data_0, raw_data_1... raw_data_7 every file contains below data

record_count=1
emailCountFailure=0
record_count=1
emailCountFailure=0
record_count=1
emailCountFailure=0
record_count=1
emailCountFailure=0
record_count=1
emailCountFailure=0
record_count=1
emailCountFailure=0
record_count=1
emailCountFailure=0
record_count=1
emailCountFailure=0

I need to open all the files and count the how many record_count and emailCountFailure is there in each file and finally sum all the count.

I tried the below code to iterate through all the file and tried to count by using grep. but output is not as expected(below).

#!/bin/ksh
MY_WORK='/a585/app/data/CCN_text'
raw_data_file_count=`ls -l raw_data_*|wc -l`
echo " $raw_data_file_count"
#raw_data_file_counter=(`ls -l raw_data_* | awk -F" " '{print $9}' | awk -F"_" '{print $3}'`)

ls -l raw_data_* | awk -F" " '{print $9}' | awk -F"_" '{print $3}' > file_counter.txt

while read line
do

counter_array+=("$line")

done < file_counter.txt


raw_data_file_counter_length=${#counter_array[@]}

echo "ARRAY LENGTH : ${raw_data_file_counter_length}"

for ((count=0; count<${raw_data_file_counter_length}; count++));
do
echo " COUNTER VALUE: ${counter_array[$count]} "
reccount="$(grep 'record_count' $MY_WORK/raw_data_${counter_array[$count]})"

echo "$reccount"

done

output:

 8
ARRAY LENGTH : 8
 COUNTER VALUE: 0
record_count=1
 COUNTER VALUE: 1
record_count=1
record_count=2
 COUNTER VALUE: 2
record_count=1
record_count=2
 COUNTER VALUE: 3
record_count=1
 COUNTER VALUE: 4
record_count=1
record_count=2
 COUNTER VALUE: 5
record_count=1
record_count=2
 COUNTER VALUE: 6
record_count=1
record_count=2
 COUNTER VALUE: 7
record_count=1

I need to sum all the "record_count" count from all the files. Is there any way to achive that?

1 Answer 1

0

You can use the -c flag of grep to count occurrences of a string in a file and loop through all of them.

#!/usr/bin/env bash

rc="0"
ecf="0"

for f in ./raw_data_*; do
    let "rc=rc+$(cat $f | grep -c 'record_count')"
    let "ecf=ecf+$(cat $f | grep -c 'emailCountFailure')"
done

echo "record_count = ${rc}"
echo "emailCountFailure = ${ecf}"

Result:

record_count = 24
emailCountFailure = 24
4
  • This is a nice answer, but you made "useless use of cat". Better would be grep pattern file instead of cat file | grep pattern.
    – user596332
    Feb 13, 2020 at 20:38
  • You are right, maybe out of habit.
    – lol
    Feb 14, 2020 at 5:58
  • thank you that worked, i am using without cat command Feb 14, 2020 at 9:05
  • @saiprudhvi If that solves your problem, please mark it as accepted by clicking on the check sign.
    – user596332
    Feb 14, 2020 at 9:23

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .