Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I want to compare two files of around 40 MB of comma separated values with lines like this:

hstar,default,"T9883Z ",0d59,c801,7332,5,20120914,4.343618767

For the two files, the last entry which is 4.343618767 in the above example varies between the two files, but almost all the other fields match identically.

I need to diff the two files to locate the few places where the entries other than the last vary between the two files.

I'm thinking the easiest way to do this is to use SED to process the two files and normalize the last field, looking for the number pattern after the seventh comma and replacing it with a fixed string like 9.999999999 on every line and then a simple diff will work.

However, I'm not sure how to construct a sed command to locate the seventh comma and replace the remaining string to the end of the line with a fixed string. What would such a sed command look like? I imagine I would need to use a regular expression but am not sure how to start the pattern after the seventh comma.

share|improve this question

3 Answers 3

up vote 3 down vote accepted

You do not have to look for the seventh column. Just go for the last one:

sed 's/,[^,]*$/,9.9999999999/'

Explanation:

,    match the comma
[    beginning of a character group
 ^   negation, i.e. do not match the following characters
 ,   comma
]    end of a character group
*    repeat the preceding thing zero or more times
$    match the end of line
share|improve this answer
    
@WilliamKF: See update. –  choroba Sep 17 '12 at 14:24
    
Thanks for the detailed explanation! –  WilliamKF Sep 17 '12 at 14:34

sed "s/,[0-9].[0-9]\+\$//" <yourfile> will output lines like this:

hstar,default,"T9883Z ",0d59,c801,7332,5,20120914

share|improve this answer

Awk is a much cleaner tool for processing tabular data:

awk -F ',' ' { print $1,$2,$3,$4,$5,$6 }' file1 > temp1
awk -F ',' ' { print $1,$2,$3,$4,$5,$6 }' file2 > temp2
diff temp1 temp2
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.