0

I have a bunch of *.csv files that use headers. Right now the files have headers and the same number of lines of data. I need to erase the data in these files, so I figured I could just overwrite the header into the files. I have file A set up properly with the header. How can I cat A into all the other csv files?

Alternatively, how could I delete the last X number of lines from the files? The path for the files is */*.csv

2 Answers 2

1
  1. Cat the header into each of the files. I am assuming that A is a file since you mentioned cat. If so, you can just copy:

    for file in */*.csv; do cp A "$file"; done
    

    If A is a variable, do

    for file in */*.csv; do echo "A" > "$file"; done
    

    Alternatively, if all you need is the first line, you can cat that into each file. That way, it will also work for different headers:

    for file in */*.csv; do head -n "$file" > /tmp/foo && 
      mv /tmp/foo "$file"; 
    done
    
  2. There are many ways to delete the last X number of lines from the files. Here are three:

    • head, change X to however many lines you want to keep:

      for file in */*.csv; do
        head -n X $file > /tmp/foo && mv /tmp/foo "$file"; 
      done
      
    • awk, change the 123 to however many lines you want to keep:

      for file in */*.csv; do
        awk 'NR<=123' "$file" > /tmp/foo && mv /tmp/foo "$file"; 
      done
      
    • perl, change the 123 to however many lines you want to keep:

      for file in */*.csv; do
        perl -ne "next if $.>123;print' "$file" > /tmp/foo && 
        mv /tmp/foo "$file"; 
      done
      
1
  • To delete the last X lines I found it easiest to use sed where <N> is the first line you want to delete and $ is used to specify the last line: sed -i '<N>,$d'. For example, to delete from the 2nd line to the end: sed -i '2,$d'.
    – Stuart
    Oct 6, 2014 at 17:58
1

You can use sed and some pipe creativity to convert a list of files (or anything) into a list of commands.

This is a test, it will just output the commands for you to see what is going to happen:

$ ls *.cvs | sed "s/.*/cat headerfile.cvs > &/"

If you like it, do the same and pipe it to your favorite shell:

$ ls *.cvs | sed "s/.*/cat headerfile.cvs > &/" | bash

Or for a recursive solution you can use find:

find . -name \*.cvs | sed "s/.*/cat headerfile.cvs > &/" | bash

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .