Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Background: It is a temporary measure to load some test data into a test environment (a Sybase database).

I have a bunch of files I have to import into database daily. These files are organized as such

30Aug2011/IMPORT_ME.txt
31Aug2011/IMPORT_ME.txt
...
8Sep2011/IMPORT_ME.txt
9Sep2011/IMPORT_ME.txt

After each import I need to run some special sql statement to fix up the date. This is what I did in the main script

import_file.sh 20Aug2011/IMPORT_ME.txt
cat rerun_import_file.tmpl | sed -e "s/XXX/8 Aug 2011/g" > rerun_import_files.sql
$ISQL -i rerun_import_files.sql

...
import_file.sh 9Sep2011/IMPORT_ME.txt
cat rerun_import_file.tmpl | sed -e "s/XXX/9 Sep 2011/g" > rerun_import_files.sql
$ISQL -i rerun_import_files.sql

So I suppose I can make it better by:

  1. find all directories that fit the date pattern

  2. sort the dates correctly (very important to the setup)

  3. parse the date and set the date according in the sql file

However my bash script-fu is not good enough to do it.

share|improve this question

1 Answer 1

up vote 2 down vote accepted

Of course, I missed the sorting part of the question. To answer that, I propose a different model that's simpler in some ways:

ls */IMPORT_ME.txt | cut -d/ -f 1 | date --file=- +%s | sort -n | sed -e 's/^/@/' \
| while read DATE
    do
        FILENAME=`date -d ${DATE} +%-d%b%Y/IMPORT_ME.txt`
        REPLACEMENT=`date -d ${DATE} +%-d %b %Y`
        import_file.sh ${FILENAME}
        cat rerun_import_file.tmpl | sed -e "s/XXX/${REPLACEMENT}/g" > rerun_import_files.sql
        $ISQL -i rerun_import_files.sql
    done

The key here is the initial conversion of all of the (filtered for validity) directory names to a sortable time format (in this case Unix 'epoch' time), sorting them, and then converting them back (to two different forms).

If you're working with dates in any more than a purely string manipulation sense, I strongly recommend reading the man page for 'date'. In fact, read it twice, think hard about it, then read it again. 'date' is an extremely powerful command in its domain.

Oh, and I hope the lesson you have learned from this experience is that you should ALWAYS use ISO format dates (date -Iseconds or date -I for examples). ISO format dates sort well as strings using e.g. sort.

share|improve this answer
    
Thanks for the great answer! Last remaining piece of puzzle: how can I ensure that the script will iterate these IMPORT_ME.txt files in correct chronological order as suggested by the directory name? I don't wanna see 1Sep comes after 1Aug instead of 2Aug for example. –  Anthony Kong Sep 12 '11 at 5:20
    
Oh. That makes it simpler in some ways. See completely changed answer above. –  Slartibartfast Sep 13 '11 at 5:31
    
@Slartibarfast: Awesome solution! Thanks! –  Anthony Kong Sep 14 '11 at 0:12

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.