Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Background: It is a temporary measure to load some test data into a test environment (a Sybase database).

I have a bunch of files I have to import into database daily. These files are organized as such


After each import I need to run some special sql statement to fix up the date. This is what I did in the main script 20Aug2011/IMPORT_ME.txt
cat rerun_import_file.tmpl | sed -e "s/XXX/8 Aug 2011/g" > rerun_import_files.sql
$ISQL -i rerun_import_files.sql

... 9Sep2011/IMPORT_ME.txt
cat rerun_import_file.tmpl | sed -e "s/XXX/9 Sep 2011/g" > rerun_import_files.sql
$ISQL -i rerun_import_files.sql

So I suppose I can make it better by:

  1. find all directories that fit the date pattern

  2. sort the dates correctly (very important to the setup)

  3. parse the date and set the date according in the sql file

However my bash script-fu is not good enough to do it.

share|improve this question
up vote 2 down vote accepted

Of course, I missed the sorting part of the question. To answer that, I propose a different model that's simpler in some ways:

ls */IMPORT_ME.txt | cut -d/ -f 1 | date --file=- +%s | sort -n | sed -e 's/^/@/' \
| while read DATE
        FILENAME=`date -d ${DATE} +%-d%b%Y/IMPORT_ME.txt`
        REPLACEMENT=`date -d ${DATE} +%-d %b %Y` ${FILENAME}
        cat rerun_import_file.tmpl | sed -e "s/XXX/${REPLACEMENT}/g" > rerun_import_files.sql
        $ISQL -i rerun_import_files.sql

The key here is the initial conversion of all of the (filtered for validity) directory names to a sortable time format (in this case Unix 'epoch' time), sorting them, and then converting them back (to two different forms).

If you're working with dates in any more than a purely string manipulation sense, I strongly recommend reading the man page for 'date'. In fact, read it twice, think hard about it, then read it again. 'date' is an extremely powerful command in its domain.

Oh, and I hope the lesson you have learned from this experience is that you should ALWAYS use ISO format dates (date -Iseconds or date -I for examples). ISO format dates sort well as strings using e.g. sort.

share|improve this answer
Thanks for the great answer! Last remaining piece of puzzle: how can I ensure that the script will iterate these IMPORT_ME.txt files in correct chronological order as suggested by the directory name? I don't wanna see 1Sep comes after 1Aug instead of 2Aug for example. – Anthony Kong Sep 12 '11 at 5:20
Oh. That makes it simpler in some ways. See completely changed answer above. – Slartibartfast Sep 13 '11 at 5:31
@Slartibarfast: Awesome solution! Thanks! – Anthony Kong Sep 14 '11 at 0:12

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .