2

I have very little knowledge about shell scripts, but unfortunately I have to write one. I want to ask about bash script iteration moving files, I need to move log files sorted by month which will be executed by cronjob. The plan was to move mtime +30 (1 month before) files into several folders and the cronjob will be executed daily e.g:

BEFORE

/home/Work/LogFiles/20131200012.log
/home/Work/LogFiles/thisLogIsDifferent.log
/home/Work/LogFiles/20120322222.log 
/home/Work/LogFiles/20140100011.log
/home/Work/LogFiles/thisLogIsDifferent2.log

AFTER

/home/Work/LogFiles/thisLogIsDifferent.log
/home/Work/LogFiles/thisLogIsDifferent2.log
/home/Work/LogFiles/2013/DEC/20131200012.log
/home/Work/LogFiles/2012/MAR/20120322222.log 
/home/Work/LogFiles/2014/JAN/20140100011.log

which I haven't get any clue the methods I had to use. So here's my awful shell script:

BASE_DIR=/home/Work/LogFiles
REPORT_DIR_YEAR=$BASE_DIR/`date +%Y`
REPORT_DIR=$REPORT_DIR_YEAR/`date +%b`

NOW=$(date +"%Y%m")

if ! [ -d $REPORT_DIR_YEAR ]; then
    mkdir $REPORT_DIR_YEAR

    if ! [ -d $REPORT_DIR ]; then
        mkdir $REPORT_DIR
    fi
fi

#THIS PART NEED TO BE RE-ARRANGED
#What I expect is not date=NOW; BUT SOME KIND LIKE date %m-1? but I still don't have any ideas about modify date function.

for file in find $BASE_DIR -maxdepth 1 -type f -mtime +30 -name '*$NOW*'
do

 month=$(ls -l $file | awk '{ print $6 }')
    case "$month" in
      "Jan") mv $file $REPORT_DIR_YEAR/$month/$file echo "$file moved to $REPORT_DIR/$file";;
      "Feb") mv $file $REPORT_DIR_YEAR/$month/$file echo "$file moved to $REPORT_DIR/$file";;
      "Mar") mv $file $REPORT_DIR_YEAR/$month/$file echo "$file moved to $REPORT_DIR/$file";;
      "Apr") mv $file $REPORT_DIR_YEAR/$month/$file echo "$file moved to $REPORT_DIR/$file";;
      "May") mv $file $REPORT_DIR_YEAR/$month/$file echo "$file moved to $REPORT_DIR/$file";;
      "Jun") mv $file $REPORT_DIR_YEAR/$month/$file echo "$file moved to $REPORT_DIR/$file";;
      "Jul") mv $file $REPORT_DIR_YEAR/$month/$file echo "$file moved to $REPORT_DIR/$file";;
      "Aug") mv $file $REPORT_DIR_YEAR/$month/$file echo "$file moved to $REPORT_DIR/$file";;
      "Sep") mv $file $REPORT_DIR_YEAR/$month/$file echo "$file moved to $REPORT_DIR/$file";;
      "Oct") mv $file $REPORT_DIR_YEAR/$month/$file echo "$file moved to $REPORT_DIR/$file";;
      "Nov") mv $file $REPORT_DIR_YEAR/$month/$file echo "$file moved to $REPORT_DIR/$file";;
      "Dec") mv $file $REPORT_DIR_YEAR/$month/$file echo "$file moved to $REPORT_DIR/$file";;
          *) echo " Do nothing " ;;
    esac

done

And yes, the case $month doesn't work with the previous for loop $file. Why? I don't know. I just copy from various sources, forums, examples in for loop, and yet it doesn't work.

2
  • Do you want to take the date from the file's name or do you want to use the files real modification date?
    – terdon
    Jan 26, 2014 at 17:34
  • I intended to get the date from file, yes I forgot to mention.. Jan 27, 2014 at 7:19

3 Answers 3

10

First of all, it is never a good idea to parse the output of ls since it can lead to all sorts of problems. A better way to get the age of a file is stat. For example:

$ ls -l 20120322222.log 
-rw-r--r-- 1 terdon terdon 0 Jan  1  2012 20120322222.log
$ stat -c %y 20120322222.log 
2012-01-01 00:00:00.000000000 +0100

So, now we know how to get the age of the file,the question is how to convert that to a three letter month name. The easiest is to use date:

 $ date -d "2012-01-01" +"%b"
Jan

Combining the two commands gives:

$ date -d "$(stat -c %y 20120322222.log)" +"%b"
Jan

So,with this in mind, you can write your script as:

#!/usr/bin/env bash
BASE_DIR=/home/Work/LogFiles


## Find those files that are older than a month
find "$BASE_DIR" -maxdepth 1 -mtime +30 -type f -name "20*" | 
 while IFS= read -r file; do
    ## Get the file's modification year
    year="$(date -d "$(stat -c %y "$file")" +%Y)"
    ## Get the file's modification month
    month="$(date -d "$(stat -c %y "$file")" +%b)"

    ## Create the directories if they don't exist. The -p flag
    ## makes 'mkdir' create the parent directories as needed so
    ## you don't need to create $year explicitly.
    [[ ! -d "$BASE_DIR/$year/$month" ]] && mkdir -p "$BASE_DIR/$year/$month"; 

    ## Move the file
    mv "$file" "$BASE_DIR/$year/$month"
done

The script above assumes that you want to get real modification date of the files, not parse the name. If you want to parse the name instead let me know and I'll modify accordingly.

6
  • Yes!! tremendous job! That's done! I really grateful for your help. Thank you very much! This totally what I've expected.. Jan 27, 2014 at 10:26
  • I used this example for making a script which sorts my photos into dated folders, however I needed to use: year=$(date -d "$(stat -c %y "$file")" +%Y) (note the "" around $file) because it was moving files with spaces. Is this ok having apostrophes inside other apostrophes, or should I be doing this a different way?
    – timhc22
    Oct 18, 2014 at 23:39
  • 1
    @unegma no, I should have quoted in the first place. Thanks for pointing out, answer edited.
    – terdon
    Oct 20, 2014 at 13:02
  • This is very confusing. terdon's edit to the answer has an extra set of enclosing quotes that timhc22's comment directly above doesn't mention. And I'm trying to do this on OS X, but the answer directly below, which claims to accomplish that, has entirely different quoting from either, and breaks on file paths with spaces in them.
    – John Smith
    Jul 20, 2016 at 0:24
  • @MichaelKupietz use the quoting approach from my answer but the stat commands from the other. OSX date doesn't support the-d flag. So, for example month="$(stat -f '%Sm' -t '%m' "$file")"
    – terdon
    Jul 20, 2016 at 6:31
2

Thanks to terdon: I managed to take his/her script and amend it so it works in OS X. I also changed the folder structure to be basis month, day and hour:

#!/usr/bin/env bash
BASE_DIR=/Users/user/


## Find those files that are older than a month
find "$BASE_DIR" -maxdepth 1  -type f |
 while IFS= read -r file; do
    ## Get the file's modification month
    month="$(stat -f '%Sm' -t '%m' "$file")"
    ## Get the file's modification day
    day="$(stat -f '%Sm' -t '%d' "$file")"
    ## Get the file's modification hour
    hour="$(stat -f '%Sm' -t '%H' "$file")"


    ## Create the directories if they don't exist. The -p flag
    ## makes 'mkdir' create the parent directories as needed so
    ## you don't need to create $year explicitly.
    [[ ! -d "$BASE_DIR/$month/$day/$hour" ]] && mkdir -p "$BASE_DIR/$month/$day/$hour";

    ## Move the file
    mv "$file" "$BASE_DIR/$month/$day/$hour"
done
2
  • This breaks for file paths with spaces anywhere in them.
    – John Smith
    Jul 20, 2016 at 0:20
  • @MichaelKupietz I fixed the quoting here, try the updated version, it should work for any filename.
    – terdon
    Jul 20, 2016 at 7:52
0

here is my solution:

#!/bin/bash

BASE_DIR="${1}"

if [ -z "${BASE_DIR}" ]; then
  BASE_DIR="/home/Work/LogFiles"
fi

if [ ! -d "${BASE_DIR}" ]; then
  echo "Error: '${BASE_DIR}' does not exists." >2
  exit 1
fi

declare -a MONTH_NAMES
MONTH_NAMES[1]='JAN'
MONTH_NAMES[2]='FEB'
MONTH_NAMES[3]='MAR'
MONTH_NAMES[4]='APR'
MONTH_NAMES[5]='MAY'
MONTH_NAMES[6]='JUN'
MONTH_NAMES[7]='JUL'
MONTH_NAMES[8]='AUG'
MONTH_NAMES[9]='SEP'
MONTH_NAMES[10]='OCT'
MONTH_NAMES[11]='NOV'
MONTH_NAMES[12]='DEC'


find "${BASE_DIR}" -maxdepth 1 -mtime +30 -type f -name '*.log' \
  | grep -e '/[0-9]*.log$' \
  | while read FILE; do

  FILENAME="$(basename "${FILE}")"

  FILE_YEAR="$(echo "${FILENAME}" | cut --bytes=1-4)"
  FILE_MONTH="$(echo "${FILENAME}" | cut --bytes=5-6)"

  FILE_MONTH_NAME="${MONTH_NAMES[${FILE_MONTH}]}"

  REPORT_DIR="${BASE_DIR}/${FILE_YEAR}/${FILE_MONTH_NAME}"
  test -d "${REPORT_DIR}" || mkdir -p "${REPORT_DIR}"

  mv "${FILE}" "${REPORT_DIR}"

  echo "'${FILENAME}' moved to '${REPORT_DIR}'"
done

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .