2

I have a process that progressively creates directories, assigns an index in ascending order to them, and stores results in the last-iteration directory, the one with the largest index. Given that the number of iterations necessary to finish the process vary per data set, I can't predict the index of the last directory. For example:

#Dataset 1 may produce
ls -d Dir*
...    Dir4    Dir5

and

#Dataset 2 may produce
ls -d Dir*
...    Dir34    Dir35

I guessed that I could create and Array that contains the name of all directories, make a copy of the last and remove all Directories

ARR=($(ls -d Dir*))
cp ${ARR[@]:(-1)} LastDirectory #Preserve my results in LastDirectory
rm Dir*

But this is sure a way to shoot myself in the foot. Let's say the program takes ten iterations to finish. Then, Dir10 will contain my results. If I list the directories and pass them to the array, Dir10 won't be in the last position and will be eliminated. This is the kind of unpredictable behavior I want to avoid!

#You can copy-paste this piece of code to replicate the problem:
mkdir Dir1 Dir2 Dir3 Dir4 Dir5 Dir6 Dir7 Dir8 Dir9 Dir10 
ls -d Dir*
Dir1 Dir10 Dir2 Dir3 Dir4 ... Dir8 Dir9
ARR=($(ls -d Dir*))
echo ${ARR[@]:(-1)}
Dir9

Is there a save way to get rid of all directories except the one with the largest index?

NOTE: I thought about using directories creation dates, but it seems this option is not supported in Linux.

  • Please add an example for indices 9 and 10. – Arjan Sep 8 '16 at 14:30
  • And unrelated to your question, you might be able to use the creation times rather than the names. – Arjan Sep 8 '16 at 14:45
  • @Arjan I included the example – je_b Sep 8 '16 at 15:38
  • @Arjan As far as I know, linux doesn't support creation dates: unix.com/shell-programming-and-scripting/… – je_b Sep 18 '16 at 11:04
  • True, @je_b, modification date then; the output of ls -t. – Arjan Sep 18 '16 at 12:59
4
+50

On my Ubuntu there is -v option to ls. From man ls:

-v natural sort of (version) numbers within text

As an alternative the -V option to sort is also meant to deal with version numbers. I decided to include it in my answer because sort works as a filter. It may be handy in general case (e.g. when you obtain your directory list from find or from a text file).

Write your array definition like this:

ARR=($(ls -d -v Dir*))

or this:

ARR=($(ls -d Dir* | sort -V))

EDIT: dave_thompson_085's comment gives useful simplification:

adding -r to either puts the desired item first, accessible with more convenient ${ARR[1]}.

  • 1
    Those are both features of GNU coreutils which will be on all but really weird distros or builds of Linux. And adding -r to either puts the desired item first, accessible with more convenient ${ARR[1]} – dave_thompson_085 Sep 18 '16 at 14:44
0

Can you (manually) create a dir named Dir1000 before you start the process, and then it creates Dir1001, Dir1002, etc.?
If the process works that way, it would be obvious and easy?

  • No, I can't. I tested your option and the system just ignores the manually created Directory. It seems the script has its own internal counter. As a side note, this is code I inherited from a colleague in a language I barely understand. I prefer not to touch what is already working and solve the problem pragmatically. – je_b Sep 21 '16 at 8:54
0

Since you cannot change the program that creates the directories, as you mentioned, can you make it write to different output directories for each task? So you would have

Task1
+ Dir1
+ Dir2
Task2
+ Dir3
+ Dir4

Otherwise, it is probably safer to look at the contents of the directories. Are there differences between the intermediate directories and the last one which contains the results? If the results are only in the last directory, you could use this information to find out which it is to copy the results away and remove the other directories.

0

Does sort have the -n -r and -k options? If it does, then use:

ARR=($( ls -d Dir* | sort -rn -k1.4 ))

To test the result, use:

mkdir Dir1 Dir2 Dir3 Dir4 Dir5 Dir6 Dir7 Dir8 Dir9 Dir10 Dir20 Dir100
ARR=($( ls -d Dir* | sort -rn -k1.4 ))
echo ${ARR[@]}

The result should be:

Dir100 Dir20 Dir10 Dir9 Dir8 Dir7 Dir6 Dir5 Dir4 Dir3 Dir2 Dir1

To save the directory with the largest index and remove the rest of them (assuming LastDirectory does not exist), use:

mv ${ARR[0]} LastDirectory # or simply mv $ARR LastDirectory
rm -r Dir* # error if only one Dir* directory, but LastDirectory is preserved

The sort -nr -k1.4 command uses the -k option to sort the directory names using the fourth character to the last character, ignoring the first three characters. The -n option sorts numerically, and the -r option reverses the order so that the largest number is first.

A note about robustness, that is, handling unexpected conditions. The solution assumes that all the names match the pattern where there are three characters, "Dir", followed by a number, and that LastDirectory does not exist. In general, the ls command is troublesome because it lists names in a way that's easy to read; but, for example, if a file name has an embedded space (blank character), the name would break this solution in unpredictable ways.

0

If you are ok with sorting the directories by date (it sounds like the directory with the highest index is also the one most recently created), you can do this:

ls -1td Dir* | tail -n +2

which lists all but the most recent directory. To delete those:

rm $(ls -1td Dir* | tail -n +2)

The key argument here is -t which sorts by date.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.