If you list the file names in a file called ~/filelist.txt
, then go to the directory where all the files are, the following script should move them into manageably-sized sub-directories, one for each day that files were created:-
cat ~/filelist.txt | \
while f=`line`; \
do ( d="`ls -l --time-style=+%y-%m-%d "$f"|awk '{print $6}'`"; \
[ -d "$d" ] || mkdir "$d"; \
mv "$f" "$d"/ \
) \
done
This will take a long time to run, because you will be running two or three programs for each file, as well as a bash
sub-shell.
I suggest that you test it first before you start by using head
instead of cat
and prefixing echo
to the mkdir
and mv
commands. Of course for this test, mkdir
will be listed repeatedly for files with the same date, as the directories are not actually being created. If you repeat with head -n 1000
or more, you should get an idea of how many files will go into each directory.
If there are too many or too few, you can change the time style format to make it weekly or hourly archives. You should aim for around 1000-10000 files in each directory, which will give you 700-7000 directories, both manageable.
Once they are moved into sub-directories, you can zip each directory, which should give you around 7000 files to move, and a small fraction of the 650GB to copy if they're text files.