Best way to manually sort random text files?

Question

I have about 1000 text files and I need to view each, and move it to a folder if it's the correct one. I can only do basic sorting by length/size, and I can't grep because the text is random. How can I do this besides manually openiing + saving each in gedit. I'm on Ubuntu Linux. Thanks

I've already done all the sorting I can based on size,wordcount,greps,date,etc. This is what's left over. I'm trying to find an easy way to view +save/ignore the rest.

Answer 1

I'm guessing you probably don't use vim... so maybe this will just serve as incentive to learn it, or maybe a brief tutorial.....

$ vim *

will open all the files;

:!mkdir mine/; mkdir notmine/

will create directories called "mine" and "notmine";

:!mv % mine/

will move the current file to a directory called "mine";

:bdel

will drop that file from the ones that vim is dealing with and move to the next;

:!mv % notmine/

will move the current file to a directory called "notmine". What's going on with these move commands is

• : enters Command-line mode
• ! starts a shell command
• % causes vim to substitute in the name of the current file. If any of the filenames have spaces in them, you'll need to double-quote the % argument, eg :!mv "%" mine/.

Hit :, then b, then the up arrow to get the :bdel command again.

Hit :, then !mv, then the up arrow repeatedly to cycle through your previous !mv ... commands. You can also just type :! and then the up arrow, but that will get the !mkdir... command as well.

In case you hadn't already guessed, the up arrow in vim's "command-line mode" (what you enter by pressing : in "normal mode") will cause it to go through the previous commands that start with whatever you've already typed there.

---

Also, on the off chance that you are doing this without any previous experience with vim, I must first commend you for making such a bold move, and then proceed to inform you that if at any point you accidentally hit a key on the keyboard and you start actually typing stuff into the file, or things otherwise work unexpectedly, hit <ESC> a couple of times, and then u to Undo any changes you made accidentally. If you go too far back (not really an issue for this example, since you're not actually going to be modifying any files), you hit <CTRL>-R to Redo stuff you just Undid. The <ESC> will bring you out of Insert mode or Visual mode, back into Normal mode, which is where all these commands actually work.

---

Okay.. after that little PSA, back to our regularly scheduled programming.

To make your file processing more efficient, you can set up a couple of macros:

qm:!mv % mine/<CR>:bdel<CR>q

This will actually have the effect of moving the current file and dropping it from vim's "buffer list", so make sure it's one of yours when you set up this macro. Note that <CR> herein indicates that you hit the ENTER key at that point; don't actually type <CR>.

qn:!mv % notmine/<CR>:bdel<CR>q

Same deal here.

Then to move a file to the "mine" directory and move on in the buffer list, you do

@m

Or if it's not yours

@n

Actually it might be wiser to map those macros to letters that are further apart on the keyboard than m and n.. but that's the basic idea.

Answer 2

As Mark said you'll need to be more specific about the files. Instead of gedit you can use the less pager or head/tail to view parts of the file, then the mv utility to move it. This should make the process a little faster.

For a programmatic approach, depending on the size of the file and if it would fit on one screen, you could do something with this logic:

for each file in the directory
       print it's contents to the screen
       prompt for yes/no
       if yes
             move file to other directory
       else
             leave file alone

so essentially your program would show you the contents of each file, and if you wanted it moved, press Y, else press N. That way each file can be handled in 1 or 2 keystrokes, depending on how you write it.

Answer 3

I was bored:

$ cat disposition
#!/bin/sh

# pick your preferred viewer
VIEWER=less
VIEWER=head

# the log file will actually be a script that undoes all the
# changes made, so it is more like an inverted log
log=/tmp/disposition.$$

prog=$0
dest=$1

case $# in
    0|1) echo "usage $prog dest_dir file ..."
         echo "  presents files and asks for a disposition"
         echo "  if disposition is 'yes' move it to dir"
         echo "  otherwise do nothing"
         exit 1;;
esac
# some shells may have limits on command lengths so for many files
# you might have to:
#    disposition dest_dir a*
#    disposition dest_dir b*
#    etc.

if [ ! -d $dest ] ; then
    echo "$prog: destination $dest is not a directory"
    exit 1
fi

cat > $log <<EOF
#!/bin/sh
# inverted log file created by $prog
#

EOF

shift           # toss away dest_dir from args
for file; do    # in "$@" is implied
    if (echo "===== $file =====" ; cat $file ) | $VIEWER; then
        read -p "move $file to $dest? " move
        case $move in
            [yY]*)
                echo mv \"$dest/$file\" \"$file\" >> $log
                mv "$file" "$dest/$file"
                ;;
        esac
    fi
done

echo "$prog: inverted log script can be found in $log"

Answer 4

I suppose you would already know that, but for the benefit of others, you can use head (or tail) to show the first few rows of the file. That will be more doable than printing the whole thing.

And of course, write bash script =)

with programming background i think this howto would be quite adequate for programming such a script http://tldp.org/LDP/abs/html/internal.html

(and im lazy enough not to write one)

Answer 5

I was wondering if naive Bayesian classifiers could apply to code and found DivMod's Reverend. What struck me was the line:

* is your code more like Guido's or Peter's 

It looks a little aged, but then again Reverend Bayes isn't getting any younger, either. It is available as the Ubuntu package python-reverend, but I'm too sleepy to subclass a recognizer right now.

Answer 6

Could you be more specific about why grep won't meet your needs? If there's no discernible "method to the madness" then by hand is the only way. You can't automatically/programatically sort things if there are no conditions you can give for the sorting.