bash script to sort out duplicates in text file

Question

I converted all the flac in my music library down to mp3, so I can move the flac to an external drive. I used dbpoweramp in windows, and I'm pretty sure it converted them all, but when I woke up this morning, the netbook had been restarted.

So I've made a list with all my music in it and used sort to alphabetize it. So I have something like this:

~/music/a/a.flac
~/music/a/a.mp3
~/music/a/b.flac
~/music/a/b.mp3
~/music/b/a.mp3
~/music/b/b.mp3
~/music/c/a.flac
~/music/c/a.mp3

Note that I have a directory with only mp3, since not my whole library was flac.

What I want is a bash script that will check that all lines that end with .flac have a line directly below them that is the same except it ends with .mp3.

How would I achieve this, and if you can explain what the script is doing, that would be cool too.

I guess it's sorta important to show any lines that end with .flac that DON'T have a matching .mp3, as they need converted still.

Answer 1

awk '
    root && $0 != root ".mp3" {printf("%d: %s.flac\n", line, root)}
    /.flac$/ {
        root=$0
        sub(/.flac$/, "", root)
        line = NR
        next
    }
    { root = "" }
' filename

How does it work?

Starting with the line beginning /.flac$/, foreach line ending with ".flac", create a variable named root containing the line minus the extension. Saves the current line number. Jump to the next line to avoid erasing the root variable just set.

Going to the first line. This expression root && $0 != root ".mp3" means: root is non-empty AND the current line ($0) is not equal to the value of the root variable plus ".mp3". If this expression evaluates to true, then the current line is not the MP3 file corresponding with the previous FLAC file.

The last line erases the value of the root variable. This part of script is reached only if the current line is not a FLAC file, so we don't want to make the comparison with the next line.

Answer 2

I found a different way to do it, using diff, but haven't made it into a bash script yet. I'll post here if I find a way to do that.

Glenn posted a great script that works, but here's how I ended up doing it.

I made two sorted files, one with .flac and one with .mp3 with find ~/music -name *.flac | sort > ~/documents/flac and find ~/music -name *.mp3 | sort > ~/documents/mp3

Then I removed the extensions in vim

vim ~/documents/flac
:%s/.....$//
:w
:e ~/documents/mp3
:%s/....$//
:wq

And then I did diff ~/documents/mp3 ~/documents/flac | grep '>' which would show nothing if everything was done right, and show me the lines of flacs that didn't have an mp3.

I'm pretty sure I get this into one or lines, but Glenn's answer worked great with the file I had.

Answer 3

It is possible using find, too.

With using while loop after find:

find . -name '*.flac' | while read file ; do test -f `dirname $file`/`basename $file .flac`.mp3 && echo $file; done

With using (lots of) subshells:

find . -name '*.flac' -exec sh -c 'test -f `echo {} |sed s/\.flac$/.mp3/` && echo {}' \;