Using find, xargs, etc. to output similarly named files

Question

I have a folder full of HTML files:

001.htm
002.htm
003.htm
…

I want to run Pandoc on them to convert them to similarly named Markdown files:

001.md
002.md
003.md

This command works on one of them:

pandoc -f html -t markdown 001.htm -o 001.md

And I want to use find and xargs to automatically run a similar command on every file in the folder.

I got as far as this:

find *.htm | xargs -I {} -n 1 pandoc -f html -t markdown -o {}

…which truncates every file in the directory, so now I'm asking before I really break something.

What is wrong with my command above, and/or what's a completely different / more efficient way to do this?

Answer 1

I managed to do that with this 1 liner. If you are flexible about the xargs and find part.

for f in ./*.htm; do pandoc -f html -t markdown "$f" -o "${f%.htm}.md"; done

If you want to act recursively (so: all .htm files in the current directory, and all subdirectories), then (assuming bash 4+) you can use the globstar shell option:

shopt -s globstar
for f in ./**/*.htm; do pandoc -f html -t markdown "$f" -o "${f%.htm}.md"; done

Answer 2

Using {} isn't flexible enough for some situations. This appears to be one of those.

A possible workaround would be to -execa script from find, like so:

find . -name '*.htm' -exec ./convert-to-md.sh {} \;

The script file should look similar to this, depending on the exact pandoc command line:

#!/bin/bash
pandoc -f html -t markdown -o "${1/.htm/.md}" "${1}"

---

If you don't want to create and save a script file for this, you can always inline the bashscript code:

find . -name '*.htm' | xargs -n 1 bash -c 'pandoc -f html -t markdown -o "${1/.htm/.md}" "${1}"' -

The additional -at the end serves to fill $0in bash, which usually includes the name of the shell script, positional arguments starting at $0.

This allows you to keep using find(even with -print0and xargs -0 if you're handling weird file names), but doesn't require creation of a separate file.

Answer 3

You appear to be missing a {} in the pandoc command

find . -name \*.htm | xargs -I {} -n 1 pandoc -f html -t markdown {} -o {}.md

But then you'll have files named 001.htm.md -- you'll have to decide if this is a problem.