Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I have a folder full of HTML files:

001.htm
002.htm
003.htm
…

I want to run Pandoc on them to convert them to similarly named Markdown files:

001.md
002.md
003.md

This command works on one of them:

pandoc -f html -t markdown 001.htm -o 001.md

And I want to use find and xargs to automatically run a similar command on every file in the folder.

I got as far as this:

find *.htm | xargs -I {} -n 1 pandoc -f html -t markdown -o {}

…which truncates every file in the directory, so now I'm asking before I really break something.

What is wrong with my command above, and/or what's a completely different / more efficient way to do this?

share|improve this question

3 Answers 3

up vote 3 down vote accepted

I managed to do that with this 1 liner. If you are flexible about the xargs and find part.

for f in ./*.htm; do pandoc -f html -t markdown "$f" -o "${f%.htm}.md"; done

If you want to act recursively (so: all .htm files in the current directory, and all subdirectories), then (assuming bash 4+) you can use the globstar shell option:

shopt -s globstar
for f in ./**/*.htm; do pandoc -f html -t markdown "$f" -o "${f%.htm}.md"; done
share|improve this answer
2  
+1. xargs doesn't allow you the same flexibility of filename modification. Do not replace *.htm with $(find...) -- filenames with spaces will be properly handled in the first case but not the second. –  glenn jackman Jan 25 '13 at 19:35
1  
@glennjackman Unless you set the bash $IFS to $'\n' for that code section, in which case spaces aren't a problem -- newlines still are though. –  Daniel Beck Jan 25 '13 at 20:05
    
Wow, there are two or three new things about the command line for me to learn from that snippet. Thanks! –  75th Trombone Jan 25 '13 at 23:44

Using {} isn't flexible enough for some situations. This appears to be one of those.

A possible workaround would be to -execa script from find, like so:

find . -name '*.htm' -exec ./convert-to-md.sh {} \;

The script file should look similar to this, depending on the exact pandoc command line:

#!/bin/bash
pandoc -f html -t markdown -o "${1/.htm/.md}" "${1}"

If you don't want to create and save a script file for this, you can always inline the bashscript code:

find . -name '*.htm' | xargs -n 1 bash -c 'pandoc -f html -t markdown -o "${1/.htm/.md}" "${1}"' -

The additional -at the end serves to fill $0in bash, which usually includes the name of the shell script, positional arguments starting at $0.

This allows you to keep using find(even with -print0and xargs -0 if you're handling weird file names), but doesn't require creation of a separate file.

share|improve this answer
    
Doesn't find handle weird file names on its own anyway? IIRC there's never a good reason to use find … -print0 | xargs -0 … –  slhck Jan 25 '13 at 19:58
    
@slhck Newlines are valid file name components. The following demonstrates how this causes scripts to fail: touch "$( echo -e 'foo\nbar' )" ; find . -name 'foo*bar' | xargs -n 1 echo File: –  Daniel Beck Jan 25 '13 at 20:00

You appear to be missing a {} in the pandoc command

find . -name \*.htm | xargs -I {} -n 1 pandoc -f html -t markdown {} -o {}.md

But then you'll have files named 001.htm.md -- you'll have to decide if this is a problem.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.