160

Does any one have a template shell script for doing something with ls for a list of directory names and looping through each one and doing something?

I'm planning to do ls -1d */ to get the list of directory names.

9 Answers 9

190

Edited not to use ls where a glob would do, as @shawn-j-goff and others suggested.

Just use a for..do..done loop:

for f in *; do
  echo "File -> $f"
done

You can replace the * with *.txt or any other glob that returns a list (of files, directories, or anything for that matter), a command that generates a list, e.g., $(cat filelist.txt), or actually replace it with a list.

Within the do loop, you just refer to the loop variable with the dollar sign prefix (so $f in the above example). You can echo it or do anything else to it you want.

For example, to rename all the .xml files in the current directory to .txt:

for x in *.xml; do 
  t=$(echo $x | sed 's/\.xml$/.txt/'); 
  mv $x $t && echo "moved $x -> $t"
done

Or even better, if you are using Bash you can use Bash parameter expansions rather than spawning a subshell:

for x in *.xml; do 
  t=${x%.xml}.txt
  mv $x $t && echo "moved $x -> $t"
done
11
  • 33
    What if the filename has a space in it? Aug 28, 2009 at 17:17
  • 9
    Unfortunately, as Daniel said, the code in this answer will break if any of the files or folders contains a space or newline in their name. It shows a very common misuse of for loops, and a typical pitfall of trying to parse the output of ls. @DanielA.White, you might consider unaccepting this answer if it wasn't helpful (or is potentially misleading), since like you said, you're acting on directories. Shawn J. Goff's answer should provide a more robust and working solution to your issue.
    – slhck
    Jan 23, 2013 at 17:48
  • 6
    -∞ You shouldn't parse ls output, you shouldn't read output in a for loop, you should use $() instead of `` and Use More Quotes™. I'm sure @CoverosGene meant well, but this is just terrible.
    – l0b0
    Apr 17, 2014 at 19:31
  • 3
    A better alternative if you really want to use ls is ls -1 | while read line; do stuff; done. At least that one won't break for whitespaces.
    – ejoubaud
    Aug 1, 2014 at 8:47
  • 1
    with mv -v you don't need echo "moved $x -> $t" Jan 15, 2016 at 12:59
85

Using the output of ls to get filenames is a bad idea. It can lead to malfunctioning and even dangerous scripts. This is because a filename can contain any character except / and the nullcharacter, and ls does not use either of those characters as delimiters, so if a filename has a space or a newline, you will get unexpected results.

There are two very good ways of iterating over files. Here, I've used simply echo to demonstrate doing something with the filename; you can use anything, though.

The first is to use the shell's native globbing features.

for dir in */; do
  echo "$dir"
done

The shell expands */ into separate arguments that the for loop reads; even if there is a space, newline, or any other strange character in the filename, for will see each complete name as an atomic unit; it's not parsing the list in any way.

If you want to go recursively into subdirectories, then this won't do unless your shell has some extended globbing features (such as bash's globstar. If your shell doesn't have these features, or if you want to ensure that your script will work on a variety of systems, then the next option is to use find.

find . -type d -exec echo '{}' \;

Here, the find command will call echo and pass it an argument of the filename. It does this once for each file it finds. As with the previous example, there is no parsing of a list of filenames; instead, a fileneame is sent completely as an argument.

The syntax of the -exec argument looks a little funny. find takes the first argument after -exec and treats that as the program to run, and every subsequent argument, it takes as an argument to pass to that program. There are two special arguments that -exec needs to see. The first one is {}; this argument gets replaced with a filename that the previous parts of find generates. The second one is ;, which lets find know this is the end of the list of arguments to pass to the program; find needs this because you can continue with more arguments that are intended for find and not intended for the exec'd program. The reason for the \ is that the shell also treats ; specially - it represents the end of a command, so we need to escape it so that the shell gives it as an argument to find rather than consuming it for itself; another way of getting the shell to not treat it specially is to put it in in quotes: ';' works just as well as \; for this purpose.

4
  • 2
    +1 this is definitely the way to go when you need to generate a file list and use it in a command. find -exec is limited by only being able to run a single commands. With the loop, you can pipe to your heart's content.
    – MaQleod
    May 10, 2012 at 22:45
  • +1. I wish more people would use find. There is magic that can be done, and even the perceived limitations of -exec can be worked around. The -print0 option is also valuable for use with xargs.
    – ghoti
    Oct 17, 2014 at 22:24
  • The loop option won't show hidden directories. The find option won´t show symlinks.
    – jinawee
    May 21, 2019 at 14:41
  • The problem with this approach is you also get the '.' directory back. That means if you are trying to do, for example, an rm of a set of subfolders, you will remove those subfolders before traversing into them. I realize this is a specific issue I am trying to solve, but worth pointing out in case anyone attempts to use this with rm. Aug 12, 2023 at 7:55
39

For files with spaces in you will have to make sure to quote the variable like:

 for i in $(ls); do echo "$i"; done; 

or, you can change the input field separator (IFS) environment variable:

 (IFS=$'\n';for i in $(ls); do echo $i; done) # subshell to restore IFS

Finally, depending on what you're doing, you may not even need the ls:

 for i in *; do echo "$i"; done;
4
  • nice use of a subshell in the first example
    – Jeremy L
    Aug 28, 2009 at 18:46
  • Why does IFS need a $ after the assignment and before the new character? Nov 29, 2012 at 18:57
  • 7
    Filenames can also contain newlines. Breaking on \n is insufficient. It's never a good idea to recommend parsing the output of ls.
    – ghoti
    Oct 17, 2014 at 22:18
  • Could replace \n with \0 (null character), as some people have suggested
    – awvalenti
    Apr 21, 2021 at 1:30
6

If you have GNU Parallel http://www.gnu.org/software/parallel/ installed you can do this:

ls | parallel echo {} is in this dir

To rename all .txt to .xml:

ls *.txt | parallel mv {} {.}.xml

Watch the intro video for GNU Parallel to learn more: http://www.youtube.com/watch?v=OpaiGYxkSuQ

4

Just to add to CoverosGene's answer, here is a way to list just the directory names:

for f in */; do
  echo "Directory -> $f"
done
2

Why not set IFS to a newline, then capture the output of ls in an array? Setting IFS to newline should resolve issues with funny characters in file names; using ls can be nice because it has a built-in sort facility.

(In testing I was having trouble setting IFS to \n but setting it to newline backspace works, as suggested elsewhere here):

E.g. (assuming desired ls search pattern passed in $1):

SAVEIFS=$IFS
IFS=$(echo -en "\n\b")

FILES=($(/bin/ls "$1"))

for AFILE in ${FILES[@]}
do
    ... do something with a file ...
done

IFS=$SAVEIFS

This is especially handy on OS X, e.g., to capture a list of files sorted by creation date (oldest to newest), the ls command is ls -t -U -r.

2
  • 3
    Filenames can also contain newlines, and they often do when users are permitted to name their own files. Breaking on \n is insufficient. The only valid solutions use a for loop with pathname expansion, or find.
    – ghoti
    Oct 17, 2014 at 22:21
  • The only reliable way to transfer a list of file names is to separate them with a NUL character, as this is the only one definitely not contained in a file path.
    – glglgl
    Jan 2, 2015 at 9:23
2

while loop with ls command in a command line


ls | while read filename;do echo "file name is $filename"; done
-3

This is how I do it, but there are probably more efficient ways.

ls > filelist.txt

while read filename; do echo filename: "$filename"; done < filelist.txt
3
  • 6
    Stick to pipes in place of the file: >ls | while read i; do echo filename: $i; done
    – Jeremy L
    Aug 28, 2009 at 18:46
  • Cool. I should say that you can also use $EDITOR filelist.txt in between the two commands. Lots of stuff you can do in an editor that is easier than on the command line. Not relevant to this question, though.
    – TREE
    Aug 31, 2009 at 18:45
  • Your solution doesn't at all address the problem with file names containing newlines and other fancy stuff.
    – glglgl
    Jan 2, 2015 at 9:24
-3

It's prefectly fine:

#!/bin/sh
for i in `ls`
do
   echo `ls -l $i`
done

Here is the output:

root@online:/usr/local/httpd/bin # more ~root/rm-in-ls.sh
    #!/bin/sh
    for i in `ls` 
    do
      echo `ls -l $i`
    done

root@online:/usr/local/httpd/bin # /root/rm-in-ls.sh
-rwxr-xr-x 1 root wheel 82516 Jul 3 04:52 ab
-rwxr-xr-x 1 root 40 3431 Jul 3 04:35 apachectl
-rwxr-xr-x 1 root 40 23881 Jul 3 04:35 apxs
-rwxr-xr-x 1 root wheel 21456 Jul 3 04:52 checkgid
-rwxr-xr-x 1 root 40 8931 Jul 3 04:35 dbmmanage
root@online:/usr/local/httpd/bin # 

Missed it ! Right !!!

HEADS UP !!!

Thats not " ' " in 'ls' !!! Thats " ` " in `ls`. The acute not apostrophe. There is a difference between ' and `. I spent a week on that to figure out when I was in college (1990).

This is "`", the key with ~ Tilde mostly shift + ~, it's called acute.wiki

If you did not get it, then copy and paste the script and run.

2
  • It doesn't play well with spaces in files/folders names. The problem is not in the quote.
    – awvalenti
    Apr 21, 2021 at 1:28
  • I am bit surprised that I got -2 on a perfect and notable example that I learned from mistake as student at my college life decades ago. Surprised that not many experts around for up vote. Apr 10, 2023 at 19:34

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .