How to write a script that accepts input from a file or from stdin?

Question

How can one write a script that accept input from either a filename argument or from stdin?

for instance, you could use less this way. one can execute less filename and equivalently cat filename | less.

is there an easy "out of the box" way to do so? or do i need to re-invent the wheel and write a bit of logic in the script?

Answer 1

If the file argument is the first argument to your script, test that there is an argument ($1) and that it is a file. Else read input from stdin -

So your script could contain something like this:

#!/bin/bash
[ $# -ge 1 -a -f "$1" ] && input="$1" || input="-"
cat $input

e.g. then you can call the script like

./myscript.sh filename

or

who | ./myscript.sh

Edit Some explanation of the script:

[ $# -ge 1 -a -f "$1" ] - If at least one command line argument ($# -ge 1) AND (-a operator) the first argument is a file (-f tests if "$1" is a file) then the test result is true.

&& is the shell logical AND operator. If test is true, then assign input="$1" and cat $input will output the file.

|| is the shell logical OR operator. If the test is false, then commands following || are parsed. input is assigned to "-". The command cat - reads from the keyboard.

Summarising, if the script argument is provided and it is a file, then variable input is assigned to the file name. If there is no valid argument then cat reads from the keyboard.

Answer 2

read reads from standard input. Redirecting it from file ( ./script <someinput ) or through pipe (dosomething | ./script) will not make it work differently.

All you have to do is to loop through all the lines in input (and it doesn't differ from iterating over the lines in file).

(sample code, processes only one line)

#!/bin/bash

read var
echo $var

Will echo first line of your standard input (either through < or |).

Answer 3

You don't mention what shell you plan on using, so I'll assume bash, though these are pretty standard things across shells.

File Arguments

Arguments can be accessed via the variables $1-$n ($0 returns the command used to run the program). Say I have a script that just cats out n number of files with a delimiter between them:

#!/usr/bin/env bash
#
# Parameters:
#    1:   string delimiter between arguments 2-n
#    2-n: file(s) to cat out
for arg in ${@:2} # $@ is the array of arguments, ${@:2} slices it starting at 2.
do
   cat $arg
   echo $1
done

In this case, we are passing a file name to cat. However, if you wanted to transform the data in the file (without explicitly writing and rewriting it), you could also store the file contents in a variable:

file_contents=$(cat $filename)
[...do some stuff...]
echo $file_contents >> $new_filename

Read from stdin

As far as reading from stdin, most shells have a pretty standard read builtin, though there are differences in how prompts are specified (at the very least).

The Bash builtins man page has a pretty concise explanation of read, but I prefer the Bash Hackers page.

Simply:

read var_name

Multiple Variables

To set multiple variables, just provide multiple parameter names to read:

read var1 var2 var3

read will then place one word from stdin into each variable, dumping all remaining words into the last variable.

λ read var1 var2 var3
thing1 thing2 thing3 thing4 thing5
λ echo $var1; echo $var2; echo $var3
thing1
thing2
thing3 thing4 thing5

If fewer words are entered than variables, the leftover variables will be empty (even if previously set):

λ read var1 var2 var3
thing1 thing2
λ echo $var1; echo $var2; echo $var3
thing1
thing2
# Empty line

Prompts

I use -p flag often for a prompt:

read -p "Enter filename: " filename

Note: ZSH and KSH (and perhaps others) use a different syntax for prompts:

read "filename?Enter filename: " # Everything following the '?' is the prompt

Default Values

This isn't really a read trick, but I use it a lot in conjunction with read. For example:

read -p "Y/[N]: " reply
reply=${reply:-N}

Basically, if the variable (reply) exists, return itself, but if is's empty, return the following parameter ("N").

Answer 4

The simplest way is to redirect stdin yourself:

if [ "$1" ] ; then exec < "$1" ; fi

Or if you prefer the more terse form:

test "$1" && exec < "$1"

Now the rest of your script can just read from stdin. Of course you can do similarly with more advanced option parsing rather than hard-coding the position of the filename as "$1".

Answer 5

You can also do:

#!/usr/bin/env bash

# Set variable input_file to either $1 or /dev/stdin, in case $1 is empty
# Note that this assumes that you are expecting the file name to operate on on $1
input_file="${1:-/dev/stdin}"

# You can now use "$input_file" as your file to operate on
cat "$input_file"

For more neat parameter substitution tricks in Bash, see this.

Answer 6

use (or chain off of) something else that already behaves this way, and use "$@"

let's say i want to write a tool that will replace runs of spaces in text with tabs

tr is the most obvious way to do this, but it only accepts stdin, so we have to chain off of cat:

$ cat entab1.sh
#!/bin/sh

cat "$@"|tr -s ' ' '\t'
$ cat entab1.sh|./entab1.sh
#!/bin/sh

cat     "$@"|tr -s      '       '       '\t'
$ ./entab1.sh entab1.sh
#!/bin/sh

cat     "$@"|tr -s      '       '       '\t'
$ 

for an example where the tool being used already behaves this way, we could reimplement this with sed instead:

$ cat entab2.sh
#!/bin/sh

sed -r 's/ +/\t/g' "$@"
$ 

Answer 7

You can also keep it simple and use this code

---

When you create a script file pass_it_on.sh with this code,

#!/bin/bash

cat

You can run

cat SOMEFILE.txt | ./pass_it_on.sh

and all the contents of stdin will be simply spewed out to the screen.

---

Alternatively use this code to both keep a copy of stdin in a file and then spew it out to the screen.

#!/bin/bash

tmpFile=`mktemp`
cat > $tmpFile
cat $tmpFile    

---

and here is another example, maybe more readable, explained here:

http://mockingeye.com/blog/2013/01/22/reading-everything-stdin-in-a-bash-script/

#!/bin/bash

VALUE=$(cat)

echo "$VALUE"

Have fun.

RaamEE

Answer 8

The simplest way and POSIX compliant is:

file=${1--}

which is equivalent to ${1:--}.

Then read the file as usual:

while IFS= read -r line; do
  printf '%s\n' "$line" # Or: env POSIXLY_CORRECT=1 echo "$line"
done < <(cat -- "$file")