85

We are doing all our scripting with Bash so far, but I'm starting to feel a bit silly about it. While we can of course do everything we want with Bash (it's quite powerful), I'm starting to wonder if we shouldn't use a proper scripting language (in our case most likely Ruby) instead.

How do you decide when to use Perl/Python/Ruby over Bash for a script? I don't think an init script with Ruby makes sense, but how about a slightly longer script that adds email accounts?

1
  • 2
    If you can do it using either, does it really matter? The choice is only interesting if the resulting script presents any differences. For example, execution time could differ drastically (not the case an issue when email accounts).
    – Dennis
    Apr 21, 2012 at 4:05

11 Answers 11

50

Given a problem that both can handle, you'll want to use the one you're most comfortable with. Ultimately, there are a lot of small details, and only experience can teach you to see them.

Bash is a general purpose scripting language just like Python, Ruby, Perl, but each has different strengths over the rest. Perl excells at text analysis, Python claims to be the most elegant of the bunch, Bash scripts are excellent at "piping stuff around", if you know what I mean, and Ruby... well, Ruby is a little special in a lot of ways.

However, the differences between them only really matter once you have a healthy amount of scripting experience under your belt. I suggest you pick one language and push it to it's limits before moving to the next. You can do a lot in a shell script, more than most people would admit. Any language is just as hard as you want to make it. After you've written a couple things in it, every language is "easy" to you.

Being familiar with the shell pays off quickly if you live in Linux, so maybe you want to start with that. If you find a task you that is impossible or impractical to solve in a shell script, use something else.

Also, bear in mind that learning shell scripting is very simple. The real power of it lies in other programs, like awk, sed, tr, et al.

3
  • I've got quite some experience with bash scripting, as with Ruby, but I sometimes still don't know what to use. Choosing based on personal preference at that moment in time seems silly. But you're right, I'll ask myself what I want to do and pick the best tool for the job.
    – futlib
    Apr 21, 2012 at 6:09
  • 1
    Personally, I'll use a scale of the languages I know well, in ascending order of complexity and power, and use the simplest that will fit the bill. But really, the only way to know for sure is writing your script in all languages and comparing the outcome. It's all more of a gut feeling than a certainity. After some years writing scripts, you will have come across most "types of problems", and know which language did well solving them.
    – mkaito
    Apr 21, 2012 at 14:20
  • 1
    Just wanted to add that using small programs in the GNU coreutils package (like tr, sort, uniq) and piping them it's possible to accomplish a lot of tasks.
    – GMaster
    Jan 11, 2016 at 0:52
63

TL;DR - use bash only for installing a better language (if it isn't already available), otherwise you're wasting unrecoverable, precious human time. If you can't do it on the command line by hand without mistakes, don't script with bash/shell.

It's 2015, so I'd consider the following:

  1. memory overhead

    • Ruby/Python runtime memory overhead compared to bash is tiny (because of shared libraries), while one probably can't maintain a non-trivial bash script anyway (i.e. one with > 100 lines) - so memory usage is not a factor
  2. startup time

    • Ruby/Python startup may be a tiny bit slower, but chances are you're not going to be running lots of full Ruby/Python processes in a tight loop 100 times per second (if you have those kinds of needs, bash/shell is too much overhead anyway and you'll probably need to drop to C/C++)
  3. performance

    • almost all typical data crunching will be faster in Ruby/Python - or at least comparable (or, you need C/C++/Haskel/OCaml/whatever anyway)
    • the real performance/bottleneck in execution (or even productivity) will almost never ever be "a lack of using bash/shell" (even Ubuntu switching dash for startup shows how bash is actually the problem - and a busybox is probably the only use case, because there is nothing more than 'bash' and 'vi' to write and run code, and there's often no way to add/download or store anything else)
    • running other processes to do the job (like sed/awk/grep) is actually a magnitude slower than than calling a method on a live object in memory
  4. productivity

    • it's too easy to make mistakes in Bash/shell compared to using "real" methods, parameters, variables and exceptions in Ruby/Python
    • Agile is mainstream, while Bash has no support for it (lacking in unit testing capabilities, libraries, OO, modularity, linting, introspection, logging, metaprograming; almost impossible to refactor without breaking something)
    • way too many incompatibilities with other shells, minor environment variables can completely break a script (and some important dev-ops tools like Puppet ignore shebang lines and pass on or rewrite important shell variables), while Ruby/Python have well-defined relatively smooth migration paths even for major version changes
    • learning a new language takes a fraction of the time alternatively spent debugging shell scripts due to shell-specific issues (notably - variable names, no booleans, no exceptions, etc.)
    • even startup scripts are a landmine (especially because they can fail during system startup), and given the recent security flaws with bash, you may be better off using plain C (with good libraries) - yes, C needs compiling, configuration, etc., but even a simple shell script may need a repository, then versioning, then packaging anyway.
    • whatever is available with sed/awk/grep is likely already built into Ruby/Python - without it being a dependency, or "differences" between versions of those tools across platforms (so what if it works on your setup)
  5. job security
    • what's the point of securing a job you don't like? (unless you love spending all those hours hard-to-debug but trivial-to-make shell script bugs)

I find there's no reason to use Bash/Shell if you have Ruby/Python installed.

And probably getting Ruby/Python installed doesn't even need a bash script in the first place (with the exception of busybox, some system tools depend on Python/Perl being present anyway).

And every time you write a shell script, you're "practicing" doing exactly that - instead of learning something more powerful/productive.

Why do people use Bash nowadays? Because it's a terrible, hard-to-break habit. A script is rarely "finished forever" after the first few minutes - no matter how strongly people tend to think that way. Along with the "it's the last bug in this script" fallacy.

Conclusion: use bash/shell only when you're absolutely forced to (like ~/.bashrc, busybox), because it's almost never "the right tool for the job" nowadays.

2
  • what about Perl? Is it purposefully omitted?
    – ZenVentzi
    Nov 6, 2019 at 14:32
  • 2
    I would argue that this is bit too much of a broad sweeping statement. If you are actually developing an application and you care about performance then use an actual programming language like perl/python/ruby (or Rust/Go/C). But if you just want something simple like running a command n times on startup then use whatever you are comfortable with. The benefit of bash is that if you don't know what environment your script is going to be ran in then pretty much every *nix has some kind of POSIX shell that will run most bash scripts.
    – JoeS
    Feb 22, 2020 at 20:02
32

I use bash when my primary focus is on file handling. This could include moving, copying, and renaming files, as well as using files as input for other programs or storing other program's output in files. I rarely write bash code that actually examines the contents of a file or generates the output to write to a file; I leave that to the other programs (which I may write in Perl or python) that I launch via bash.

I use Perl and python when my primary focus is on reading data from files, processing that data in some way, and writing output to files. If I find myself using (in Perl) the system command, back ticks or (in python) the subprocess module too extensively, I consider writing the script in bash. On the other hand, I sometimes start adding so much functionality to a bash script that eventually it makes more sense to rewrite it in Perl/python rather than deal with bash's limited (by comparison) support for variable scoping, functions, data structures, etc.

1
  • 2
    I used t0 also bail on Bash whenever file reading/text generation was required, but after learning the =~ operator supported by [[ ]] I have been loving me some Bash for simple file reading May 15, 2014 at 20:32
19

I like the criteria set out in this blog post.

  • If there aren’t any arguments to pass in, it’s probably a shell script.
  • If there isn’t much for control logic (besides a single loop or if/else) it’s probably a shell script.
  • If the task is automation of command-line instructions it’s almost definitely a shell script.
9

I found this Perl versus Bash analysis useful...

http://blogs.perl.org/users/buddy_burden/2012/04/perl-vs-shell-scripts.html

For convenience, I'm copying a summary of that author's 1) when bash is better findings and 2) when perl is better conclusion...

When Bash is Better...

  • Job Failure
  • Commands on Exit
  • Processing Job Output Lines
  • Here Documents
  • File Equivalencies
  • File Timestamp Comparisons
  • Tilde Expansion

When Perl is Better...

Perl still beats the crap out of bash for most applications. Reasons I might prefer Perl include (but are not limited to):

  • It’s going to be faster. Mainly because I don’t actually have to start new processes for many of the things I want to do (basename and dirname being the most obvious examples, but generally cut, grep, sort, and wc can all be eliminated as well).
  • String handling in bash is rudimentary at best, and the whole $IFS thing is super-clunky.
  • Conditionals in shell scripts can be wonky.
  • Quoting in shell scripts can be a nightmare.
  • bash’s case statement leaves a lot to be desired beyond simple cases (NPI).
  • Arrays in bash suck. Hashes in bash (assuming your bash is new enough to have them at all) suck even harder.
  • Once processing files or command output goes beyond the simple case I listed above, Perl starts really smoking bash.
  • CPAN.

So it’s not like bash is going to take over for Perl any time soon. But I still find, after all these years, that many times a simple shell script can sometimes be simpler than a simple Perl script. As I say, I welcome all attempts to convince me otherwise. But, then again, there’s nothing wrong with having a few different tools in your toolbox.

1
  • 1
    Interestingly, in Ruby, all (?) of the points do not apply, because Ruby has at_exit(), realpath(), expand_path(), stat(), heredocs, and exceptions (fail "error" unless system("foobar")). Jul 4, 2015 at 14:14
6

In my experience bash versus python is a trade off between development time and flexibility. A rudimentary solution to a problem can usually be established in a bash script more quickly than it can be in a python script.

Python will tend to make you think more about the structure of your solution than will the equivalent bash script. Python has more expressive power than a bash script and so it tends to scale and modify better over time. It also remains more readable in general.

Bash is closer to the file system and can be great for first draft solutions to problems which are NOT well defined. For this reason, a bash script might be a good first choice to prototype something in with the full intention of porting it to python once the problem is better understood.

5

Bash is a Unix shell it includes a scripting language. It is rather command processor. you control the way how you run commands, you actually run them.

Perl/Ruby/Python are general purpose languages.

When you want a shell script, you use Bash

If you want more complex task or not related to shell. Use Python etc.

I would never compare these languages actually. Python etc. are portable. You can run them anywhere. Bash is for Unix only.

Python etc. have tons of reusable libraries solving millions of tasks.

It's almost the same if you ask. "When to use Paint and when to use Photoshop"

For processing emails I would use Ruby, again, because it has a lot of reusable libraries.

But the best way would be combine bash and ruby. That would be right. Like you create a email processing script in ruby and bash script would invoke that ruby script and run other commans ds.

So whenever you need command processor you use bash. You run unix commands and control them.

UPDATE after 7 years (Mar 2019)

While the main part of my answer didn't change I would like to point out that.

Bash is a powerful scripting language too. For text processing it could an absolute legit choice.

Please read mkaito's comments below. They are all completely true.

5
  • 1
    Bash's scripting subset is just as general purpose as any general purpose scripting language. Add general programs like sed into the mix, and you got yourself covered for 90% of all scripting needs you'll ever have.
    – mkaito
    Apr 21, 2012 at 3:06
  • 1
    bash is a command processor so its power is to run other unix commands and programs in their scripts.like you mentioned sed etc. He is asking for when to choose certain language
    – bakytn
    Apr 21, 2012 at 3:16
  • 1
    The shell is a glorified program launcher indeed, but it's scripting capabilities are far beyond that. I suggest you learn some real shell scripting.
    – mkaito
    Apr 21, 2012 at 3:34
  • I can sleep on the floor but i would rather do it on a bed. The languages cant be compared they have dofferent purposes.
    – bakytn
    Apr 21, 2012 at 3:50
  • 2
    Well, if Bash is the floor, something like Haskell must be a nailboard :-)
    – mkaito
    Apr 21, 2012 at 14:15
1

Shell scripts like bash, ksh, zsh, sh, and fish are notoriously surprising, compared to high level general purpose languages like Ruby, Python, or Perl. While a shell script may begin its life as a shorter file than the equivalent general purpose script, the surprises lead to lots of defensive wrapping code, such as set -euo pipefail to enable strict modes.

For exampmle, most shell languages continue executing lines in a shell script, even when one of the commands fails. By contrast, a general purpose language fails immediately on the first error, and often loadly, resulting in safer, more predictable behavior in scripts of even light complexity.

1

Highly biased article.

I don't find bash to be all that hard to debug. Python is often much too rigid, whereas bash allows you to be very creative. If you're a good out-of-the-box thinker, you'll like bash.

I run bash scripts on millions of DNA sequencing reads in thousands of files, and it serves me just fine. And contrary to what everyone says, the same versions of the scripts in C++ don't actually run all that much faster (on a few minutes separate them).

I think bash, like perl, is not the most user friendly/easy to read. That scares people away because most people are not great abstract thinkers. But brighter, more creative programmers tend to love it and make frequent use of it. If you know yourself and you know you have a brain, don't be scared off by bash. If you're a basic thinker, maybe stick to something like Python. To each their own.

1

As rule of thumbs, use the simplest language that has good enough performance for your task at hand. And its particularities only to the extend they are truly useful.

And about readability, Bash is horrible if your programming style is horrible. If you just throw code over there, it becomes obscure.

But if you split code into the shortest functions, and name things in a simple understandable manner, it's the clearest language you can find. Because it's very terse.

As specimen, this is my latest code in Bash. Notice how easy to understand and to type it is:

#! /bin/bash


mainFunction () {
    file="${1}"

    checkFile "${file}"
    executeFile "${file}"
}


changeToThisProgramDir () {
    cd "$( dirname "${BASH_SOURCE[0]}" )"
}


checkFile () {
    file="${1}"

    checkFileNotEmpty "${file}"
    checkFileExist "${file}"
    checkFileIsExe "${file}"
}


checkFileExist () {
    file="${1}"

    if [ ! -f "${file}" ]; then
        echo "The file doesn't exist: ${file}" >&2
        echo "If the name was correct either type: exeCute \"pathToYourExeFile\""
        echo "Or just open with exeCute from the file manager"
        exit 1
    fi
}


checkFileIsExe () {
    file="${1}"
    mime=$(fileMime "${file}")

    if [ "${mime}" != "application/x-dosexec" ]; then
        echo "Not an exe: ${file}" >&2
        exit 1
    fi
}


checkFileNotEmpty () {
    file="${1}"

    if [ "${file}" == "" ]; then
        echo "No file specified" >&2
        echo "Either type this: exeCute \"pathToYourExeFile\""
        echo "Or just open with exeCute from the file manager"
        exit 1
    fi
}


execute () {
    function="${1}"
    command="${2}"
    error=$(eval "${command}" 2>&1 >"/dev/null")

    if [ ${?} -ne 0 ]; then
        echo "${function}: ${error}" >&2
        exit 1
    fi
}


executeFile () {
    file="${1}"
    type=$(fileType "${file}")

    if [ "${type}" == "MS-DOS executable" ]; then
        execute "executeFile" "dosbox \"${file}\" -forcescaler normal2x -exit -fullscreen"
    else
        execute "executeFile" "wine \"${file}\""
    fi
}


fileMime () {
    file="${1}"

    attributes=$(file --brief --mime "${file}")
    IFS=";" read -r -a attributes <<< "${attributes}"
    echo "${attributes[0]}"
}


fileType () {
    file="${1}"

    attributes=$(file --brief "${file}")
    IFS="," read -r -a attributes <<< "${attributes}"
    echo "${attributes[0]}"
}


changeToThisProgramDir
mainFunction "${@}"
0

From the "Llama Book,"

Perl tries to fill the gap between low-level programming (such as in C or C++ or assembly) and high-level programming (such as “shell” programming [e.g., bash]). Low-level programming is usually hard to write and ugly, but fast and unlimited; it’s hard to beat the speed of a well-written low-level program on a given machine. And there’s not much you can’t do there. High-level programming, at the other extreme, tends to be slow, hard, ugly, and limited; there are many things you can’t do at all with the shell or batch programming if there’s no command on your system that provides the needed functionality. Perl is easy, nearly unlimited, mostly fast, and kind of ugly.

Not the answer you're looking for? Browse other questions tagged .