How do I learn bash shell scripting?

Question

I am a mac user and a web developer and I would like to create automated tools to manage my workflow. Say a single command to start up a type of website.

What parts of bash shell scripting should I learn? There are so many:

• sed
• awk
• grep
• find

Or should I be learning something else to archive command line automation for my web projects?

Answer 1

You need to be aware that different from other scripting or programming languages, shell scripting heavily relies on the command-line applications provided to you by the OS. This means, since OS X is a BSD, that much of the things that work on Linux, will work slightly different on your computer, rendering a lot of tutorials useless. In general, you can expect your command line tools to have fewer features and flexibility than the Linux counterparts, although there are exceptions.

if you're unfamiliar with the shell and don't feel an overwhelming need to learn it, you might be better off using your favorite web scripting language (PHP, Ruby, Python) from the command line. The advantage there is the probably more consistent API across the entire library of functions.

---

In bash itself, learn

• Conditions (including use of test / [...] for defensive programming when handling files)
• Loops
• Variable assignment
• Functions
• Input and output redirection and piping
• Calling programs and shell functions (including how to capture their output and read their return values), both regular calls and with command substitution.
• Options. I love set -u and refuse to work with scripts that cannot handle this.
• Variable types, arrays, globbing

The problem here is that bash is very tied into the system with things like job control, and, for creation of complex scripts, has a rather steep learning curve.

In my experience *no section of the bash man page is unnecessary. Do it long enough and everything in there is applicable somehow (and if it's just to answer questions on SU).

---

Some (in my experience) commonly used programs in shell scripts:

• File handling and navigation: find, ls, cd, rm, mv, cp, chmod, chown, chflags, dd, ln
• Program execution: xargs, find -exec
• Data transformation: sort, wc, cat, sed, awk, grep

Specific to OS X are probably the following:

launchctl, open, dscl, PlistBuddy, plutil, osascript, automator, networksetup, systemsetup, growlnotify (if you have Growl)

And of course, depending on the exact nature of what you want to do, there are others...

date (BSD variant — very different from GNU), mount, fsck, kill, sendmail

---

Some, e.g. netcat require you to install third party packages e.g. using Homebrew, MacPorts, or Fink.

If your scripts heavily use these command line programs, or can be written in such a way, bash scripts are a good way to automate this.

---

Since you're on OS X, also look into AppleScript (via AppleScript Editor) and Automator. Both of these can be combined with shell scripts to achieve quite nicely integrated script solutions. If you're looking into controlling other OS X applications (e.g. your web browser), these are indispensable. While you can open URLs in different browsers using open on the command line, retrieving data e.g. from loaded tabs requires Automator or AppleScript.

Also, other scripting languages such as ruby, python are also pre-installed on your Mac and can be used to easily create simple helper utilities. These are more of a replacement of bash and regular UNIX utilities. It depends on the nature of what you want to do.

If you want to move a handful of files around your disk, zip them up and transfer them via SSH, bash is probably the best choice. As soon as you have real programming logic (anything involving a lot of objects or numbers), you should probably look into other scripting languages.

Answer 2

I've posted a list of (OS X and bash centric) bookmarks at OS X Terminal Tutorials - Super User.

One of the best ways to learn a scripting language is by modifying existing scripts. Take a look at stuff like:

• brettterpstra.com/tag/bash
• hayne.net/MacDev/Bash
• gist.github.com/Lri
• lri.me/urlalias
• github.com/holman/spark

A few books with freely downloadable code examples:

• Wicked Cool Shell Scripts
• Bash Cookbook

Answer 3

I think for you, just Basic usage of about 20-25 commands is enough, and basic architecture of Linux is must. Depending upon your work type,you can start learning about these bash commands only:

vi,ls,find,sed,awk,grep,cat, service startup and shutdown, netstat,nmap,traceroute,ping,mv,cp,rm etc.

These are most common commands, which are used in bash.

Besides that, you can follow this tutorial:

http://www.faqs.org/docs/bashman/bashref_toc.html

http://blog.emson.co.uk/2009/06/18-useful-bash-scripts-for-web-developers/

Also, you need to learn basic shell scripting to automate some processes.