You'll get your Mac news here from now on...

Help TMO Grow

Subscriber Login

Advertising Info


The Mac Observer Express Daily Newsletter


More Info

Site Navigation

Home
News
Tips
Columns & Editorials
Reviews
Reports
Archives
Search
Forums
Links
Mac Links
Software
Reports
Contact

Mac OS X Command Line 101
by Richard Burton


Special Characters & The Command Line
Part VI of this series...
May 31st, 2002

"'Quotes, unquotes, and quotes.'"
"That's three quotes?"
"Yes."
"Add another quote and make it a gallon."
-
Zeppo and Groucho Marx, Animal Crackers

This series is designed to help you learn more about the Mac OS X command line. If you have any questions about what you read here, check out the earlier columns, write back in the comments below, or join us in the Hardcore X! forum.

Depending on how you look at it, Unix has either three or four forms of quotes (as if you couldn't guess from the header), each useful in its own way. Quoting allows you to suppress the special meaning of certain characters, or to enforce other special characters, or even embed commands within commands.

So far, we have seen that the Unix shell attaches certain meanings to particular characters, such as the tilde (~). This puts great power at your fingertips; however, it can also make things a bit sticky in certain circumstances. Suppose you do the following from your command line:

    [localhost:~] dr_unix% ls -d Documents/A*
    Documents/Acrobat User Data             Documents/AppleWorks User Data
    [localhost:~] dr_unix% ls -d Documents/Acrobat User Data
    ls: Data: No such file or directory
    ls: Documents/Acrobat: No such file or directory
    ls: User: No such file or directory
    [localhost:~] dr_unix% 
    
What gives? Well, remember that the shell uses the spaces as the delimiter between elements in a list. Or, translating from Geek to English, when you give ls a list of files to list, it assumes that the spaces separate the file names. Thus, it looks for three files: Documents/Acrobat, User, and Data. The space is a special character to the command line. The best way to get around this is to enclose the filename in quotes:
    [localhost:~] dr_unix% ls -d "Documents/Acrobat User Data"
    Documents/Acrobat User Data
    [localhost:~] dr_unix% 
    
This tells the shell "ignore the special meanings of the characters inside the quotes". In this case, it is equivalent to saying "a space is not a filename separator, but part of a filename". The reason you don't need to do this when you do something like ls * is that, behind the scenes, Unix and tcsh are clever enough to know that things like spaces in filenames can cause issues, so they are dealt with quietly. Still, you can see why Unix people prefer to use the '_' to separate words in a file or directory name; it avoids a lot of trouble. Apple has already been bitten by this. Remember the updated version of iTunes and how it could, under certain circumstances, delete a disk if the name of the disk had a space in it? Apple forgot the quotes.

In general, it is best to avoid special characters where they might cause confusion. However, we live in the world as read, and sometimes you just have to deal with other people's decisions. In addition, we will eventually want to create our own tools, and it only makes sense to be aware of them anyway. So just what are these special characters that can cause trouble to us command-line mavens?

Characters Use
* ? [ ] ^ { } ~ Filename pattern matching and expansion
$ Used to denote variables
! ^ History reference and substitution
& Used to send commands to the background
| Pipe
> < Redirects output/input
; Command seperator
space Argument seperator
tab Filename completion (in tcsh)
esc Filename completion (in csh, may cause problems in tcsh)
( ) Subshell execution
` Quote used for command substitution
\ ' " Quote characters

Some of these will be familiar, others will induce a condition known as MEGO (My Eyes Glaze Over). For now, don't worry about them except to note that they can cause problems if you aren't aware of them. We'll concentrate on the four characters used in quoting: \, ', ", and `.

The \ is an escape character; it turns off the special meaning of the character following it. For example, earlier we used the double quote to turn off the special meaning of spaces in a filename.

    [localhost:~] dr_unix% ls -d Documents/A*
    Documents/Acrobat User Data             Documents/AppleWorks User Data
    [localhost:~] dr_unix% ls -d "Documents/Acrobat User Data"
    Documents/Acrobat User Data
    [localhost:~] dr_unix% 
    
However, Unix being Unix, we could also have done this:
    [localhost:~] dr_unix% ls -d "Documents/Acrobat User Data"
    Documents/Acrobat User Data
    [localhost:~] dr_unix% ls -d Documents/Acrobat\ User\ Data/
    Documents/Acrobat User Data/
    [localhost:~] dr_unix% 
    
What happens is that the \s are telling the shell "the character after me has a special meaning; ignore that meaning and use it literally". Therefore, the shell interprets the command argument Documents/Acrobat\ User\ Data/ as one file, not three.

The two "normal" forms of quoting are the single quote (') and the double quote("). Why two forms? Well, for one thing, if what you have to say involves a single quote or double quote, one can turn of the special meaning of the other. Consider the echo command, which simply spits out whatever list you give it:

    [localhost:~] dr_unix% echo This is a test.
    This is a test.
    [localhost:~] dr_unix% 
    
Now, what if you want echo to print something with an apostrophe in it?
    [localhost:~] dr_unix% echo You can't say that
    Unmatched '.
    [localhost:~] dr_unix% echo "You can't say that"
    You can't say that
    [localhost:~] dr_unix% 
    
Without the double quotes, echo interprets the ' as a quote character and looks for another one to mark the end of the quote. However, there is not an ending quote, so the shell spits back an error. Enclosing the list in double quotes turns off the special meaning of the single quote.

And of course, you can do the reverse.

    [localhost:~] dr_unix% echo 'You can"t say that'
    You can"t say that
    [localhost:~] dr_unix% 
    
In fact, one can even get silly with them, thus:
    [localhost:~] dr_unix% echo '"' "'"
    " '
    [localhost:~] dr_unix% 
    
What's more, if you want to use both the " and ', you can use the backslash to escape both of them:
    [localhost:~] dr_unix% echo Gee isn\'t that \"swell\".
    Gee isn't that "swell".
    [localhost:~] dr_unix% 
    
That is a lot simpler and less error-prone than:
    [localhost:~] dr_unix% echo Gee isn"'"t that '"'swell'"'.
    Gee isn't that "swell".
    [localhost:~] dr_unix% 
    
It should be noted that, in spite of the examples I've shown so far, single and double quotes have slightly different behavior. Obviously, they interpret each other as regular old characters, as we've seen above. In general, single quotes are more strict. They suppress the special meanings of all characters except for the ! to denote history substitution. (We'll get to history and history substitution in a later column; for now, don't worry about it.) The double quotes ignore the special meaning of everything between them except for the ! for history substitution, the $ to indicate a variable (again, more on that later), and the backticks (`command`) for command substitution.

[Before moving to command substitution, I should note that some shells, notably the Bourne shell and Korn shell, handle single and double quotes a bit differently than csh and tcsh do. So if you are not using the default tcsh in OS X, you may not be able to duplicate this behavior with exactly the same commands.]

[You have been warned.]

The last form of quoting, using backticks (`), is probably the most powerful. What backticks do is take the command enclosed in them, run the command, and place the output of that command into the command line. An example will best illustrate this.

Remember our old friend, the wc command, which returns a count of the characters, words, or lines in a file, depending on the command line option? It is used thus:

    [localhost:~] dr_unix% wc -l Adam.txt test_1.txt who_list
           1 Adam.txt
           1 test_1.txt
           5 who_list
           7 total
    [localhost:~] dr_unix% 
    
Now, what if you wanted the information for only certain files? Well, you could, of course, type out a whole list of files. However, that isn't fun. What you'd like to do is have the system determine which files you want, and then run the wc command on those files, and do it all on the fly. Wouldn't that be neat? Wouldn't that appeal to your laziness? Naturally, in Unix, there is a simple way.

Let's say there are some number of files ending with ".doc" in your ~/Documents directory. At any time, the names and numbers of these files can change. And you want to know 1) what files are there, and 2) how many words are in each. You know you can use wc -w to get a count of the number of words for a list of files. You also know that you can use ls ~/Documents/*.doc to get a list of those files. So if you can combine them, you don't need to write your own tool from scratch.

You could change directories (cd ~/Documents) and try wc -w *.doc but that's requiring more steps than you want. Besides, you want to be able to run this command from anywhere, and you don't want to change your directory. How to do it? Use the other form of quotes, backticks.

    [localhost:~] dr_unix% wc -w `ls ~/Documents/*.doc`
         650 /Users/dr_unix/Documents/6197.doc
        1443 /Users/dr_unix/Documents/ISIHAC_LOG.doc
        2093 total
    [localhost:~] dr_unix% 
    
Here's how command substitution works. The shell parses the command you send,

wc -w `ls ~/Documents/*.doc`

and sees the `s. It takes the text in between the two `s,

ls ~/Documents/*.doc

and runs that as a command. The ls command returns two files:

/Users/dr_unix/Documents/6197.doc
/Users/dr_unix/Documents/ISIHAC_LOG.doc

This output is then substituted for the ls command in the command line:

wc -c /Users/dr_unix/Documents/6197.doc /Users/dr_unix/Documents/ISIHAC_LOG.doc

This command is run, and the results are printed to the screen.

         650 /Users/dr_unix/Documents/6197.doc
        1443 /Users/dr_unix/Documents/ISIHAC_LOG.doc
        2093 total
    [localhost:~] dr_unix% 
    
Admittedly, this example is pretty contrived. You could just as easily type in wc -c ~/Documents/*.doc and get the same result. After all, the ls command is just building a list of files, and the '*' wildcard is doing the same thing. However, I chose this example because 1) I wanted to show how command substitution works, 2) this example is fairly straightforward, 3) it is a reminder that, in Unix, there is more than one way to skin a cat, 4) it is a powerful tool that, as we build up our command-line vocabulary, will become more and more important, and 5) it hints at some of the power behind the simple things we've done so far.

Backticks are just one example of how a Unix shell will allow you to combine commands. We will examine more of these after we discuss a related topic, redirecting input and output.

You are encouraged to send Richard your comments, or to post them below.


Most Recent Mac OS X Command Line 101 Columns

Command Line History & Editing Your Commands
November 22nd

Pico: An Easy To Use Command Line Editor
November 1st

Understanding The "grep" Command In Mac OS X
October 4th

Command Line History & Editing Your Commands
September 6th

Mac OS X Command Line 101 Archives

Back to The Mac Observer For More Mac News!


Richard Burton is a longtime Unix programmer and a handsome brute. He spends his spare time yelling at the television during Colts and Pacers games, writing politically incorrect short stories, and trying to shoot the neighbor's cat (not really) nesting in his garage. He can be seen running roughshod over the TMO forums under the alias tbone1.



Today's Mac Headlines

[Podcast]Podcast - Apple Weekly Report #135: Apple Lawsuits, Banned iPhone Ad, Green MacBook Ad

We also offer Today's News On One Page!

Yesterday's News

 

[Podcast]Podcast - Mac Geek Gab #178: Batch Permission Changes, Encrypting Follow-up, Re-Enabling AirPort, and GigE speeds

We also offer Yesterday's News On One Page!

Mac Products Guide
New Arrivals
New and updated products added to the Guide.

Hot Deals
Great prices on hot selling Mac products from your favorite Macintosh resellers.

Special Offers
Promotions and offers direct from Macintosh developers and magazines.

Software
Browse the software section for over 17,000 Macintosh applications and software titles.

Hardware
Over 4,000 peripherals and accessories such as cameras, printers, scanners, keyboards, mice and more.

© All information presented on this site is copyrighted by The Mac Observer except where otherwise noted. No portion of this site may be copied without express written consent. Other sites are invited to link to any aspect of this site provided that all content is presented in its original form and is not placed within another .