::: CEE UCL HelpDesk: Introduction to Unix

:::

Introduction to Unix at CEE UCL

A Very Brief History of Unix

"In a word, don't reboot the machine at all. Period."
- Thomas Edison

Unix is a powerful computer operating system that was first created at the AT&T Bell Laboratories in 1969 on a cast-off DEC PDP-7. A small group of researcher wrote a small general-purpose time-sharing operating system, and named it Unix. In the process of developing Unix, the authors also had to create a new programming language called C (it was the third version of the language -- the first two versions were called BCPL and B). Since then, it has spread world-wide, with hundred of thousands of systems installed, from microcomputers to the largest mainframes.

Unix is very popular among the scientific, engineering, and academic communities due to its multi-user and multi-tasking environment, flexibility and portability, electronic mail and networking capabilities, and the numerous programming, text processing and scientific utilities available. It has also gained widespread acceptance in government and business.

There are several reasons the Unix systems are so successful. First, because it was written in C, it is portable -- Unix systems run on a range of computers from microprocessors to the largest mainframes; this is a commercial advantage. Second, the source code is available and written in a high-level language, which makes the system easy to adapt and modify to particular requirements. Finally, and most important, it is a stable operating system with rich and productive system capabilities.

Over the years, two major forms (with several vendor's variants of each) of Unix have evolved: AT&T Unix System V and the University of California at Berkeley's Berkeley Software Distribution (BSD). However, in a nutshell (called system kernel) they are quite similar. Examples of popular Unix variants include SUN Solaris and SunOS, IBM AIX, HP Apollo, SGI Irix, DEC Ultrix/OSF, BSD, Linux, etc. Sun SPARC workstations in CEE Unix Computing Laboratory (UCL) run Sun's version of Unix called Solaris which is an AT&T Unix System V variant.

It's All about Kernel...

When you use Unix, several layers of interaction are occurring between the computer hardware and you.

The first layer is the kernel, which runs on the actual machine hardware and manages all interactions with the hardware. All applications and commands in Unix interact with the kernel, rather than the hardware directly, and they make up the second layer.

On top of the applications and commands is the command-interpreter program, the shell, which manages the interaction between you, your applications, and all available Unix commands.

The third layer is a windowing system such as X (or X-windows depending on whom you ask). The windowing system usually interacts with the shell, but it can also interact directly with applications.

The final "layer" is you, the user. You will interact with the entire operating system through just the shell, or through a combination of the shell and the window system.

The default graphical environment/window system manager came with Solaris is called Common Desktop Environment or CDE (of course, OpenWindow comes automatically with any Sun machine). CDE interface based on X11/R6 (X version 11 Release 6) and Motif GUI (an enhanced extension of X). There are various types of screen modes, called terminal types that are X-oriented (i.e., GUI terminal such as "xterm" and "dtterm"), and a few that are text-oriented (i.e., text-only terminal such as "vt100" and "tty"), which is what remote users normally get, i.e., "pine" e-mail program in ODU student e-mail account.

Some Precautions

"Time is the best teacher. Unfortunately, it kills all its pupils"
- Hector-Louis Berlioz

There are a few important items that I would like you to remember to effectively use the computing systems here in CEE UCL.

Please realize, however, that you are not dealing with DOS/Windows/MacOS anymore. You cannot simply turn off a machine without risking damage to the machine, terminating programs that might have been running for days, and angering all the people who happen to be logged in the network, both on-site and remote, not to mention bringing down the wrath of the system administration. Rebooting a machine would tend to cause you many more problems than you thought you had when you started reaching for the power. If you need help, please ask someone in the lab first.
You should never share your account password with anyone. If someone discovers (or you tell someone) your password, not only will they have access to your personal files, but they will have a much better chance to launch attacks against the security of the entire system.
Your account password is the key to accessing and modifying all of your files. If another user discovers your password, he or she can delete all your files, modify important data, read your private correspondence, and send mail out in your name. You can lose much time and effort recovering from such an attack.
The best way for you to learn about the Unix in general is to experiment, and to learn from man (=manual) pages that are available on-line. To read on-line manual for a specific command you're curious/stuck with, use man command. If you would like to read the manual on the manual or how to use the manual, for example, you need only to type "man man." That will get you started. We'll talk about this man page again a little bit later.
Just like in any other laboratory environment, use your common sense on how to behave in the laboratory and be always courteous to other users - don't be a hog. Since Unix is a multi-user, network operating system after all. Threatening you with your account and forcing a plethora of red tapes to make you comply with the CEE UCL laboratory policy (yes, the long one) is the last thing in the wish list. Please help maintaining current "common sense"-based policy.

Unix Crash Course 101

"Pluralitas non ponenda est sine necessitate"
- Multiplicity is not to be asserted when it is unnecessary

I realize that many of you have never had the experience with the Unix system. I repeat, the best way for you to learn about the Unix is to experiment, and "Read The Fine Manuals" (RTFM) that are available to you. Of course, introductory books and references from the library (you know, the place where they keep all the sci-fi stories) and other sources are always useful additions for accelerating your learning curve.

For example, the system administrators (a.k.a. Unix gurus) are the people responsible for maintaining the system in CEE UCL. Reading the instructions and experiment is how they learned what they know, and how you can learn what you need to know to maximize your experience with Unix.

Before proceeding to "Unix Crash Course 101," there're several important idiosyncrasies of Unix you should know in beforehand;

Unix is case sensitive. This means that Unix distinguishes between uppercase and lowercase letters, i.e., Bill and bill don't mean the same thing to Unix. Most command names and files are entirely in lower-case. Therefore, you should generally plan to type in lower-case for most commands, file and directory names.
There are a number of different "flavors" of Unix shells (called command interpreters) available in CEE UCL network. By different "flavors" I mean different shells that handle your input in their own unique way. Depends on the type of your account, your default login shell could be the Bash (Bourne Again shell) shell or tcsh (T-shell) shell. Majority of the Unix topics in this exercise will be identical with other shells, but there'll be some shell-specific differences.
It is possible to determine which shell is in use by typing "echo $SHELL" at the command prompt. The response for the Bash shell would be "/usr/local/bin/bash*" that "*" stands for its version number. Another popular shell is the Tc shell (=tcsh) which would respond with "/usr/local/bin/tcsh."
Unix commands can only be entered at the shell prompt. Unix commands begin with a command name, often followed by flags and arguments (options, filenames and/or other expressions) and ends with a key. In a sense, Unix commands are similar to verbs in English.
The option flags act like adverbs by modifying the action of the command, and filenames and expressions act like objects of the verb. The general syntax for a Unix command is:

command [flags] argument1 argument2 � � �

The brackets around the flags is a shorthand way to indicate that they are often optional, and only need to be invoked when you want to use that flag. Also, flags need not always be specified separately, each with their own proceeding hyphen (or frequently referred as "minus sign"). Many times, the flags can be listed one after the other after a single hyphen. You'll find some examples later on (such as " ls" and "ps" commands).
For the time being, following example would illustrate typical command syntax:

chmod -R 755 test_99

"chmod" is the program called (=command), "-R" is the flag, and "755" and "test_99" are arguments. This command tells the computer to change the permission level of directory "test_99" and its subdirectories to "755" recursively.

If output scrolls up on your terminal screen faster than you can read it, you can suspend it by typing . To resume the display, type . will abort a process, and will discard the output to the screen until another is entered. Be sure to note that although the output doesn't appear, the process is still running. suspends the current program. You can resume the suspended program by giving the "fg" (foreground) command, or resume it in the background with "bg." Of course, you need to find program's process ID first using "ps" command.

Here's a quick summary of special keys that are frequently used in Unix.

Key Combination	It does
Ctrl-h	^h erases the last character on the command line. (=Backspace key)
Ctrl-c	^c interrupts a command or process in progress and returns to the command line. This will usually work; if it doesn't, try typing several ^c's in a row. If it still doesn't work, try typing ^\, q (for quit), exit, ^d, or ^z.
Ctrl-d	^d generates an end-of-file character. It can be used to terminate input to a program, or to end a session with a shell, i.e., logout.
Ctrl-s	^s stops the flow of output on the display, i.e., halt. To return back to the normal state, use Ctrl-d (^q)
Ctrl-u	^u erases the entire command line. It is also called the "line kill" character.
Ctrl-w	^w erases the last word on the command line.
Ctrl-z	^z suspends a command or process in progress.
Ctrl-\	^\ quits a program and saves an image of the program in a file called "core" for later debugging.

Unix is based on a multi-user computing environment. When you start a Unix session on a workstation, you are placed in a directory that contains your files. This directory is called your home directory or login root directory. For example, a user "wclinton" would have a home directory at "/home/wclinton." You can create, copy, move, and remove files as well as create subdirectories that reside in your home directory. However, you can not create, copy, move, and remove files/subdirectories in other users' home directories, because you do not have permissions on them.
If you want to be proficient with Unix, you need to be familiar with six basic elements of Unix. They are: commands, files, directories, your environment, processes, and jobs. We will explore each of these elements in a little greater detail later on, but first you need to learn how to initiate a session on a Unix system.

First Log in, Password, Log out

"'I see!', said the blind carpenter, as he picked up his hammer and saw"
- William Shakespeare, MacBeth

In order to use a workstation operating under Unix you need to "log in." This "log in" process attempts to protect against unauthorized use of the computer equipment. It also lets each user define their own personalized working environment on the same workstation and even work on the same workstation at the same time (both directly and remotely).
To log in to a workstation, you are then asked to enter your login ID (= user ID), and password. Login to the CEE UCL system with your login ID and initial password provided by Dr. Yoon. Keep in mind Unix commands are all case-sensitive. The characters will not appear when you type your password to provide confidentiality, so type carefully!

login: [type your Unix login ID]
password: [type your initial password]
Since you're login for the first time, it will take some time to create and prepare your home directories/mail system/desktop settings, etc. The window-like desktop environment is called Common Desktop Environment (CDE) and you can use the icons to access various system resources and applications. E-mail, personal preferences, editors, printers, etc. and exit capabilities are all offered through point-and-click access.
When "Desktop" is ready, press and hold down right mouse button anywhere in the "Desktop." "Workspace Menu" will pop up. While holding down right mouse button, select "Program," then select "Console" and release mouse button. You'll have a new text window (or "Terminal") with system command prompt. You are ready to begin entering Unix commands.
The word "Console" means that you're physically sitting in front of the actual workstation, using its keyboard and viewing its display screen. (in contrast to a 'Terminal' that is what you get when you log in from a remote location via network)
The system prompt in your "Console" should be in forms of

your login ID: [workstation name] Current_diretory>

and it is the computer's way of saying "OK, I'm ready! What do you want me to do?" For example, user ID of "napoleon" logins to a workstation named "sparky" will have a system prompt look like

napoleon: [sparky] ~>

Note that "~" always represents your home directory. (we'll talk about it little bit later in detail)
From now on, the system prompt "your login ID: [workstation name] Current_diretory>" will be abbreviated as a single "$" sign.
First, you need to validate/update your user information since this is the first time you ever log in to CEE UCL network. When the UCL system administrator created your user account, only a skeleton user information was created, and that user information is call a 'User Credential' in Sun Solaris system. In order to gain access to CEE UCL network, you need to validate & update your User Credential. Without a valid User Credential, you will not be able to have any access to any workstation in CEE UCL, neither directly nor remotely.
Now, update your user credential by typing following command. (Do not type "$" sign as a part of command. "$" means the system command prompt.)

$ rlogin rhino (remote login to the network server)
$ chkey -p
Updating nisplus publickey database.
Generating new key for 'unix.XXX@cee.odu.edu'
Please enter the Secure-RPC password for [your User ID] : [type your login password]
Please enter the login password for [your User ID] : [type your login password]

Congratulation, you become a valid user with a full user credential.
Next step is to change initial temporary password to a password of your own choice.
When you first receive your account information from Dr. Yoon, you will probably be given a temporary password (that you just used to log in). You should change your password to something else right now.
Password is such an important security measure in Unix. When you change your password, you MUST pick new password that is hard to guess. Password is case-sensative (upper and lower case mean different things). You should pick a password at least 6 characters-long, including at least one number AND one upper case character.
Don't neglect creating a "good" password. A good password should be easy to remember for you but hard for others to guess. Words in the dictionary, nicknames, and anything related to you are poor choices for a password. One way of generating a password is to use the first letter of each word in a strange yet memorable sentence. For example, Apwh9w could be your password based on the sentence; A pig would have 9 wings.
Occasionally, the system administrators may run standard password-cracking utilities on the accounts. If they can guess your password (passwords that can be found in a dictionary are the easiest!) you will find out rather abruptly. If this all seems extreme, please realize that I'm not anxious for your account to be cracked by someone in Bulgaria, for example, to be used as a convenient way to probe and hack the security of someone else's computers.
Changing to a new password is done with the "nispasswd" command. After typing "nispasswd," you will be prompted first for your current temporary password and then twice for a new password.

$ nispasswd
nispasswd: changing password for [your User ID]
Enter login (NIS+) passwd: [type your login password, i.e., current password]>
New password: [type password of your choice, i.e., new password]
New password: [type new password again to confirm]

You can change your password anytime you want using the same "nispasswd" command. Make it a habit to change your password regularly.
Your User credential is now updated. DO NOT forget your new password. It's your sole responsibility to remember your new password. You should never share your account password with anyone. Exit from the remote server to the console workstation.

$ exit
[log out from the remote server and get back to your console]
$
[You should see console workstation name in your prompt now]
Press and hold down right mouse button anywhere on the "Desktop." 'Workspace menu' will pop up. While holding down right mouse button, select "Program'," then click on "Terminal," and release mouse button. You'll have a second text window with system command prompt.
Try various on-line introductory materials from your new Unix environment. Click on "Help Viewer" icon displayed in the "CDE Desktop Panel" at the bottom of the screen. (it is the second icon from the right side, or the one next to "Trash bin" icons)
I'd highly recommend you to go through on-line introductory materials: "Common Desktop Environment" and "Overview and Basic Desktop Skills" to become familiar with your new Unix environment.
To logout (or Quit), press and hold down right mouse button. Once again, "Workspace" menu will appear. While holding down right mouse button, select "Logout, " and release mouse button. Click on "OK" confirmation button to logout.
If you click "Logout" and walk away without confirming logout, your account will still be accessible to any other person in the lab. It is like leaving your car unlocked with key inside. Never ever let this happen. Another way to logout is to click on "Exit" icon in the Desktop Panel at the bottom of the screen.
Keep in mind that you can't just turn a workstation off when you're done, since other people are still using it. Therefore, NEVER turn off a workstation.

Do and Don't of Managing Your Password

There are some key points you need to remember to protect yourself and your account. If you take a close look at following "do and don't,", they are basically flat out good common sense. (However, we're living in an era when "good" common sense becomes a rare commodity) Remeber, 99% of security compromise is resulted from careless holes in user's side not from outside hackers.

Remember, "security" means preventive proactiveness, and you should never be confused it with "scooping aftermath."

NEVER give your password to *ANYONE*

The whole purpose of having a password in the first place is to ensure that *NO ONE* other than you can use your account.

NEVER write your password down

Especially never write your password anywhere near your computer.

NEVER let anyone look over your shoulder
when you enter your password

"Shoulder surfing" is the most common way that accounts are hacked. Here's a common sense password etiquette you may take a look.

NEVER e-mail your password to anyone

Sounds so evident however you'd be really surprised to find out how many people completely disregard the security when e-mailing. Remember, your e-mail is by nature a unencrypted, text file that anyone can read if one can get a hold on yours.

DO change your password on a regular basis

There is no better way to thwart a would-be hacker/cracker than to change your password as often as possible. Your system administrator should be able to tell you your system's recommendation on how often you should change your password, but a good rule of thumb is to change it at least every three to six months. (I do agree with you on that this is such a hassle, however)

DON'T pick a password that is found in the dictionary

When you set your password, it is encrypted and stored into a file. It is really easy for a "hacker/cracker" to find your password by encrypting every word in the dictionary, and then looking for a match between the words in his encrypted dictionary and your encrypted password. If he finds a match, he has your password and can start using your account at will.

NEVER use your user id as your password

This is the easiest password to crack. Yet sounds unbelievable, quite number of users are still doing it. If you're one of them, change your password right now!

DON'T choose a password that relates to you personally
or that can easily be tied to you

Some good examples of BAD passwords are: your name, your wife/husband/sons/daughters' names, your relatives' names, your dogs/cats/pets' names, nicknames, birthdates, license plate numbers, social security numbers, work ID numbers, and telephone numbers. No, this is about neither dealing with an espionage case nor getting "eternally" paranoid. It is just a good common sense!

DON'T use passwords that are foreign words

The hacker can get a foreign dictionary, and ...

DO use a password that is at least . . .

eight characters long and that has a mix of letters and numbers. The minimum length of a password should be no shorter than six characters long.

NEVER use the same password on different systems or accounts

Another common mistake that we all make. Think why you're using a password in the first place.

ALWAYS be especially careful when you telnet or rlogin . . .

to access another computer over the Net. When you telnet or rlogin, your system sends your password in plain text over the Net. Some crackers have planted programs ("snoopers") on Internet gateways for the purpose of finding and stealing these passwords. If you have to telnet frequently, change your password just as frequently. If you only telnet occasionally, say, for a conference trip out of state/oversea, set up a new password (or even a new account) just for the trip. When you return, change that password (or close out that account).

Techniques for Generating "Good" Passwords

"Never trust anybody who says "trust me." Except just this once, of course"
- John Varley

The best passwords - the ones that are the easiest for you to remember, and the ones that are the hardest for crackers to crack - are passwords that are like those fake words you used to create when you would cram for a test.

For example, to remember that "the Law of Demand is the inverse relationship between price and quantity demanded," I created the word tLoDitirbp&qd. No one could hack that as a password. Best of all, it is EASY to remember (well, its easy for an Economist to remember).

Here an example for generating good passwords:

Sentence	Possible password
a big fat Pig would have 9 wings	abfPwh9w
In 1995 we had SNOW in Norfolk	I95whSiN
he got 12,000 dollars from lottery, NOT!	hg12KflN!

Sentences are easy to remember, and they make passwords that are nearly impossible to break (and please do NOT use these sample passwords as your own password).

If you notice weird things happening with your account:

Change your password IMMEDIATELY!
Let your system administrator know about it.

It is very common for someone, whose account has been hacked, to dismiss the signs as technical problems with the system.

If your account has been hacked AND if you don't take any measure immediately, not only will they have access to your personal files - delete all your files, modify important data, read your private correspondence, and send mail out in your name -, it very often puts the security of the entire system at risk.

Finding Help

"That was Zen. This is Tao"
- Peter da Silva

Unix has more than 200 different "basic" commands available. Except for the ones restricted to the system administration, anyone can access and use those commands. Good news is that information and help files about those Unix commands are always accessible on-line.

They are known as man pages because they were originally pages of the Unix Programmer's Manual. One sidenote; the familiar abbreviation "RTFM" commonly used in e-mail and Internet nowadays actually came from a Unix jargon, "Read The Fine Manuals" - and figure out yourself instead of keep asking questions.

O.k., let's learn how to "RTFM" from inside of Unix. Let's say I want to find out more about a command called "date". To do that, I simply type following at system prompt

$ man date

As you can guess, the "man" command can be followed by the name of any other command. If you can't remember the exact name, you can do a keyword search. For instance, if I need to find command(s) that can be used to copy files, you use "man" command with keyword 'copy':

$ man -k copy

Here, the "-k" is called a "flag" and is to instruct the "man" command to do a keyword search on the Unix command database reside in the system. As a result, man command then gives you a list of Unix commands that have something to do with the keyword 'copy.' After examining the keyword search results, you then use man command again to learn about a particular command of your choice:

$ man command

where "command" is the command you want to know more about. You'll find, especially at first, that it takes a bit of deciphering to understand what the man page really says. But if you start with simpler commands now, then the more complex entries won't be as hard to read.

"man" uses the "more" command to let you page through the reference pages. You can press spacebar or "f" to forward/view the next page, and "b" to go back to previous page or "q" to quit viewing.

Working with the Unix file system

"What if everything is an illusion and nothing exists?
In that case, I definitely overpaid for my carpet"
- Woody Allen

In Unix, almost everything you do involves manipulating files - creating them, changing them, moving them around, and so on. It is, therefore, very important that you understand how the Unix file system is put together. There are two major concepts to understand: files and directories.

Files

Everything in Unix is treated as a file. Instructions telling the computer how to "do" each Unix command are stored in files. Text (like computer program listings, messages, e-mail, reports, etc.) is stored in files. A file can have just about any name, but should not include any of the following system characters: "'\,*?!$ and blank spaces. If you stick with letters, numbers, and periods you'll be safe.

Upper and lower case letters are distinct in Unix filenames. For example, Unix treats "apple_orang.txt" and "Apple_Orange.txt" as two completely different files.

Directory

A directory is a special kind of file. It contains a list of files and the information belonging to those files. This would include things like who "owns" (created) the file, where it is on disk, how long it is, and who can use it.

Every file in the system is listed in a directory somewhere on the system. Since a directory is simply a list of files, it can contain any file in it, including other directories. This makes the system begin to look like a tree, branching out at each new directory.

Directories are organized in a hierarchy. Your account is assigned to a directory called your "home directory". You own this directory and everything subordinate to it. When you first log in, Unix sets your current working directory to your home directory.

There are a number of methods for specifying which directory and file you are interested in. For example, you can change the current working directory with the "cd" (change directory) command:

You type	It does
$ cd	change to home directory (i.e., /home/[your login ID])
$ cd [directory]	change to directory
$ cd data	change to the 'data' directory underneath your home directory
$ cd ./data	change to the 'data' directory underneath your current directory
$ cd ~/data	change to the 'data' directory underneath your home directory

Note that "~" (tilde) always represents your home directory no matter where you are in the directory tree. Also, keep in mind that in Unix, directory names are separated by a slash "/" not like the backslash "\" used in wintel machines.

In Unix, there are two special directory notations that you should always remember. The first is ".". The "." means "current directory" or "the directory that I'm in right now." At any time, type "pwd" (print working directory) to find what "." means at that moment.

The other special directory is ".." (parent directory). The directory ".." is the directory always one-level above the current working directory no matter where you are.

For example, to list the files in the directory just above your current directory (without changing directory), I could type:

$ ls ..

Listing Directories and Files

To see is believing, indeed. You can display names of files and subdirectories in current directory with "ls" (list) command (e.g., "ls -al"). "ls" is equivalent of the DOS command "dir" if you're familiar with DOS.

Files with a period as the first letter of the file name are hidden files (compare the difference between an "ls" and an "ls -a"). Hidden files (called "dot files") typically contain important system/user setting or configuration data, and the idea of making them 'hidden' from normal ls command is based on "Out of sight, Out of mind" approach - to make them safer from accidental tempering.

Here are some useful flags to the ls command:

Flags	It will
a	show hidden files
l	show long listing
C	show short listing (in columns)
t	list in order of time (creation)
r	list in reverse order
R	list recursively

These flags can be combined as needed. Here are some useful combinations, please examine them and get familiarize yourself with their characteristics.

ls -al	ls -l	ls -C
ls -lt	ls -lrt	ls -lR

There is a system command alias for "ls -Flas | more" called "ll" and you can just type "ll" to get a full information of a directory. I'd recommend you to make a habit to list directory and files as often as possible.

The Unix shell interprets a number of characters in a special way. These characters are known as wildcard characters. Usually these characters are used to describe filenames or directory names. The handling of files is simplified by using wildcard characters to match files that match particular patterns.

Wildcard	It means
*	An asterisk matches any number of characters in a filename, including none.
?	The question mark matches any single character.
[ ]	Brackets enclose a set of characters, any one of which may match a single character at that position.
-	A hyphen used within [ ] denotes a range of characters.
~	A tilde at the beginning of a word expands to the name of your home directory. If you append another user's login name to the character, it refers to that user's home directory.

Here are some examples on how to use wildcard characters:

You type	It will
$ cat c*	Displays any file whose name begins with c including the file c, if it exists.
$ ls *.c	Lists all files that have a .c file extension.
$ cp ../xyz? .	Copies every file in one directory up that is four characters long and begins with xyz to the current working directory. (The names will remain the same.)
$ ls xyz[34567]	Lists every file that begins with xyz and has a 3, 4, 5, 6, or 7 at the end.
$ ls xyz[3-7]	Does exactly the same thing as the previous example.
$ ls ~	Lists your home directory.
$ ls ~/project	Lists "project" directory under the home directory.

By using wildcard characters, you can pre-filter and zero-in the list of files and directories that you're really interested in. Then, common operations can thus be performed on a group of common files using a single command.

FYI, some often seen directory names in Unix file system are:

Directory name	It usually contains
/bin	executables - binary files (compiled programs) and commands
/lib	library (for programming, applications, etc.)
/include	header files (for programming)
/src	source codes (C/C++/Fortran/Sh/Pascal/Perl, etc.)
/tmp	temporary files
/archive	backups and archives

Of course, you can use your own directory naming convention, however for the consistency's sake, I'd recommend you to use the same convention as shown above.

Creating/Renaming/Deleting Files and Directories

You can also make a new directory with the "mkdir" (make directory) command, and you can remove an empty directory with the "rmdir" (remove directory) command. If there is any files under a directory you want to delete, you need to use -R (recursive) flag to delete the directory and files under it.

Let's assume that following examples are done in the home directory of user "wclinton."

You type	It does
$ mkdir timber	create a directory timber in current directory
$ mkdir ~/grisly	create a directory grisly in home directory
$ mkdir /home/wclinton/grisly	the same as above example
$ rmdir starr	delete a directory starr in current directory
$ rmdir -R ~/whitewater	delete a directory whitewater AND all files/subdirectories under it
$ mkdir -R /home/wclinton/whitewater	the same as above example

Time to time, you'd like to know where in the file tree a directory resides before using any of "mkdir" or "rmdir" command. In such case, use "pwd" (print working directory) command to display the absolute pathname of your current working directory.

Directories under Unix can be renamed and moved around with a single command. For example, whole "/data/life_saver" directory (and whatever under it) can be moved to home directory by :

$ mv ~/data/life_saver ~

Files can be moved around in a similar way. To rename a file under Unix, you use the "mv" (move) command. For example, to change the name of file "foo" to "boo" in current directory, you can type :

$ mv foo boo

You can also copy files with the "cp" (copy) command. Here is an explanation by examples:

$ cp batman robin

makes a duplicate of the file "batman" and gives it the name "robin" in the current directory. Note that the filenames can include pathnames as well. Let's say that you're currently in "/home/wclinton/project" directory.

$ cp /home/wclinton/project/batman ./robin

Above "cp" command makes a copy of the file "batman" found in the "/home/wclinton/project" directory and places it in the current working directory (= /home/wclinton) into a file called "robin. "

If you don't specify the filename for "cp" command, the same filename (=source filename) will be used to create a duplicate.

$ cp catwoman /home/wclinton/project

Above "cp" command makes a copy of the file "catwoman" found in current directory to a file "/home/wclinton/project/catwoman. " The filename, "catwoman" will remain unchanged.

Same manner, following "cp" command makes a copy of the file "joker" found in "/home/wclinton/project" directory to the current directory. The filename, "joker" will remain unchanged.

$ cp /home/wclinton/project/joker .

You can also use a wildcard character, an asterisk that matches any number of characters in a filename, to copy all the files (but not the subdirectories) from a directory all at once.

$ cp /home/wclinton/project/* .

will copy all the files (but not the subdirectories if there is any) from "/home/wclinton/project" directory into the current directory at once. Hence, you can copy all the files and subdirectories (if there is any) from "/home/wclinton/project" directory to the current directory by using the "-R" (recursive) flag as follows:

$ cp -R /home/wclinton/project/* .

Same manner, to copy everything from the "/home/wclinton/secret" directory into the "shredder" directory under the home directory (assuming the "shredder" directory already exists):

$ cp -R ~/secret/* ~/shredder

Keep in mind that all wildcard characters can be used for "cp" and "mv" commands.

Another useful flag for "cp" command is the "-i" (interactive) flag which will prompt for confirmation if you are about to overwrite an existing file(s) that happens to be the same filename(s) you're copying from. A "y" answer means that the copy should proceed. Any other answer prevents "cp" from overwriting. It is a nice cautionary measure, however it can be a tiresome routine if used repeatedly.

You can delete a file with the "rm" (remove) command. Be careful if you type "rm *", because Unix doesn't ask you if you're sure you want to do that. Also, there is NO UNDELETE MECHANISM for the Unix file system. When a file is gone, it's gone! BE CAREFUL. ONLY DELETE WHAT YOU WANT TO DELETE.

If you're unsure about whether a wildcard will pick up only what you want, do "ls" with that wildcard first, to see what it matches, then run the "rm. "

You type	It does
$ rm uh-oh.txt	delete a file uh-oh.txt in current directory
$ rm ~/brainstorm/not_a_good.idea	delete a file not_a_good.idea under directory brainstorm
$ rm /home/wclinton/brainstorm/not_a_good.idea	the same as above example

Of course, similarly like in "cp" command, you can use "-i" (Interactive) flag with "rm" command to confirm before removing any files.

The bottomline is that you should know what you're doing when you're using "cp", "mv", "rm" and "rmdir" commands.

Make a good habit to visualize what will be the result before issuing any one of those commands.

Permission Level

Unix is all about levels of privilege, permission and privacy, just like a human society (happily or sadly, in that regards). Let's use an example shown below, which is a hypothetical, partial listing of the home directory of user "wclinton" using "ls -Flas" command.

total 831
 18 drwxr-xr-x 97 wclinton users    8704 Jan  2 20:13 ./
 18 drwxr-xr-x 97 wclinton users    8704 Jan  2 20:13 ./
  1 dr-xr-xr-x  2 root     root        6 Jan  2 20:16 ../
768 -rwx------  1 wclinton users  384000 Nov 18 17:56 catwoman
 34 -rw-rw----  1 wclinton A-team  16963 Dec 11 12:53 plan-b.txt
  2 lrwxrwxrwx  1 wclinton users      16 May  7 1998  joker -> /project/joker/
  4 drwx------  3 wclinton users    1536 Nov  1 1997  project/
  2 drwxrwsrwx  2 wclinton A-team   1024 Nov  3 10:37 public_poll/
  2 drwx------  8 wclinton users     512 Jul 20 16:55 secret/

Just at a glance, you can see there's a lot of information. These are typical Unix file/directory characteristics, and you do need to be able to interpret such information precisely and succinctly. Here's how to.

The first column gives the size in blocks, and next 10 characters of each line show the file/directory permission (or mode). The first letter is "d" for directories or "l" for symbolic link (=alias) or "-" for regular files. The next 9 characters are grouped into three rwx-triples that deal with different permission levels.

The first triple indicates the file/directory owner's permissions.
The second triple indicates the group's permissions.
And the third triple gives permissions for users who are not the owner (i.e., public) and who are not in the same group.

It is a representation of hierarchical permission structure starting from owner; owner belongs to a group, and the group belongs to the public.

Also, among rwx-triple, "r" stands for "read" permission, "w" for "write" permission, and "x" for "execute" permission.

"Write" means to modify as well as to delete a file/directory. "Execute" means to execute the file as a program. For a directory, "execute" permission is interpreted to mean permission to search the directory for a specified file.

For example, "-rw-r--r--" means that anyone can read the file, but only the owner can change (=read or delete) it.

Another example is "-r--------" which means that only the owner can read the file and no one can change it including the owner. Note that this keeps "rm" or "mv" from removing the file, so it can protect you from yourself! (of course, it is a bit extreme example though)

As another example, "-r-xr-x---" allows the owner and group to read and execute the file, but other users not belong to the owner's group cannot access the file -- and nobody can write to it. (of course, with one exception - system administrator or "root")

In summary,

Permission	It means
r	the file/directory is readable
w	the file/directory is writable (or deletable)
x	the file is executable/the directory is accessible
-	the indicated the permission is not granted

Now, once again, using the same listing,

total 831
 18 drwxr-xr-x 97 wclinton users    8704 Jan  2 20:13 ./
 18 drwxr-xr-x 97 wclinton users    8704 Jan  2 20:13 ./
  1 dr-xr-xr-x  2 root     root        6 Jan  2 20:16 ../
768 -rwx------  1 wclinton users  384000 Nov 18 17:56 catwoman
 34 -rw-rw----  1 wclinton A-team  16963 Dec 11 12:53 plan-b.txt
  2 lrwxrwxrwx  1 wclinton users      16 May  7 1998  joker -> /project/joker/
  4 drwx------  3 wclinton users    1536 Nov  1 1997  project/
  2 drwxrwsrwx  2 wclinton A-team   1024 Nov  3 10:37 public_poll/
  2 drwx------  8 wclinton users     512 Jul 20 16:55 secret/

Let's take a look at the third line in above example. At fourth line, reading from right to left, a file named "catwoman," and the last time that file's contents were modified was 5:56 P.M. on November 18 of current year. The file contains 384,000 characters, or bytes. The owner of the file belongs to the group "users" and his or her login name is "wclinton." The next "1" tells the number of links for this file.

Finally, the dashes and letters tell you that no users or groups have permissions to read, write, and execute the file "catwoman" other than its owner, "wclinton."

Now, can you read what the remaining lines of above listing mean?

As long as a file/directory belongs to your ownership, you can change its permission level any time. In order to change the permissions, you can use the command "chmod" (=CHange permission MODe).

Also, you can change ownership (chown) as group information (chgrp) of a file or a directory. For their usage and flags, please RTFM.

Displaying Text from File(s)

"To my daughter Leonora:
Without whose never failing sympathy and encouragement,
this book would have been completed in half the time"
- P.G. Wodehouse

Of course, to be of use, files must be accessed and looked into. You can always open a text editor and read the content of a text file, but most times, you just want to take a quick look the file. To read/print the contents of a file on your screen, you can use either the "cat" (=catenate) or the "more" command or the "less" command or the "page" command. For example:

$ cat foo.txt    (or)
$ more foo.txt    (or)
$ less foo.txt    (or)
$ page foo.txt

will display the contents of a file "foo.txt" on your screen.

Catenate means "to connect in a series." The "cat" command displays the contents of a file. If more than one file is placed in the command line, i.e., "cat jimmy shook," -- FYI, "jimmy" and "shook" refer to male and female Chesapeake blue crabs -- the files are displayed in succession. (of course, file "jimmy" first, then file "shook") It is here that "cat" command derives its name.

$ cat jimmy shook

Either command works, but "more" or "less" or "page" is the intelligent version.

With "more" or "less" or "page," the system stops at the end of a screenful and wait for you to hit a key to continue. If you use "cat, " everything flies by without stopping. (of course you can use to pause the screen and to resume, but this can be annoying)

Using "more" or "less" or "page, " there are several keys that make more continue in different ways. key makes more only print out one additional line and then pause again. gives you a full screenful.

When using "more/less/page" and "cat" you must be careful. Any file that is not a text file should not be viewed through "more/less/page" or "cat." (such as binary file)

If you're not so sure what kind of file you're dealing with, you can use "file" command first to determine file type before displaying the file.

Unix Process

In Unix, every event is called a "process." Understanding the concept of "process" and maximizing process(es) in many different combinations is how you make the most out of Unix.

Unix Process Control

Unix is a multi-tasking/multi-threading/multi-user environment. (yup, the real thing, finally)
There are always a number of different programs from different users running at the same time. Each Unix program is called a "process."

Each process has a "process ID" or "pid" assigned to it. You can look at processes that are currently executing by using the "ps" command.

For examples:

You type	It does
$ ps	list processes belong to you only
$ ps -Af	list all processes, in a long format (bash shell)
$ ps -aux	list all processes, in a long format (tcsh shell)

"ps" command will give you rich information about what, whose, when, how long processes are currently running.

Normally, a program/command will terminate correctly when it finishes doing what has been instructed. However, time to time a processes may get strayed and cause some problems such as hung-up or terminal lock-up. In such cases, you can selectively send a signal to that particular process with the "kill" command saying "please terminate yourself" with an exclamation mark. (no reboot or reset button necessary)

You can do this by typing "kill -9 pid" where pid is the process ID number of the process you want to terminate (you need to find this pid using "ps" command first).

If the process is still around after a "kill -9", then it is either hung up in the Unix kernel, becoming a "zombie" process, or you are not even the owner of the process! ("kill" command will tell you if you try to kill something you don't own) BTW, "zombie" process(es) will automatically be cleaned up by Unix kernal.

To learn more about "ps," type "man ps."

Redirection

A program/command that normally reads its input from the terminal/keyboard (standard input) or normally writes its output to the terminal (standard output). However it may become annoying if you would rather like to send the input from a file instead of the keyboard or send/capture the output to a file instead of the terminal.

This annoyance can be avoided if you use the redirection operators. This redirection is one of the core concepts of Unix. The redirection operators are "<", ">", and ">>".

The "<" (=input) operator is used to send input to a program/command. Here is an explanation by example:

$ mailx -s "It's me, again" complaint@lottery.va.gov
< where_did_my_money_go.txt

(all entered in the same line)

Instead of using e-mail programs such as "pine" or "dtmail," etc., the user can send an existing text file "where_did_my_money_go.txt" in current directory to "complaint@lottery.va.gov" with a subject field of "It's me, again." (with a file containing a list of e-mail address, this is usually how SPAM e-mails are sent out)

The ">" (=output) operator is used to create a new file and send the output from a program/command to it. Let's use the previous example from cat command.

$ cat jimmy shook > crabcake

The contents of files, "jimmy" first, then "shook," by cat command will be redirected to a new file "crabcake" instead of displayed on the terminal.

In other word, it is the same as combining files "jimmy" and "shook" and saving the result into a new file "crabcake." Now, a caution - if there is a file called "crabcake" already exists, it will be overwritten with contents of "jimmy" and "shook."

If you do not want to overwrite but "append/add" at the end of an exiting file, you can use ">>" (=append) redirection operator.

$ cat fake_crabmeat >> crabcake

The contents of file, "fake_crabmeat" by cat command will be redirected AND appended at the end of existing file "crabcake" which already contains the contents of "jimmy" and "shook."

Also, ">" (=output) operator preceded by a 1 (optional) denotes for standard output and 2 for standard error (=error message from a program/command if there's any). For example,

$ cat fake_crabmeat >> crabcake 2>
trouble_in_crabcake

(all entered in the same line)

will append output of cat command to "crabcake" and error messages (if any) to "trouble_in_crabcake."

You can also combine both kinds of redirection as in

$ program < datafile (or input) > outputfile

The data in the file "datafile" will then be used as input for "program" (or command) and all output will be stored in "outputfile."

Pipes

A pipe is a convenient way to channel two or more commands/programs without creating an intermediate file. The output of one command/program can be fed directly into another command/program by connecting the two commands/programs via a pipe.

Thus it allows a user to create powerful new commands by chaining existing commands together. This is another important core concept of Unix.

Let's say you wanted to get an alphabetical listing of the current processes. Using "redirection" technique we went through above (and the man pages for "ps" and "sort"), you already know how to do this:

$ ps -Af > processes (if you're using bash shell)
$ ps -aux > processes (if you're using tcsh shell)

$ sort processes

This works, but it creates a file (namely "processes") which you don't want. The pipe symbol, "|" lets you bypass this intermediate file. The above two commands can be replace with the following :

$ ps -Af | sort (if you're using bash shell)
$ ps -aux | sort (if you're using tcsh shell)

The result (=output) of "ps -Af" command will instantaneously become an input to "sort" command. As a result, above piping will produce an alphabetically sorted list of all your processes.

It is possible to connect a series of commands by additional pipe symbols. You can pass sorted processes output through the more command to obtain one-screenful of display of the results. This is accomplished by piping:

$ ps -Af | sort | more (if you're using bash shell)
$ ps -aux | sort | more (if you're using tcsh shell)

One important point to recognize is that if a command isn't capable of reading from standard input, it cannot be placed to the right of a pipe symbol since each command in a pipeline is executed as a separate process.

Pipe List

A list is a sequence of one or more pipelines separated by one of the operators ";", "&", "&&", or "||", and terminated by one of ";", "&", or "". Of these list operators, "&&" and "||" have equal precedence, followed by ";" and "&", which have equal precedence.

If a command is terminated by the control operator "&", the shell executes the command in the background in a subshell. The shell does not wait for the command to finish, and return to the normal waiting mode and as a result, the user can continue working with other programs or using other commands.

In other word, if you're running a big program or long task (such as printing), simply by putting "&" at the end of the command, you can immediately start next command instead of waiting. This is what "multi-tasking" really means.

$ netscape &

Commands separated by a ";" are executed sequentially; the shell waits for each command to terminate in turn. The return status is the exit status of the last command executed.

$ ps -AF | sort > ps_dump ; date > ps_mark ;
cat ps_mark ps_dump > ps01

(all entered in the same line)

The "&&" and "||" are conditional control operators. "&&" denotes AND list and "||" denotes OR list, respectively. An AND list has the form

$ command1 && command2

command2 is executed if, and only if, command1 executed correctly, i.e., a serial execution. If command1 failed, command2 will not be executed.

An OR list has the form

$ command1 || command2

command2 is executed if and only if command1 did not executed correctly, i.e., a parallel execution or a contingency. If and only if command1 failed, command2 will be executed.

Dealing with Disk Usage

In Unix, when you logged in, all files belonging to you (such as your home directory) will be "mounted" from a network fileserver. It is a "virtual" file system called NFS (Network File System).

Thus, as a user, the issue of which workstation you log in becomes not an important one, but the issue of which "network" becomes. This is quite a contrast to typical PCs that still heavily depend on its internal hard disk(s) for handling files and directories.

Since Unix is a multi-user environment, the disk space in the network fileserver is also shared by many users, and subsequently each user's disk usage directly affect the remaining disk space available to other users.

Disk Usage

The "du" (disk usage) command displays the number of kilobytes consumed by each file and directories recursively. By default, file sizes are written in 512-byte units, rounded up to the next 512-byte unit.

$ du -s *
(display disk usage of your files and directory at current directory)

$ du -s ~/*
(display disk usage in your home directory and below)

This can be also useful for finding out who the big "disk hogs" are when you're suddenly running low in disk space without even using any. Typing "du -s *" from the parent directory of your home directory, e.g., "/home", or directly typing "du -s /home/*" gives a grand total of the kilobytes consumed by each user under "/home" directory.

Free Disk Space

The "df" (disk free) command displays the amount of disk space occupied by mounted or unmounted file systems, directories, or the amount of used and available space, and how much of the file system's total capacity has been used in the fileserver.

If directory name (e.g., /home/wclinton) is specified, "df" reports on the file system that contains directory. If not, "df" reports on all mounted file systems.

$ df -k .
(display free disk space and usage of all mounted file systems in kilobytes)

This is often quite handy when determining whether there is enough space to store a big file on a particular hard drive in the fileserver. For further flags and details, please see "man df."

File Archive and Compression

When a big file(s) or directory(ies) is not used frequently but you still need to keep it in your account for future use, you may consider compressing it or archiving it or both to conserve disk space.

There are several common tools for archiving and compressing; "tar", "compress/uncompress", "gunzip(=gzip) " and "zip/unzip".

The "tar" (tape archiver) command is useful for storing or "archiving" a bunch of files and directories. It archives and extracts files to and from a single file called a "tarfile. " The "tape" in tar command originated from the "old days" when backup and archive were solely done using magnetic tape devices, but it doesn't have to be nowadays and it can be any file. (well, still tape backup is the most common means to do regular system backups).

The syntax for "tar" command is

$ tar [key] [name ...]

where key is specified by a plethora of options (see abridged list below and unabridged list in the man pages) and name is either the file name or device name (such as magnetic tapes, CD-R, WORM, MO disk, etc.). Here are some of the more commonly used keys:

tar key	It will
c	Creates a new tape
t	Lists the contents of a tarfile without untarring a tarfile
f	File. Use the [name] argument as the name of the tarfile instead of the default tape device.
v	Turns verification on
x	Extracts selected files. If no file argument is given the entire contents of the tar file is extracted

Here is some examples:

$ tar cvf oink.tar /home/wclinton/project

Create a tarfile that archives all files and subdirectories in "/home/wclinton"

$ tar tvf /dev/rmt0/jumbo.tar

List contents of a tarfile in a magnetic tape ("/dev/rmt0" is a standard name for a mounted tape drive)

$ tar xvfp ~/archive/noway.tar ~/project

Extract all files from "noway.tar" tarfile in "/home/wclinton/archive" directory to "/home/wclinton/project" directory. If "noway.tar" tarfile contains subdirectoies, they'll be also extracted under "/home/wclinton/project" directory.

It is a common practice to compress the resulting tarfile when archiving (=tarring) is completed. Compression is done by using either "gunzip (=gzip) " or "compress" command.

$ gzip -9 oink.tar

Compress "oink.tar" using gzip with a maximum compression. Resulting file will be "oink.tar.gz"

$ compress oink.tar

Compress "oink.tar" using compress command. Resulting file will be "oink.tar.Z"

"gzip" is better in compression speed and efficiency than "compress." However, "compress" is a standard command comes with the Unix OS, whereas "gzip" is a GNU program you need to install.

Thus, if you're preparing a "gzipped tarfile" file(s) for others, make sure the person who will receive the "gzipped & tar'd " file" (or often called as "tarball") has "gzip" installed in one's system. (if not sure, use "compress" instead)

Now, here's how to uncompress and untar (=extract) a compressed and tarred archive. Here, I'm using the pipeline technique that we discussed previously.

$ gzip -cd oink.tar.gz | tar xpvf -

Decompress and extract all files/directories from "oink.tar.gz" to current directory

$ uncompress oink.tar.Z | tar xpvf -

Decompress and extract all files/directories from "oink.tar.Z" to current directory

Further details on "compress" can be found from man pages. For "gzip," type "gzip -h" for additional flags information.

zip/unzip compression method popular in wintel PCs is also available via "zip/unzip" command. Their usage and flags information can be also found by typing "zip -h" and "unzip -h."

A Summary: Basic Unix Commands for a Tenderfoot

Let's take a look again at basic Unix commands that are absolutely essential for your successful journey into Unix. Keep in mind that all Unix commands are all case-sensitive. For further command syntax and description, please RTFM.

	Unix	DOS
Change directory	cd	cd
List directories and files	ls	dir
Create a directory	mkdir	md/mkdir
Delete a directory	rmdir	rd/rmdir
Copy a file/directory	cp	copy/xcopy
Move a file/directory	mv	move
Rename a file/directory	mv	ren/rename
Delete a file	rm	del/erase
Change file permission	chmod	attrib
Display contents of file	cat filename	type filename
Display file with pauses	more/less/page filename	type filename \| more
Find string in file	grep/egrep/fgrep	find
Get help	man	help
Display free disk space	df	chkdsk
Display disk usage	du	chkdsk
Display date and time	date/cal/gcal	date, time
Print file	lpr	print
Display print queue	lpq/lpstat	print
Print working directory	pwd	cd
Text editor	vi/emacs/ed/etc.	varies

On-line Tutorials and Good Housekeeping

Try various on-line introductory materials from your new Unix environment. Click on "Help Viewer" icon displayed in the "CDE Desktop Panel" at the bottom of the screen. (it is the second icon from the right side, or the one next to "Trash bin" icons)
I'd highly recommend you to go through on-line introductory materials: "Common Desktop Environment" and "Overview and Basic Desktop Skills" to become familiar with your new Unix environment.
To logout (or Quit), press and hold down right mouse button. Once again, "Workspace" menu will appear. While holding down right mouse button, select "Logout, " and release mouse button. Click on "OK" confirmation button to logout.
If you click "Logout" and walk away without confirming logout, your account will still be accessible to any other person in the lab. It is like leaving your car unlocked with key inside. Never ever let this happen. Another way to logout is to click on "Exit" icon in the Desktop Panel at the bottom of the screen.