Standard streams stdin stdout stderr

In Unix and Unix-like operating systems, as well as certain programming language interfaces, the standard streams are preconnected input and output channels between a computer program and its environment (typically a text terminal) when it begins execution. The three I/O connections are called standard input, standard output and standard error.

Background

In most operating systems predating Unix, programs had to explicitly connect to the appropriate input and output data. On many of those systems, this could be an intimidating programming challenge created by OS-specific intricacies such as obtaining control environment settings, accessing a local file table, determining the intended data set, and handling the correct case of a card reader, magnetic tape drive, disk drive, line printer, card punch, or interactive terminal.

Unix provided several groundbreaking advances, one of which was to provide abstract devices: it removed the need for a program to know or care what kind of devices it was communicating with. Older operating systems forced upon the programmer a record structure and, frequently non-orthogonal data semantics and device control. Unix eliminated this complexity with the concept of a data stream: an ordered sequence of data bytes which can be read until the end of file. A program may also write bytes as desired and need not (and can't easily) declare how many there will be, or how they will be grouped.

Another Unix breakthrough was to automatically associate input and output by default—the program (and programmer) did absolutely nothing to establish input and output for a typical input-process-output program (unless it chose a different paradigm). In contrast, previous operating systems usually required some—often complex—job control language to establish connections, or the equivalent burden had to be orchestrated by the program.

Since Unix provided standard streams, the Unix C runtime environment was obligated to support it as well. As a result, most C runtime environments (and C's descendants), regardless of the operating system, provide equivalent functionality.

Standard input (stdin)

Standard input is data (often text) going into a program. The program requests data transfers by use of the read operation. Not all programs require input. For example, the dir or ls program (which displays file names contained in a directory) performs its operation without any stream data input.

Unless redirected, input is expected from the text terminal which started the program.

The file descriptor for standard input is 0 (zero); the corresponding variable is FILE* stdin; similarly, the variable is std::cin.

Standard output (stdout)

Standard output is the stream where a program writes its output data. The program requests data transfer with the write operation. Not all programs generate output. For example the file rename command (variously called mv, move, ren) is silent on success.

Unless redirected, standard output is the text terminal which initiated the program.

The file descriptor for standard output is 1 (one); the corresponding variable is FILE* stdout; similarly, the variable is std::cout.

Standard error (stderr)

Standard error is another output stream typically used by programs to output error messages or diagnostics. It is a stream independent of standard output and can be redirected separately. The usual destination is the text terminal which started the program to provide the best chance of being seen even if standard output is redirected (so not readily observed). For example, output of a program in a pipeline is redirected to input of the next program, but errors from each program still go directly to the text terminal.

It is acceptable—and normal—for standard output and standard error to be directed to the same destination, such as the text terminal. Messages appear in the same order as the program writes them, unless buffering is involved. (For example, a common situation is when the standard error stream is unbuffered but the standard output stream is line-buffered; in this case, text written to standard error later may appear on the terminal earlier, if the standard output stream's buffer is not yet full.)

The file descriptor for standard error is 2; the corresponding variable is FILE* stderr. The C++ standard header provides two variables associated with this stream: std::cerr and std::clog, the former being unbuffered and the latter using the same buffering mechanism as all other C++ streams.

Most shells allow both standard output and standard error to be redirected to the same file using

 >& filename

Bourne-style shells allow standard error to be redirected to the same destination that standard output is directed to using

 2>&1

SSH without password:

a@A:~> ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/a/.ssh/id_rsa):
Created directory '/home/a/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/a/.ssh/id_rsa.
Your public key has been saved in /home/a/.ssh/id_rsa.pub.
The key fingerprint is:
3e:4f:05:79:3a:9f:96:7c:3b:ad:e9:58:37:bc:37:e4 a@A

Now use ssh to create a directory ~/.ssh as user b on B. (The directory may already exist, which is fine):

a@A:~> ssh b@B mkdir -p .ssh
b@B's password:

Finally append a's new public key to b@B:.ssh/authorized_keys and enter b's password one last time:

a@A:~> cat .ssh/id_rsa.pub | ssh b@B 'cat >> .ssh/authorized_keys'
b@B's password:

From now on you can log into B as b from A as a without password:

a@A:~> ssh b@B hostname
B

Linux Authentication Using OpenLDAP, Part One

Linux Authentication Using OpenLDAP, Part One

Introduction This is the first of two articles that will discuss a number of issues with LDAP authentication on Linux. In this installment, I will discuss an overview of LDAP, installing and configuring OpenLDAP, migrating to OpenLDAP and setting up LDAP queries. In this series, I will focus on Red Hat Linux version 7.1 (with some comments about earlier revisions;) however many of the same principles apply to Debian and other Linux distributions.

Authentication, PAM, and NSS Authentication is the process wherein a user logging on to a Linux system has their credentials checked before being allowed access. Usually, this means that a user needs to provide a login name and a password. Many different programs provide authentication, each using a different method. For example, the basic Unix login program provides a simple text interface for a user to enter a user ID and password. Graphical login systems such as XDM (or GDM or KDM) provide a different interface. Programs such as SSH can authenticate users based on things like RSA or DSA keys as well as passwords. There are many different authentication suites or protocols available on Linux today. Like all traditional Unix systems, Linux is capable of authenticating users against entries in the /etc/passwd and /etc/shadow files, but it also supports such authentication schemes as Kerberos, RADIUS, and LDAP, which stands for Lightweight Directory Access Protocol.

PAM (which stands for Pluggable Authentication Modules) is a set of libraries provided with most modern Linux distributions, and it is installed by default in Red Hat Linux. The PAM libraries provide a consistent interface to an authentication protocol. An application can use the PAM libraries to allow the use of any authentication protocol within that application, so that if the system administrator wants to change from, for example, /etc/passwd authentication to LDAP, the application does not have to be re-written or recompiled. PAM requires a PAM module for each authentication system. There are many such PAM modules on the above web site.

Unfortunately, PAM only provides part of the information needed to keep track of users on a Linux system. In addition to being able to check that a user has entered the correct password, a Linux system needs other information, such as the user's numeric user ID, their home directory, default shell, etc. This information, which would normally be stored in the /etc/passwd file, can be determined through a system interface known as NSS, or Name Service Switch.

Only some authentication schemes provide enough information to be useful to NSS. For example, Kerberos only stores user authentication information, not details such as a home directory or default shell. Therefore, there is no NSS module for Kerberos.

What is LDAP?

LDAP, or Lightweight Directory Access Protocol, is a network protocol that is used for accessing information in an object-oriented database. LDAP includes features that make it useful to both PAM and NSS, as it can authenticate users, as well as providing user information such as home directory names and default shells to NSS.

An LDAP Server or Directory Server (sometimes called a DS for short) is a server that can send and receive information in the LDAP protocol. Typically, an LDAP server will be a piece of software that listens on the standard LDAP ports (389 and sometimes 636) for connections, and responds to LDAP queries and requests. To draw an analogy with databases, LDAP is the equivalent of SQL, and an LDAP server is like a database server such as Oracle or MySQL.

LDAP servers are particularly useful for storing information about people. This is because of the object-oriented nature of LDAP. Unlike a relational database, an object in an LDAP directory can contain an arbitrary number of attributes, and each attribute can have an arbitrary number of values. This is useful for many reasons. For example, a database row containing a column for a phone number would allow a single entry in that phone number column for each row in the database table. A person, however, may have more than one phone number, and so LDAP allows multiple phone numbers to be stored in the same person object. Note the slightly different terminology here: we say an LDAP 'directory' as opposed to a 'database', we call entries in the directory 'objects' instead of 'rows', and we call field values of an object 'attributes' instead of 'columns'.

Why Use LDAP?

There are a number of reasons why we might use LDAP:

• LDAP allows us to centralize the information about users, passwords, home directories, etc, in a single place on a network. If we were using /etc/passwd files, for example, we would have to make sure that all passwd files were kept in sync across the network, which would be an absolute nightmare on a large network with users changing passwords regularly.
• LDAP offers encrypted transactions. Most LDAP servers offer encrypted connections using SSL (either using Start TLS on port 389 or LDAPS on port 636), which is more secure than some mechanisms by which plain text passwords are sent over the network. An LDAP directory is also useful for other purposes. For example, it can quickly and easily be used as a company's staff e-mail and contacts directory.
• It is possible to use LDAP in a tree structured manner, unlike the /etc/passwd or NIS tables which basically store users in a flat structure. With a large number of users it makes sense to divide them into organizational units so that they can be found and managed more easily. In the long term, this makes managing an LDAP directory less onerous than managing /etc/passwd files or an NIS/NIS+ database.

OpenLDAP

OpenLDAP is an open source implementation of an LDAP directory server. OpenLDAP is installed by default with Red Hat 7.1 or later, and is available on Red Hat versions from 6.2 onwards. Note that earlier releases of Red Hat used release 1 of the OpenLDAP product. Although this is still considered a stable release by the OpenLDAP team, for a number of security reasons, I would advise against using it. For example, it does not support SSL or schema checking. Your OpenLDAP version should be 2.0.7-3 or later.

Installing OpenLDAP

As usual, you can install OpenLDAP from the source code by obtaining the source files from the OpenLDAP web site and following the compilation instructions. My preference, however, is to install the OpenLDAP packages from the RPM files as follows. Note that you will need to install both the server and client packages if you want to set up an OpenLDAP server. First, put your Red Hat CD-ROM into your CD-ROM drive and use the following sequence of commands:

1. mount /dev/cdrom /mnt/cdrom
2. cd /mnt/cdrom/RedHat/RPMS
3. rpm -Uhv openldap-2.0.7-14.i386.rpm openldap-servers-2.0.7-14.i386.rpm
openldap-clients-2.0.7-14.i386.rpm
4. umount /mnt/cdrom

It's possible that the packages are already installed on your system (to verify this you can do rpm -q openldap.) It is also possible that you may hit one or more dependencies in installing the above RPMs. In particular, the OpenLDAP packages require the OpenSSL package, and at least the krb5-libs package.

LDAP,PAM and NSS libraries

Using LDAP will almost certainly require you to install the PAM libraries for LDAP. In Red Hat 6.2 and later, these are packaged in with the nss_ldap package (since the pam_ldap libraries are not much use without the nss_ldap libraries and vice-versa). These are normally installed by default - to test this you can run rpm -q nss_ldap. If the nss_ldap package is not installed, you can install it using RPM as follows:

 
 mount /dev/cdrom /mnt/cdrom
 cd /mnt/cdrom/RedHat/RPMS
 rpm -Uhv nss_ldap*.rpm
 umount /mnt/cdrom

If you need to obtain the source code for the pam_ldap and nss_ldap libraries, they are available from PADL.com at the following locations:

• http://www.padl.com/pam_ldap.html for pam_ldap
• http://www.padl.com/nss_ldap.html for nss_ldap

Configuring OpenLDAP

Configuration of OpenLDAP is done through the /etc/openldap/slapd.conf file. There is a manual page describing the contents of the slapd.conf file (see man slapd.conf) as well as an excellent administration guide on the OpenLDAP web site. As a starting point, you might like to use the following simple configuration file:

#
# See slapd.conf(5) for details on configuration options.
# This file should NOT be world readable.
#
include         /etc/openldap/schema/core.schema
include         /etc/openldap/schema/cosine.schema
include         /etc/openldap/schema/inetorgperson.schema
include         /etc/openldap/schema/nis.schema
include         /etc/openldap/schema/rfc822-MailMember.schema
include         /etc/openldap/schema/autofs.schema
include         /etc/openldap/schema/kerberosobject.schema

#######################################################################
# ldbm database definitions
#######################################################################

database        ldbm
suffix          "o=MyCompany,c=AU"
rootdn          "uid=root,ou=People,o=MyCompany,c=AU"
rootpw          secret
directory       /var/lib/ldap
# Indices to maintain
index   objectClass,uid,uidNumber,gidNumber     eq
index   cn,mail,surname,givenname               eq,subinitial

#
# ACLs
#

access to dn=".*,ou=People,o=MyCompany,c=AU"
  attr=userPassword
by self write
by dn="uid=root,ou=People,o=MyCompany,c=AU" write
by * auth

access to dn=".*,o=MyCompany,c=AU"
by self write
by dn="uid=root,ou=People,o=MyCompany,c=AU" write
by * read

access to dn=".*,o=MyCompany,c=AU"
by * read

defaultaccess read

One thing that should be noted in the configuration file above: users should replace "o=MyCompany,c=AU" throughout the file with a Base DN which represents their organization. Note that I prefer to use the X.500 style specification above, but you could use the DNS specification which is "dc=mycompany,dc=com,dc=au" or similar. For example, if your company was called "farnarkle.com" you could use "dc=farnarke,dc=com", or you could use "o=farnarke,c=US". Remember this Base DN, it will be important later.

I have elected to include some elementary Access Control in the file. The standard slapd.conf file included with Red Hat Linux does not include ACLs, but they are mandatory for real use. You may want to expand on the above ACLs (see the slapd.conf manual or the administrator's guide.)

I have included a default root password - 'secret'. This is a bad idea once you have data in your LDAP directory. We will deal with this later.

I have not included TLS certificates, keys, or other information. I would consider this to be a security issue on a network, because without these the server will operate entirely in plain text mode. This will be covered later.

Once you have a working slapd.conf file, you should be able to start your server. This is easy enough to do, you can just run the following command:

/etc/rc.d/init.d/ldap start

Provided that the slapd.conf file is correct, you should be able to use pstree to see a running slapd process. If the slapd.conf file is incorrect, look for error messages (running slapd -d here might help), fix up any problems you see, and try again.

Migrating to OpenLDAP

Once you have your LDAP server started, you will have an empty directory. The first thing you need to do is to populate it with data from your existing authentication database.

Using the supplied LDAP tools

OpenLDAP provides a suite of tools to migrate data from your existing NIS or /etc/passwd database into LDAP. If you currently run another authentication scheme such as Kerberos or S/Key, and you are migrating to LDAP, then I'm afraid you are on your own.

In Red Hat Linux 7.1, the migration tools are in /usr/share/openldap/migration/. In Red Hat 6.2 and earlier they were in /usr/lib/openldap/migration/. In either case, open a shell window, change to that directory, and get to work. First, edit the migrate_common.ph file. Around line 72 you will see a couple of lines like this:

  $DEFAULT_MAIL_DOMAIN = "babel.com.au";
 $DEFAULT_BASE = "o=Babel,c=AU";

You will need to edit these two lines, providing your default mail domain and the Base DN that you defined earlier in the slapd.conf file. Next, it's a simple matter of running the migration tools. This can be done using a simple command, assuming that you are migrating from /etc/passwd files to LDAP:

migrate_all_online.sh

(make sure that your LDAP server is running before using the above command). This will ask you for the root DN and password (enter the password secret that we defined in the slapd.conf file), and will start to populate your LDAP directory.

Setting up LDAP queries

Having data in your LDAP directory is all very well and good, but at some stage you are going to want to query that data. There are a standard set of command line-based LDAP query and management tools provided with OpenLDAP. These include ldapadd, ldapmodify, and ldapsearch. Each of these tools has a man page, and you would do well to read these man pages in detail.

The standard configuration file for these tools is /etc/openldap/ldap.conf. The file format of this file is fairly simple; on a single system it need only contain the following two lines:

  BASE o=MyCompany,c=AU
 URI ldap://127.0.0.1

Remember to substitute your Base DN that you defined in the slapd.conf file instead of the o=MyCompany, c=AU entry shown above.

On a network, you may have to substitute the IP address of your LDAP server instead of 127.0.0.1 shown above. For those of you who understand LDAP concepts a little better: OpenLDAP doesn't (yet) support SLP or DNS RR based location, so you have to be fairly precise about the location of the server -- either an IP address, a host name from /etc/hosts, or something that can be found in DNS.

Once you have done that, you should be able to perform a simple search. You could start by looking for your root user by using the following simple command:

  ldapsearch -x 'uid=root'

You should see an entry fairly similar to this one:

version: 2

#
# filter: uid=root
# requesting: ALL
#

# root,People,MyCompany,AU
dn: uid=root,ou=People,o=MyCompany,c=AU
uid: root
cn: root
sn: root
mail: root@mycompany.com.au
objectClass: person
objectClass: organizationalPerson
objectClass: inetOrgPerson
objectClass: account
objectClass: posixAccount
objectClass: top
objectClass: kerberosSecurityObject
objectClass: shadowAccount
shadowMax: 99999
shadowWarning: 7
krbName: root@MYCOMPANY.COM.AU
loginShell: /bin/bash
uidNumber: 0
gidNumber: 0
homeDirectory: /root
gecos: root

# search result
search: 2
result: 0 Success

# numResponses: 2
# numEntries: 1

Now that you have come this far, stop and smile. You have managed to get LDAP working, which is sometimes not an easy task!

This brings us to the end of the first installment of this two-part series. In the next article, we will continue the discussion of OpenLDAP and Linux, covering subjects such as: Setting up PAM and NSS for LDAP, LDAP Tools, making OpenLDAP more secure and generating SSL keys for OpenLDAP.

Crash Course in Linux File Commands

Although GUI desktops such as KDE and GNOME help users take advantage of Linux features without functional knowledge of the command-line interface, more power and flexibility are often required. Moreover, a basic familiarity with these commands is still essential to properly automate certain functions in a shell script.

This article is a "crash course" in Linux file commands for those who are either new to the operating system or simply in need of a refresher. It includes a brief overview of the more useful commands as well as guidance regarding their most powerful applications. Combined with a little experimentation, the information included here should lead to an easy mastery of these essential commands. (Note: When a kernel tweaked with Oracle Cluster File System (OCFS2) is involved, some of these commands may behave somewhat differently. In that case, Oracle provides an OCFS2 toolset that can be a better alternative for file command purposes.)

Note that all the included examples were tested on SUSE Linux 8.0 Professional. While there is no reason to believe they will not work on other systems, you should check your documentation for possible variations if problems arise.

Background Concepts

Before delving into specifics, let's review some basics.

Files and Commands

Everything is treated as a file in the Linux/UNIX operating system: hardware devices, including the keyboard and the terminal, directories, the commands themselves and, of course, files. This curious convention is, in fact, the basis for the power and flexibility of Linux/UNIX.

Most commands, with few variations, take the form:

command [option] [source file(s)] [target file]

Getting Help

Among the most useful commands, especially for those learning Linux, are those that provide help. Two important sources of information in Linux are the on-line reference manuals, or man pages, and the whatis facility. You can access an unfamiliar command's man page description with the whatis command.

$ whatis echo

To learn more about that command use:

$ man  echo

If you do not know the command needed for a specific task, you can generate possibilities using man -k, also known as apropos, and a topic. For example:

$ man -k files

One useful but often-overlooked command provides information about using man itself:

$ man man

You can scroll through any man page using the SPACEBAR; the UP ARROW will scroll backward through the file. To quit, enter q,!, or CTRL-Z.

Classes of Users

Remember the adage "All animals are equal but some animals are more equal than others?" In the Linux world, the root user rules.

The root user can log in from another username as su, derived from "superuser." To perform tasks such as adding a new user, printer, or file system, log in as root or change to superuser with the su command and root password. System files, including those that control the initialization process, belong to root. While they may be available for regular users to read, for the sake of your system security, the right to edit should be reserved for root.

The BASH shell

Other shells are available, but BASH, the Bourne Again Shell, is the Linux default. It incorporates features of its namesake, the Bourne shell, and those of the Korn, C and TCSH shells.

The BASH built-in command history remembers the last 500 commands entered by default. They can be viewed by entering history at the command prompt. A specific command is retrieved by pressing the UP ARROW or DOWN ARROW at the command prompt, or by entering its number in the history list preceded by "!", such as:

$ !49

You can also execute a command by its offset from the highest entry in the history list: $ !-3 would execute event number 51, if there were 53 events in the history list.

Like other shells in the UNIX/Linux world, BASH uses special environment variables to facilitate system administration. Some examples are:

• HOME, the user's home directory
• PATH, the search path Linux uses to search for executable images of commands you enter
• HISTSIZE, the number of history events saved by your system

In addition to these reserved keywords, you can define your own environment variables. Oracle, for example, uses ORACLE_HOME, among other variables, that must be set in your environment for an Oracle installation to complete successfully.

Variables can be set temporarily at the prompt:

$HISTSIZE=100

Or, set permanently, either on a system wide basis in /etc/profile (requires root privileges), or locally in .profile.

The value of an environment variable can be viewed with the echo command using a $ to access the value.

$ echo $HOME
/home/bluher

All current environment variables can be viewed with env.

Regular Expressions and Wildcards

Many Linux commands use the wildcards * and ? to match any number of characters or any single character respectively; regular pattern-matching expressions utilize a period (.) to match any single character except "new line." Both use square brackets ([ ]) to match sets of characters in addition to *. The *, however, has a similar but different meaning in each case: Although it will match one or more characters in the shell, it matches zero or more instances of the preceding character in a regular expression. Some commands, like egrep and awk, use a wider set of special characters for pattern matching.

File Manipulation Commands

Anatomy of a File Listing

The ls command, used to view lists of files in any directory to which a user has execute permission, has many interesting options. For example:

$ ls -liah *
22684 -rw-r--r--    1 bluher   users         952 Dec 28 18:43 .profile
19942 -rw-r--r--    1 scalish  users          30 Jan  3 20:00 test2.out
925 -rwxr-xr-x    1 scalish  users         378 Sep  2  2002 test.sh

The listing above shows 8 columns:

• The first column indicates the inode of the file, because we used the -i option. The remaining columns are normally displayed with the -l option.
• The second column shows the file type and file access permissions.
• The third shows the number of links, including directories.
• The fourth and fifth columns show the owner and the group owner of the files. Here, the owner "bluher" belongs to the group "users".
• The sixth column displays the file size with the units displayed, rather than the default number of bytes, because we used the -h option.
• The seventh column shows the date, which looks like three columns consisting of the month, day and year or time of day.
• The eighth column has the filenames. Use of -a in the option list causes the list of hidden files, like .profile, to be included in the listing.

Working with Files

Files and directories can be moved (mv), copied (cp) or removed (rm). Judicious use of the -i option to get confirmation is usually a good idea.

$ cp -i ls.out ls2.out
cp: overwrite `ls2.out'?

The mv command allows the -b option, which makes a backup copy before moving files. Both rm and cp accept the powerful, but dangerous, -r option, which operates recursively on a directory and its files.

$ rm -ir Test
rm: descend into directory `Test'? y

Directories can be created with mkdir and removed with rmdir. However, because a directory containing files cannot be removed with rmdir, it is frequently more convenient to use rm with the -r option.

All files have ownership and protections for security reasons. The file access permissions, or filemode, comprise the same 10 characters described previously:

• The first character indicates the type of file. The most common are - for a file, d for a directory, and l for a link.
• The next nine characters are access permissions for three classes of users: the file owner (characters 2-4), the user's group (5-7) and others (8-10), where r signifies read permission, w means write permission, and x designates execute permission on a file. A dash, -, found in any of these nine positions indicates that action is prohibited by that class of user.

Access permissions can be set with character symbols or, binary masks using the chmod command. To use the binary masks, convert the character representation of the three groups of permissions into binary and then into octal format:

User class:	Owner	Group	Others
character representation:	rwx	r-x	r--
binary representation:	111	101	100
octal representation:	7	5	4

To give write permission to the group, you could use:

chmod g+w test.sh or chmod 774 test.sh

Default file permissions are set with the umask command, either systemwide in the /etc/init.dev file or locally in the .profile file. This command indicates the number to be subtracted from 777 to obtain the default permissions:

$ umask 022

This would result in a default file permission of 755 for all new files created by the user.

A file's ownership can be changed with chown:

$ chown bluher ls.out

Here, bluher is the new file owner. Similarly, group membership would be changed as follows:

$ chgrp devgrp ls.out

Here, devgrp is the new group.

One piece of information that ls does not provide is which files are text, and which are binary. To find this information, you can use the file * command.

Renaming Files

Two popular ways to give a file more than one name are with links and the alias command. Alias can be used to rename a longer command to something more convenient such as:

$ alias ll='ls -l'
$ ll

Notice the use of single quotes so that BASH passes the term on to alias instead of evaluating it itself. Alias can also be used as an abbreviation for lengthy pathnames:

$ alias jdev9i=/jdev9i/jdev/bin/jdev

For more information on alias and its counter-command unalias, check the man page for BASH, under the subsection "SHELL BUILTIN COMMANDS". In the last example, an environment variable could have been defined to accomplish the same result.

$ export JDEV_HOME=/jdev9i/jdev/bin/jdev
$ echo $JDEV_HOME
/jdev9i/jdev/bin/jdev
$ $JDEV_HOME

Links allow several filenames to refer to a single source file using the following format:

ln [-s] fileyouwanttolinkto newname

The ln command alone creates a hard link to a file, while using the -s option creates a symbolic link. Briefly, a hard link is almost indistinguishable from the original file, except that the inodes of the two files will be the same. Symbolic links are easier to distinguish because they appear in a long file listing with a -> indicating the source file and an l for the filetype.

Looking In and For Files

File Filters

Commands used to read and perform operations on file contents are sometimes referred to as filters. The sed and awk commands, already discussed at length in previous OTN articles, are two examples of filters that will not be discussed here.

Commands such as cat, more, and less let you view the contents of a text file from the command line, without having to invoke an editor. Cat is short for "concatenate" and will print the file contents to standard output (the screen) by default. One of the most interesting options available with cat is the -n option, which prints the file contents with numbered output lines.

$ cat -n test.out
  1  This is a test.

As cat outputs all lines in a file at once, you may prefer to use more and less because they both output file contents one screen at a time. Less is an enhanced version of more that allows key commands from the vi text editor to enhance file viewing. For example, d scrolls forward and b scrolls backward N lines (if N is specified before d or b.) The value entered for N becomes the default for subsequent d commands. The man page utility uses less to display manual contents.

Redirection and Pipes

Redirection allows command output to be "redirected" to a file other than standard output, or, input. The standard symbol for redirection, >, creates a new file. The >> symbol appends output to an existing file:

$ more test2.out
Another test.
$ cat test.out >> test2.out
$ cat test2.out
Another test.
This is a test.

Standard input to a file can be redirected with the <>

$ cat <>  
 Error messages are redirected and appended with 2> and 2>> using the format:  
 
 
$ command 2> name_of_error_file
  
 To avoid unintentionally overwriting an existing file, use the BASH built-in command set:  
 
 
$ set -o noclobber
  
 This feature can be overridden using the >! symbol between your command and output file. To turn it off, use +o in place of -o. 
 
  Redirection works between a command, or file, and a file. One term of the redirection statement must be a file.  
 
  Pipes use the |symbol and work between commands. For instance, you could send the output of a command directly to the printer with: 
 
 
$ ls -l * | lpr
  
 A command in the history list can be found quickly with: 
 
 
$ history | grep cat
  
  More Filters 
 
  Grep, fgrep and egrep all print lines matching a pattern. All three commands search files for a specified pattern, which is helpful if you can't remember the name of a needed file. The basic format is: 
 
 
grep [options] PATTERN [FILE...]

$ grep -r 'Subject' nsmail
  
 CTRL-Z will terminate output of the above or any other command.  
 
  Perhaps the most useful option with grep is -s. If you search through system files as anything other than root, error messages will be generated for every file to which you do not have access permission. This command suppresses those messages. 
 
  Fgrep, also invoked as grep -F, looks only for fixed strings, rather than the regular expressions that grep accepts. While egrep accepts patterns containing a wider selection of special characters, such as |, which signifies the conditional OR operator.  
 
 
$ egrep 'Subject|mailto' *
  
  Finding Files 
 
  The GNU version of the find command is powerful, flexible and more forgiving than classic versions found on UNIX systems. It is useful for tasks involving a directory structure, including finding and executing commands on files. The basic format of the find command is:  
 
 
$ find startdirectory options matchcriteria [actionoptions] 
  
  If you know the name of a file, or even part of the name, but not the directory it is in, you can do this:  
 
 
$ find . -name 'test*'
./test
./jdevhome/mywork/EmpWS/EmpBC4J/test
  
 Unlike classic UNIX systems, the -print action at the end is not required in Linux, as it is assumed if no other action option is designated. A dot ( . ) in the startdirectory position causes find to begin a search in your working directory. A double dot, .., begins a search in the parent directory. You can start a search in any directory.  
 
  Note that you can use wildcards as part of the search criteria as long as you enclose the whole term in single quotes. 
 
 
$ find . -name 'test*' -print
./test.out
./test2.out
  
 To produce a list of files with the .out extension:  
 
 
$ find /home -name '*.out'
  
 Remember, however, that you will probably get numerous "Permission denied" error messages unless you run the command as supersuser. 
 
  One of the most powerful search tools is the -exec action used with grep: 
 
 
$ find . -name '*.html' -exec grep 'mailto:foo@yahoo.com' {} \;
  
 , look for an html file, *.html, and execute -exec the grep command on the current file, {}.  When using the -exec action, a semicolon, ;, is required, as it is for a few other actions when using find. The backslash, \, and quotes are needed to ensure that BASH passes these terms through so they are interpreted by the command rather than the shell. 
 
    Now in Command 
 
  There are many more useful commands available in Linux, and powerful ways to utilize them, than can be covered here. Moreover, there is often more than one way to accomplish many tasks.  
 
  We have looked at only some of the most commonly used and instructive Linux file commands. A mastery of these basic but critical tools should move your Linux education to the fast track. With the man pages at your fingertips, and a willingness to experiment, you now have enough information to begin exploring the power of Linux file operations. 
 
  In my next article, I'll provide a similar explanation of Linux system commands.