ITS 2320 - Linux II

Chapter 1, Exploring Linux Command-Line Tools


This lesson introduces the student to concepts that relate to using command line utilities. Objectives important to this lesson:

  1. Working on the command line
  2. Using filters with text streams
  3. Streams, pipes, and redirects
  4. Searching with a regular expression utility
  5. File editing

This chapter covers a lot of material from Linux I. If you do not remember Linux I, you should read it carefully. The authors explain that this book is about getting ready for a Linux Professional Institute Certification 1 (LPIC-1) version 5 exam. They point out that the exam is about versions of Linux that were available when it was written. In that regard, they suggest that you may want to install the versions they used when they wrote the text:

  • CentOS 7 Everything
  • Ubuntu Desktop 18-04 LTS
  • Fedora 29 Workstation
  • openSUSE 15 Leap

This is not a list of all Linux distributions, but it is the list suggested by your authors. As a correction, I found I could not download a copy of CentOS 7, so I pulled down CentOS 8 instead. On the Ubuntu side, I did find a download for Bionic Beaver, so I pulled that down as well.

By the way, the first pages of each download document describe downloading and installing VMware with a free license from the college. Note that you can only do this once a year, so it is IMPERATIVE that you grab a screen image of your license number during this process, or that you write it down and do not lose it. If you have already installed VMware for another course, you do not have to download it again, and you certainly don't have to do it twice for this assignment.

The text continues with some tips on easily reaching a command line.

  • Ctrl-Alt-F2 will typically take you to the tty utility, which will let you log in to a command line environment.
  • If you are in an Ubuntu GUI, try Ctrl-Alt-T to open a terminal emulator.
  • If you are running a CentOS or Fedora GUI, try the Activities menu, search for term, and open a terminal session.
  • If you are running an openSUSE GUI, try the Applications menu, search for term, and open a terminal session.

In each of the cases above, you will probably be in the default shell for your Linux distro. If that is not the one you want, several others may be available:

  • Bourne ShellBash - a revised version of the Bourne shell, usually available on all distros
  • Dash - a smaller shell usually found on Debian systems
  • KornShell - from Bell Labs, compatible with Bash but with more C language characteristics
  • tcsh - TENEX C shell; has more C language features
  • Z shell - combines features of Bash. KornShell, and tcsh, with more enhancements for Bourne features

The text mentions the fact that the default shell for a distro is often called by /bin/sh, which is linked to the actual shell executable. In the case of Bash, the link may point to /bin/bash. Try the commands on pages 5 and 6 to confirm what the default shell is for your system.

The next few pages discuss the pitfalls of using the echo command.

  • Remember that you have to be careful about enclosing a quotation mark that you want to appear on the screen inside a pair of another kind of quotation marks.
  • Be careful about using the $, which usually means "show me the value of" and is immediately followed by a variable name.
  • If you get tired of messing with sets of quotation marks, use the backslash character as a signal to echo that the character that follows the backslash is just a character to print. This is a backslash: \

The text reminds us that there are internal (contained in the shell program) and external (contained in their own files) commands. This does not usually matter, but the difference is important when you are troubleshooting where a problem is: bad copy of the shell, or missing/bogus copy of a file.

Before we go any further, we should consider why we might want to use a command line interface instead of a graphical user interface. This video discusses some of the reasons, and it also presents a tutorial on things you might not remember from Linux I.

So, flexible (although cryptic) options, speed of command entry, and speed of execution of file activities.

The text revisits variables, mentioning several environment variables which affect how the workstation runs. Several are discussed in later chapters, which is good. Talking about them out of context is not very enlightening. For now, know that the set command will show you most environment variables and their current values. You may wish to see the value of the path variable, which can be different for each user. Review: what is the purpose of the path variable?

You should review the material in this first section of the chapter, reading more carefully anything that is not familiar. For a more personable discussion of commands that matter, play the video below.

  1. sudo
  2. touch
  3. echo
  4. cat
  5. mkdir
  6. mv
  1. ls
  2. wget
  3. find
  4. stat
  5. chmod
  6. rm

The second section of the chapter discusses file editors, which means it discusses text editors. It talks a lot about the vi editor, which is typically used because it is always there, and because the user has nothing else.

Text editors can be line editors or screen editors, the difference being what you see and what you can edit at any given moment.

  • Line editors - these apply changes to one line of text at a time
  • Screen editors - these allow a user to move the cursor about on the page and change anything in any order

The main reason for mentioning this is to console us that vi is a screen editor, and that it is more user friendly than its ancestors. (The earlier ones must have been a real pain.) Other editors exist, and you may want to consider any of them as alternatives. See this short review of several editors that can be added to most UNIX/Linux systems.

vi is called vi because it is meant to be more visually oriented than older editors. As the text explains, it immediately displays changes that you make. (Rather what we would hope for, isn't it? Other editors did not display their changes to their users. You may wish to gasp in horror.)  The book then tells us that vi has three modes, each of which is used for a different purpose. These modes are called by other names in other documentation, which can lead to a lack of understanding when looking for supplemental training material.

  • insert mode - When vi first starts, you should be in command mode (see below). You can enter the insert mode by typing a lower case i. This is the mode in which you do most of your composing and editing. Key presses made in this mode are assumed to be text, until you press a command to enter another mode.
  • command mode - You can enter this mode from the mode above by pressing the Escape key. This mode allows you to issue commands to vi, like save, delete, and quit. Any key presses made in this mode will be treated as commands to the program.
  • ex mode - You enter this mode by first entering command mode, then pressing the colon key. You can issue commands in this mode that are more complex than those available in command mode. The text explains that this mode emulates the operation of another editor, called ex, which I feel sure you have never used or heard of.

If that is not enough to make you run screaming into the night, congratulations. Read the text, and keep breathing deeply and slowly.

Some people are passionate about their chosen text editor, mainly because it has features that they use a lot. This video is about the vim editor, an improved version of vi. The presenter likes it very much, and that is really the point. If you use something because it does what you need it to do, you will probably like it, or you will find something you like better. And you will only do that if you have a reason to use a text editor.

Sometimes you don't need a real editor. If you just want to see what's in a text file, you can use the cat command. If you want to combine the contents of one file with those of another, you could do this with cat and the >> redirection operator, as you should remember, but there is another way:
cat file1 file2 > file3

This syntax would read file1 into memory, append the contents of file2, then write the resulting file as file3, The fact that the contents of file2 are appended to those of file1 in memory is unexpected, but that's how it can work.

To see only the first or last ten lines of a file, try the head or tail commands. It is useful to know what you want to with a file in order to use the right tool.

Selection commands are commands that extract information from files.

  • comm - compares files, shows differences
  • cut - selects columns of data
  • diff - compares files, selects differences
  • grep - searches for data, selects lines or rows where it is found
  • head - selects and shows lines from the beginning of a file
  • tail - selects and shows lines from the end of a file
  • uniq - selects unique lines (rows) in a file
  • wc - counts characters, words, or lines

Manipulation and transformation commands do something with or to data or files.

  • awk - starts an instance of awk, which allows you to use awk commands to manipulate data (yes, it's a language)
  • cat - creates, combines, and/or displays files
  • chmod - changes a file's security mode, which means it is used to grant or remove rights to a file
  • join - assuming files contain tables of data, this combines those tables into one table, like doing a SELECT command in Boolean logic
  • paste - takes columnar data from two files and creates a table containing those columns in a new file
  • pr - prints selected files
  • sed - performs edits on data in files
  • sort - places data in a specified order

The text moves on to discuss regular expressions, which is a strange name for a method of searching for text in a file. Let's try to combine some ideas. Use the grep utility, listed above, to search for text in a file, sed to edit a file/stream, and sort to arrange data.

The following video demonstrates the head, tail, and more commands. I looked at a lot of them that were dreadful, so enjoy this one for its clarity and brevity.

Another idea in the chapter is about data streams. Streams are communication channels between a program (or an OS) and its hardware/software environment. They exist for multiple programming languages and for the operating systems we are talking about. We generally care about three streams:

  • stdin - the standard input stream, typically fed by a user's keyboard and mouse; its stream number is 0
  • stdout - the standard output stream, typically feeding to a monitor and/or speakers; its stream number is 1
  • stderr - the stream used to notify users about errors, which is typically done on the same hardware used by the stdout stream; its stream number is 2

C, UNIX, and Linux treat all these streams like files, just like they do with everything else. They send data that is meant for a stream into it the same way they send data to any other file. The operating system you are running defines what the streams connect to.

The text explains that we can redirect the output of a process to a file (including the three streams) by using greater than symbols. Most processes take their input from a default location, such as stdin, but we can tell a process to take input from a file using less than symbols.

  • sending the output of a process to a file: command_name > filename
    The example above would take the output of a command and write it to a file, overwriting that file if it already existed.
  • appending the output of a process to a file: command_name >> filename
    The use of the double greater than signs in this example would take the output of a command and write it to the end of a file, creating that file if it does not already exist.
  • Collecting errors in a file: command_name 2>> filename
    The example above would take any error messages generated by the command and write/append them to a file, creating that file if does not already exist. Note the use of the numeral 2, meaning file stream number 2, to collect only error messages. Had the operator been 1>>, the command would have captured standard output, but not errors, in the log file instead.
The text describes using the less than sign to use a file's contents as input for a command.You should remember that < feeds the command on its left while > sends output to the file on its right.

That ought to be enough for right now.