CS 1110 - Introduction to Programming

Starting Out with Python
Chapter 6, Files and Exceptions

Objectives:

This lesson covers chapter 6, which reading and writing files, and with handling some types of run-time errors. Objectives important to this lesson:

1. File input and output
2. Using loops to process files
3. Processing records
4. Exceptions
   

Concepts:

Chapter 6

File Input and Output

The chapter opens with the thought that sometimes you need to save data in a file, so that is available for later use. This could be later use by the same program, or by another program, but it works the same: a file will store your data for longer terms, providing you can access the operating system command to manage it. The text mentions something you should already know, that major applications all save, open, use, and close files. Even web browsers save files that contain information about your having been to a web site. On page 288, the text illustrates the contents of three variables being written to a file on a disk. On page 289, the text simplifies the concept into three steps used by programs that read and write to existing files:

1. Open the file, giving the running program access to the file's contents.
2. Process the file, which may include searching, reading, and writing to it.
3. Close the file, which releases the connection between the program and the file. This is as important as the other steps, if you intend to ever use that file again.

Although all files are stored as binary data, there is a difference between files that are considered text files those that are considered binary files, which are more often executable files. Python can use both types of files, but this text will discuss text files, which will be readable by standard editor, such as Notepad.

From the early days of computing, we inherit an early method of access called sequential access, which means that we read a file starting at one end, and move through each bit of data to the other end. This was a fine way to do things when we stored data on tape drives, but it is less robust than another method called direct access, or random access. This second method became available when data began to be stored on disks and drums, on which we can move the read/write head to any part of the drive we need it to access next, without having to pass through all of the data between that new position and its last position. For the programs in this text, the data files will be small enough that there will be little difference between the two methods, so we are told that we will use sequential access with them.

The text becomes a bit elementary for a minute, telling us about filenames and extensions. This is not terribly relevant to the more important information on page 290 that for a program to be able to transfer information to a variable from a file, there needs to be an intermediate file object that acts a bit like a driver between your program and the operating system you run under. You can also think of the file object as being a special kind of variable, which is how it is illustrated on page 291.

file_variable = open('filename', 'mode')

In this example, we supply the program with a variable name to use for the file object, we specify that we are calling the open() function, and we pass two arguments to it: the actual filename, and the operational mode we are about to use. The two arguments are passed as strings. The text offers three mode strings as examples, but there are several more. I will give you a better list:

The text warns us that the w modes are destructive. If we open a file that already exists in w or w+ mode, we will destroy any contents already in that file. This should only be done when you want to start fresh with an empty file and you do not need the contents that were in the file before..

On page 292, we see that we can add a path to the name of a file in the filename string. A path is not necessary when the target file is in the same folder as the program itself. When a path is necessary, note that the example in the text puts a letter r before the filename string. We are cautioned to do this when the operating system uses backslashes as directory separators in a path, but it would not be wrong to use it regardless. The r is a switch that causes the interpreter to read each character in the string as a raw character, not as part of an escape sequence. So there's a good review question: give an example of a path that would require the r switch, and an example of one that would not.

Methods

In the section about writing data to a file, the text introduces a new kind of function, one that is part of a file object. Objects can have several parts, one kind of part being a method, which is really a function. The file object is created already having a method called write(). It is used by calling it with dot notation, following the name of the file object, as the text shows us.

file_variable.write(string)

We are telling the file object to run its write() function, writing the string that we pass as an argument to the file associated with the file object. As the text explains, we can pass a literal string, or a variable that contains the string to pass. The text warns us that we should not call the write() function of a file object unless the file was opened in one of the modes above that are allowed to write to files. If we call a write() function for a file object that was opened in read only mode, the function call will produce an error.

Another method that is part of file objects is the close() function, which will close the connection to the file, but first it will flush the buffer (a small section of memory) that may still be holding data that has not been written to the file yet. Typically, data is written to a buffer first, then written to the file. Closing the file completes any writes that still have to be done, breaks the connection, and removes the file object from RAM. In example program 6-1, we see a file opened in w mode. Three calls are made to the file object's write() method, then a call is made to its close() method. Note that the close() call does not require an argument. In this example program, the author has included an escape sequence to write a newline character (\n) at the end of each string. This will make it easier to read the file.

You may wonder about the open() function that we called at the start of this lesson. It creates a file object, but it is not part of one, so it does not need dot notation to call it.

Sometimes you only want to read data from a file, which is what example program 6-2 does. The open() function is called, opening a file in r mode. To actually read the data in a file, the program called the read() method of the file object. The lines that do this are like these:

infile = open('philosophers.txt', 'r')
file_contents = infile.read()

The example program closes the file next, then prints the contents of the file_contents variable, which continues to hold the data that was read into it, regardless of the file being closed. The read() method does one thing: it copies the entire contents of the specified file. The example file has three lines in it, so all three were copied to the provided variables.

The text points out that we may have only wanted one line of the file, or we may have wanted each line in a separate variable. The next method we learn about is the readline() method that reads one line at a time from a file. The first time it is called, it reads the first line in a file. Each time it is called after that, it reads the next available line, the one after the one that it just read. The limitation of this method is that it reads data until it sees a newline character. If the file you are reading has no newline characters, it will not return anything from the file.

For this method to be useful with a file, the file needs to have newline characters at the end of each logical line. Example program line_read.py shows that each line of the sample file can be read separately, each into a new variable. It also prints the value of each variable, and the text points out that each call to the print() function generates a new line. Since each of these strings contains a newline character, the output of the three print() calls in the program has blank lines separating the actual lines of data. We may not want that, so we will address removing newline characters in a moment, First, the author wants to discuss putting newline characters in the data.

The text changes course a bit next, to consider that when we have asked users for data in our programs, we have not terminated any of those data elements with newline characters. The sample program write_names addresses the issue. It asks the user to enter the names of three friends, assigning them to variables in each case. It then opens a file, with the intention of writing the collected data to it. The program then calls the write() method of the file object, but instead of using just the name a variable, it calls the method like this:

myfile.write(name1 + '\n')

As you may recall, the program is using the + operator to concatenate (add) the newline character to the end of the data stored in the name1 variable. It then passes the newly constructed string as the argument to the write() method. This would make the resultant file usable with the readline() method.

That's nice for making the file compliant with readline(), but what if you don't actually want the newline characters in your data when you start to use it? Sample program strip_newline.py introduces another method that is pretty flexible. It works like this:

infile = open('philosophers.txt', 'r')
line1 = infile.readline()
line1 = line1.rstrip('\n')

To understand what is happening, you need to know that rstrip() a method, but it is not a file object method. All strings in Python have three methods you can use to shed unwanted characters. (They have other methods, too, but these are important now.) The example program uses rstrip(), which is a string method for stripping characters from the right end of the string. This method can be use without arguments, but it is more useful, as in this case, to tell it precisely what to strip off and discard. As may be obvious, you are not limited to stripping off newline characters. You could supply any character as an argument, and the rstrip() method would be happy to throw away the named character from the right end of the string.

The text moves on to appending data, which is the third method of writing, triggered by using the a or a+ mode. You use the write method as shown in the other examples. The difference is that this method is not destructive. It always preserves data that is already in the file.

Following that section is the method many of you have been waiting for. The write() method only reads and writes strings. This is a problem when the data we want to write is actual numeric. Up to now, we have not cared about data types because the functions we have used did not care and the variables did not care. Let's look at some of the code the next sample program uses:

outfile = open('numbers.txt', 'w')
num1 = int(input('Enter a number: '))
outfile.write(str(num1) + '\n')

The text opens/creates a file to which it will write. The user is prompted to enter a number, which is stored in a variable. On the third line, let's start from the innermost part on the right, as the computer will do. The str() function is called, and handed the value of num1 as an argument. This converts the number the user entered into a string, a series of characters that only looks like a number. The program then concatenates a newline character to the end of the string. This is something it can't do with num1, because that variable holds a number, not a string. Think about that for a moment, because it matters. As the text tells us, but does not explain, we cannot do math with strings. In the next step on this line, the new string, which has not been saved in a variable, is passed to the write() method of the file object, and it is written to the specified file.

The book tells us that when we read data from a text file, it is always read as a string, mainly because the functions that do so treat the data that way. It shows us that the int() function can be used to convert data from string type to int type. Previously, we used this to convert from float to int, but it is nice to know that the int() function can convert from string as well. The float() function works for this as well, if we know to call it instead of int(). In the sample program read_numbers, the program opens a file, reads a line, feeds the line to the int() function, and assigns the result to a variable. It then does math with the numbers, something that you would have assumed to be possible anyway. Run that program without the int() calls, and you will see that they are necessary.

Using Loops to Process Files

It may have crossed your mind, looking at the chapter examples so far, that the author could have written several of them using loops instead of calling readline() each of the times he needed to do so. The trick to going that way is to know when we have reached the end of the file. The author tells us two ways we can know that.

First, when using readline(), the return value will be an empty string when we read beyond the end of a file. That being said, the author makes a plan for his loop, in pseudocode and in a flow chat. His logic is interesting and sensible. What if there is nothing in the file we are trying to read? Well, that would mean we really don't need to run a loop to read the contents, but only in that case. So, he sets his first read outside the loop , then builds a while loop with an entry condition that says while the return value of a read is not an empty string, process what was read, use it, then read the next line, and test again. This would continue until the program tries to read the next line and comes up empty. This is the method used in example program read_sales.py, which reads the data in a file containing text versions of numbers. By the way, when we read a line from the file first to see if there is any point to running the loop, that is called doing a priming read. Sort of like checking to see if there is any (thing you like) in the bottle before pouring a glass of it. We will examine this program in class.

The second method the book shows us that uses a loop requires that it be a for loop. The while loop used calls to readline(), but a for loop is special. In Python, the code to use a for loop, as shown in example program read_sales2.py, should look strange to you. It takes advantage of a built-in assumption the for loop makes. Here is a sample of the code:

sales_file = open('sales.txt', 'r')
for line in sales_file:
    amount = float(line)

The program opens a file to read it, and creates a file object called sales_file. That should look fine to you. The next line starts a for loop, creates a variable called line, and assigns it a value from the file object. That looks unreasonable. What is happening is that the for loop knows something we are about to learn. When the argument after the word in is a file object, the first value to assign to the variable is the contents of the first line of the file it points to. Say what? The for loop knows that we are taking values from a file. It pulls lines from it, one at a time, and assigns them to the loop variable each time the loop runs. That's quite handy, just impossible to guess without knowing that's how Python for loops can work. So, now that you know the secret of this magic trick, you can do it yourself. By the way, we did not have to call the variable "line". We could have called it presto or basketball just as well: its name is not where the magic resides. And the loop will run until the file is out of lines to load, so it does the check for end-of-file all by itself. Aren't for loops, marvelous?

Processing Records

Records are lines in a database table, as the text says, " a complete set of data about an item". Most databases use tables, in which each column contains one kind of information, and each row (line) is about one entry (item) in the database. In the text we see an example from an employee database. Each line in the file contains the record for one employee, and each record contains three fields that hold specific kinds of information about that employee.

Example program save_emp_records.py show how we might save records to a file. It first asks the user how many records we will enter. The program then opens a new file called employees.txt. The program then begins a for loop that will run from 1 to the number of records to enter + 1. The program is a little hard to understand, but the main idea is that it runs a loop, in each iteration it asks for data for each of the three fields, it adds a newline character to each variable in which you save data when it writes that data to the file, then it loops around for the next record. This looks odd, because it does not seem to be making any difference from one record to the next. The next example file reads that data file and put it on your screen. The key to each of them working is knowing how many fields are in a record.

The text goes on about records for several gray pages, but we will move on to the next topic.

Exceptions

The text explains that an exception is an error that stops the program, which describes many of the errors you may have seen in your programs up to now. In the example program division.py, the user is asked to enter two numbers, the program divides one by the other, and the answer is given to the user. This is fine, until the user enters a 0 as the divisor, which leads to an undefined math operation: division by zero. To avoid this kind of error, now that we anticipate it happening, example program division.py shows us that we can set up a conditional test that carries out the division problem if the divisor is not 0, and reports a warning to the user if the divisor equals 0. This approach to avoiding such a problem is a simple error trap.

The text warns us that some errors can be anticipated, but we can't always avoid the problems created by users who don't follow instructions. Even if we prompt a user to enter a number in numerals, the user may still enter a word or a nonsense string. To provide for as many kinds of error conditions as possible, look at example programs gross_pay1.py and gross_pay2.py. The programmer encounters and error when the first one has a user enter a word instead of a value. Note that the error generated has a name in the last line of the error output. It is called a ValueError, which is one word. In the second program, the programmer has set up two sections for the program, one called try: which is followed by the code that we intend to run. This code is called the try suite. The program tries to run that code, but if a ValueError occurs, the program does not halt. It jumps to the other section of the program which is called except ValueError:, which tells the user that an error has happened. Execution then falls to the next line of code after the part we might call the except suite, but the text calls it the exception handler, which is a more common name for it. The two sections together are a try/except statement. I don't care for the name much, but we are stuck with it.

In order to account for more kinds of identified errors, we can have more than one except clause. Look at example program sales_report1.py. The author has tested this program and identified two exception conditions that may occur, an IOError and a ValueError. There is a separate except clause for each one, and one more for good measure: an except clause that has no named exception type associated with it. Think of this one as a catchall for any exceptions that we did not anticipate. This is a good practice, because we cannot be sure our testing caught all possible errors. This kind of exception clause can be the only one you need, if you do not wish to present a specific message for each kind of error.

Okay, so we can catch exceptions that we anticipate, and we can catch exceptions that we did not anticipate. There are two more magic words that belong in this discussion. The first one is else. If the try suite executes successfully, the except clauses are all skipped. We can add an else: suite to the program after all the except clauses, which will be the next block to execute.

The other magic word is finally. This allows us to create a finally: suite, which is executed as a block after the try:suite has been tried, and after any exceptions have been caught. In other words, it runs after the previous suites, regardless of what they did. As you might imagine, it must appear after the try: suite, and after all of your exception handlers.