Chapter 10, Testing and Quality Assurance for Production Web
This lesson presents some material from chapter 10. Objectives
important to this lesson:
Production vs. development environments
Policies and related terms
Building a test plan
Security gaps and plugging holes
Web site deployment and monitoring
Not that I'm complaining, but have you noticed that the last
several chapters have been shorter than in the first half of the
book? (You've been reading the book, right?)
The chapter opens with some facts of life about software development
of any kind.
programs that you haven't tested
programs on the server that you use for real
life (the production server)
you do write code,
test programs, experiment, and correct errors on a server that
the public does not see and does not use
(the development server)
Precautions are required not just because they are good ideas.
They are required because we are trusted by our customers and
clients. I am reminded of an ad I saw many years ago from the Eli
Lilly company. It was a picture of a couple standing
by a small lake. The simple caption read, as I recall, "For over a
hundred years, we've been making medicines as though people's
lives depended on them." The implied responsibility you have when
people depend on you that much is profound. If you don't feel it
at this moment, you need to think about it for a while.
The text uses the phrases development
environment and production
environment, but the only difference between those
phrases and the ones I just used is the scope and/or scale of your
project. The platform where you create is not the platform where
you interact with customers. No professional organization would
use only one environment for both. Having a development
environment lets you take chances, try new things, and fail
without disrupting your service to your customers. It gives you
the freedom to reload, to find solutions, and to test interaction
with other products before you decide it's safe to copy the new
version to the production environment.
The text gives us some terminology on pages 252 and 253, but I
will warn you that it is not universal. For small changes, you may
not need to go through all four stages before you roll out a patch
to production. You may call the stages by other names on a large
application project, and you may take more steps in a very large
one. Regardless, these are the four development stages the text
Pre-alpha - This is the phase in which a project is discussed.
It may go on to planning, or it may be rejected.
Alpha - This phase includes all development of the project
that takes place before the customer sees a prototype.
Beta - In this phase, the developers are confident, but
cautious, so this version is show to testers to find errors that
the developers did not see, or did not know how to test. This
phase can cover multiple iterations of the project.
Release Candidate - This phase has a product that has been
well tested. The developers and testers have confidence that
there may be no problems with this version of the product.
Limited live testing should be conducted when possible. At all
stages, however, remember von Moltke's first rule. I prefer the way
I first heard it: no plan of battle ever survives first contact
with the enemy. The lesson is to expect to make changes and to
have to change your plans.
The text spends five pages on policies (company rules) and other
related terms that properly belong in a course on policies and auditing. The truth here is
similar to the truth above: different concepts are called by
different names in different organizations. The items below can be
called by different names, but there is typically a hierarchy like
this in large organizations. A policy is a rule,
or a set of rules, that affects how we want our organization
and its employees to function. The idea behind a
policy may start with a principle, which is often a broad,
general statement of what we believe to be right, true,
or beneficial. A policy is more detailed, and more
specific about what we expect our people to do.:
Principle - a general statement about what we believe
or require in our area of authority (we will use only two
computer vendors at a time); what we expect
Policy - rules about the conduct of our organization
with regard to particular actions (we will limit ourselves to
particular models chosen by the IT department); how we will
approach the expectation
Standard - a method or process that may be procedural
or technical (orders are to be placed by approved requesters
within each work area); what steps are to be followed to assure
general compliance with policy
Procedure - a detailed set of steps to follow to be in
compliance (requests are to be made to your manager, who will
forward approved requests to your authorized requester);
variations or limitations that apply to specific work areas, to
be followed if they apply to your area
Guideline - a suggested addition to any of the items
above that is recommended but optional (submit your requests two
weeks before the end of a quarter to allow processing time); do
this to make it work better
Definitions - when there is room for misinterpreting
any of the above items, definitions should be offered to clarify
their language and intent; if you didn't understand something,
On page 258, the text begins a discussion about plans that lead
to good testing. Testing, as noted above, is expected in all
environments, development and production (and more, if you have
them). Some projects will have separate test
environments, which can be useful for developers who are
able to watch while testers stress the new product. The text
provides separate lists of elements to test for web sites and for
applications. For a decent discussion this week, I will assign you
to report on a few web sites that violate one or more of the
elements on pages 258 and 259. (This is Discussion 10, found in
The section on finding security gaps is just a bit of advice
about taking a minimal approach to rights and permissions.
Restrict all permissions that are not required. Test security
flaws that are being corrected, but also test those that have been
problems in the past, for any web site that has had a notorious
The text recommends a systematized approach to correcting flaws.
Determine what they are, prioritize existing lists of problems,
develop and carry out mitigation plans, and continue testing.
Making changes in a system often causes new problems, so testing
throughout is a never-ending, tiresome job. I would recommend that
you celebrate finding no problems, finding problems, and
correcting problems. There should be recognition for each of these
events because they keep the system running.
Continue the reading assignments for the course.
Discussion 10 - Web Site Analysis -- Lack of Testing?
Perform lab 5 in our local set, and lab 7 on the