Chapter 10, Testing and Quality Assurance for Production Web Sites
This lesson presents some material from chapter 10.
Objectives important to this lesson:
Production vs. development environments
Policies and related terms
Building a test plan
Security gaps and plugging holes
Web site deployment and monitoring
Not that I'm complaining, but have you noticed that the last
several chapters have been shorter than in the first half of the book?
(You've been reading the book, right?)
The chapter opens with some facts of life about software development of
you don'tpublish programs that you haven't tested
you don'twrite programs on the server that
you use for real life (the production server)
you do write code, test programs,
experiment, and correct errors on a server that the public
does not see and does not use
(the development server)
Precautions are required not just because they are good ideas. They are
required because we are trusted by our customers and clients. I am reminded
of an ad I saw many years ago from the Eli
Lilly company. It was a picture of a couple standing by a small
lake. The simple caption read, as I recall, "For over a hundred years,
we've been making medicines as though people's lives depended on them."
The implied responsibility you have when people depend on you that much
is profound. If you don't feel it at this moment, you need to think about
it for a while.
The text uses the phrases development
environment and production environment,
but the only difference between those phrases and the ones I just used
is the scope and/or scale of your project. The platform where you create
is not the platform where you interact with customers. No professional
organization would use only one environment for both. Having a development
environment lets you take chances, try new things, and fail without disrupting
your service to your customers. It gives you the freedom to reload, to
find solutions, and to test interaction with other products before you
decide it's safe to copy the new version to the production environment.
The text gives us some terminology on pages 252 and 253, but I
will warn you that it is not universal. For small changes, you may not
need to go through all four stages before you roll out a patch to
production. You may call the stages by other names on a large
application project, and you may take more steps in a very large one.
Regardless, these are the four development stages the text gives us:
Pre-alpha - This is the phase in which a project is
discussed. It may go on to planning, or it may be rejected.
Alpha - This phase includes all development of the project
that takes place before the customer sees a prototype.
Beta - In this phase, the developers are confident, but
cautious, so this version is show to testers to find errors that the
developers did not see, or did not know how to test. This phase can
cover multiple iterations of the project.
Release Candidate - This phase has a product that has been
well tested. The developers and testers have confidence that there may
be no problems with this version of the product. Limited live testing
should be conducted when possible. At all stages, however, remember von Moltke's first rule. I prefer the way I first
heard it: no plan of battle ever survives first contact with the enemy.
The lesson is to expect to make changes and to have to change your
The text spends five pages on policies (company rules) and
other related terms that properly belong in a course on policies
and auditing. The truth here is similar to the truth above:
different concepts are called by different names in different
organizations. The items below can be called by different names, but
there is typically a hierarchy like this in large organizations. A policy
is a rule, or a set of rules, that affects how we want
our organization and its employees to function.
The idea behind a policy may start with a principle, which is
often a broad, general statement of what we believe to be right,
true, or beneficial. A policy is more detailed,
and more specific about what we expect our people to do.:
Principle - a general statement about what we
believe or require in our area of authority (we will use only two
computer vendors at a time); what we expect
Policy - rules about the conduct of our organization
with regard to particular actions (we will limit ourselves to
particular models chosen by the IT department); how we will approach
Standard - a method or process that may be
procedural or technical (orders are to be placed by approved requesters
within each work area); what steps are to be followed to assure general
compliance with policy
Procedure - a detailed set of steps to follow to be
in compliance (requests are to be made to your manager, who will
forward approved requests to your authorized requester); variations or
limitations that apply to specific work areas, to be followed if they
apply to your area
Guideline - a suggested addition to any of the items
above that is recommended but optional (submit your requests two weeks
before the end of a quarter to allow processing time); do this to make
it work better
Definitions - when there is room for misinterpreting
any of the above items, definitions should be offered to clarify their
language and intent; if you didn't understand something, read this
On page 258, the text begins a discussion about plans that lead to good testing. Testing, as noted above, is expected
in all environments, development and production (and more, if you have
them). Some projects will have separate test environments,
which can be
useful for developers who are able to watch while testers stress the
new product. The text provides separate lists of elements to test for
web sites and for applications. For a decent discussion this week, I
will assign you to report on a few web sites that violate one or more
of the elements on pages 258 and 259. (This is Discussion 10, found in
section on finding security gaps is just a bit of advice about taking a
minimal approach to rights and permissions. Restrict all permissions
that are not required. Test security flaws that are being corrected,
but also test those that have been problems in the past, for any web
site that has had a notorious failure.
The text recommends a systematized approach to correcting
flaws. Determine what they are, prioritize existing lists of problems,
develop and carry out mitigation plans, and continue testing. Making
changes in a system often causes new problems, so testing throughout is
a never-ending, tiresome job. I would recommend that you celebrate
finding no problems, finding problems, and correcting problems. There
should be recognition for each of these events because they keep the
Continue the reading assignments for the course.
Discussion 10 - Web Site Analysis -- Lack of Testing?
Perform lab 5 in our local set, and lab 7 on the publisher's site.