Testing
The Testing Team has the task of developing test plans, testing the software
during development and generating test reports, and retesting the software
during acceptance testing and maintenance.
There are three levels of test plans to do develop for the software (some of
which you have been introduced to under different names):
- Validation Test Plan - This
tests to see if all aspects of the Requirements Specification Document
that will be implemented in the senior project (as per the written
agreement with the client) have been met. These tests should be from
the user's point-of-view i.e. a "black-box" test, with no
knowledge of the design and code. Thus, it is highly recommend that
the validation test be done immediately after the requirements have been
agreed upon by the client and the project team. Use cases are often
easily translatable into test cases.
- Integration Test Plan - This
tests to see if all aspects of the architectural (high-level) design have
been met. The integration test plan should be developed before
detailed design and implementation of the design begins. (Of course,
there are often several iterations of the design and implementation, so
there should be several iterations of the Integration Test Plan as
well. There are two sublevels of the integration test:
- Object and Independent
Module Testing - This tests messages passed between methods (functions)
of when more than one class is involved (including functions that are
"independent" i.e. not part of a class. A single test
case often involves a sequence of calls to different functions.
- Class Testing - This
tests message passing between methods of a particular class. If
there is no object-oriented aspect of the design - which would mean that
there are no classes, only independent modules - then class testing would
not be applicable to the software.
- Unit Test Plan - This tests
to see if each single module (function) works correctly on its own.
This is done using the detailed design (e.g. pseudocode, flowcharts) and
the actual source code.
Note that all of the plans on how to test the code should be
developed before any actual testing begins. Test implementation is
done in the reverse order of their creation i.e. unit testing is done first,
then integration testing (class testing, then object and independent module
testing), then validation testing. Every test case at every level has an
expected result. Once the tests are implemented and the actual results
can be filled in, then the test plan becomes a test report. Thus
the same template can be used for both the plan and the report, since the form
of two are same, only that the test plan does not have the actual results
filled in.
There are other types of testing, which may be applicable to your project,
but will either not be formalized or is not applicable in this project process:
- Regression Testing - This is
when some aspects of the software need to be retested depsite earlier
successful tests, due to the fact that some other part of the source code
was changed, and it might change the test results. This includes
testing done during the maintenance cycle. Although it is highly likely
that you will have to do some regressing testing, you will not be required
to separately document them outside of the normal test reporting
mechanisms.
- Alpha Testing - This is when
the development team has potential or actual future users of the software
use the software under controlled conditions (usually with members of the
development team present). Such testing by definition has the form
of a validation test; if you do any testing during project development, it
should be included with the validation test reports.
- Beta Testing - This is when a
potential user of a product which will be commercially marketed (i.e. is
not for a single, specific client) is given a demonstration copy of the
software in order to use it for the purpose of finding possible bugs, and
provide overall feedback. Since projects using this process are for
a particular client, Beta testing does not apply here.
- Acceptance Testing - This is
the testing that the customer does to ensure that the requirements
specified in their written agreement with the project team has been
met. Since acceptance testing is done by the customer and not by the
project team, a test plan or report is not needed; however, you will need
to track defects (errors) found by the customer and their status using a
separate template.
The following templates should be used:
Note that the validation template should be used once for
all of the validation tests; the same goes for the integration and unit test
levels. There are numerous test cases, however; there should be one test
case template for each case, and the same template is used for testing at all
levels.
Inspections: There should be at least one inspection of each test
plan (before any implementation of that plan begins) and one inspection of each
test report (sometime before delivery of the software to the customer for
acceptance testing). Other inspections can be done as deemed necessary by
the Project Manager.
Web Pages: Test plans and test reports should be placed in three
separate sections in the main project
page, the inspection reports page, and the historical archive page: one
for validation tests, one for integration tests, and one for unit tests.
Submitter: Testing Team Leader
Document File Names:
- Testing<ProjectName>ValidationMMDDYYYY.html
- Testing<ProjectName>IntegrationMMDDYYYY.html
- Testing<ProjectName>UnitMMDDYYYY.html
- Testing<ProjectName>TestCase##_MMDDYYYY.html
(e.g. TestingBankingSystemTestCase09_10122002.html for test case 9 for the
"Banking System" project, either initially created or updated on
October 12, 2002)
- MMDDYYYY should be the date
of submission to the project web site
Note that the naming convention is the same for test plans
and test reports.