Test Scenario Template

Test Scenario Template:

In Excel sheet this would start up with

Functional Area

Test Set Name(s)

Responsible Tester

falling in template this would contain

1.) Requirement Identifier (Source Document field in QC for Test Plan):

this may be Functional ID, Technical ID, Business Requirement ID, Business Rule or Other Identifier. This field can be used to load to requirements for mapping testcases you may also use this fields for populating the field source document in test plan for filtering to fine test cases to link to requirement.

2.) Requirement Description:

This is the description of the requirement or rules to be tested

3.)Requirement Priority :

This is the priority of the requirement to be used for priortising the test cases. This should be done by the Business area prior to be finalised for approvals and consumed by project team for delivary

4.)Scenarios to Test  (Test Case Description in QC):

This will be mapped as the test case description in quality center Senarios should be detailed and specific to the requirement

5.)How to validate scenarios:

What types of transaction or interaction with the application will validate the identified test scenarios.This should be very high level validation.

6.)Types of Tests to Execute :

List all the appropriate types of testing provide adequate test coverage of the scenarios

7.)Source Document for where Requirements originated:

Enter the source document from where the requirements originated this will map to source document field used in requirement section of quality center

8.)Subject (folder path – use the ‘\’ to indicate the folder levels):

Path in test plan from the quality center to load test cases.

9.)Test Case Name:

This is used in test plan remember to use the test case naming tool to verify.

10.)Created for IS IntakeID:

IntakeID is used by Quality Center to link a test cases for a project if project is new make sure you request InstakeID to be added by QC Admin.

11.)Release Date:

Release Date of the project make sure it aligns with release date with Quality Center.

12.)Designer:

Person writes the test cases and loaded to quality center

13.)Step Name/Number:

Step name or number to be loaded to quality center

14.)Step Description:

Test step description

15.)Expected Test Result:

Expected result for eash test step used to compare against actual results

16.)Application:

Enter the application the test case applies to refer to quality center for list of valid application

17.)Test Team Owner:

Enter the test team owner the test case applies to refer to quality center for list of valid test teams

18.)Test Area:

Enter the test area the test team applies to. Refer to quality center for list of valid test teams

19.)Workflow Days:

The field is required and used to capture the total work flow days to test the test case.

20.)Effort(hrs):

This field is required and used to capture effort hrs to test the test cases.

Contii.. preparation of test scenarios

Milestone: inside the table it included with QA Project Plan template and training plan too

	ISBA-QA Milestone Dates for Project:  –
	NOTE: This document is recommended for projects that do not have their own project plans.
	Milestone	Type	Responsible	Project Plan Date	Actual Date
1	Finalized Requirements & Prioritize	Deliverable	Business Analyst
2	Requirements Review	Meeting	TPM or SDM Schedules
3	QA LOE and Resource Plan	Deliverable	QA Test Lead
4	Finalized System Access and Test ID’s*	System Access*	QA Test Lead
5	Automation Identified*	Automation*	QA Test Lead
6	Finalized Technical Documentation	Deliverable	ISBA Design and/or Development
7	Technical Documentation Review	Meeting	TPM or SDM Schedules
8	QA Tasks in Project Plan*	Deliverable	QA Test Lead	6/4/2012
	QA Project Plan template
9	Test Planning Review Meeting (QARR Phase 1)	Deliverable /  Meeting	TPM or SDM Schedules	6/6/2012
			QA Test Lead Facilitates
10	FQT/Unit Testing Plan Review	Meeting	TPM or SDM Schedules	6/11/2012
	(QARR Phase 2)
11	Test Cases Finalized	Deliverable	QA Test Lead	6/18/2012
12	Quality Center Configuration Completed*	Deliverable*	TPM or SDM Schedules	6/18/2012
	(QARR Phase 3)		QA Test Lead Facilitates
13	FQT/Unit Test Results Review	Meeting	TPM or SDM Schedules	6/18/2012
14	DIT Checklist and DIT Results	Meeting	TPM or SDM Schedules	6/22/2012
15	Customer Walkthrough*	Meeting*	TPM or SDM Schedules	NA
			QA Test Lead Facilitates
16	Test Execution/Passed Dates & %s	Deliverable	QA Test Lead sets Execution Dates & %	6/25/2012
			TPM or SDM confirms Passed Dates & %
17	QA Test Execution Review*	Deliverable / Meeting*	TPM or SDM Schedules	8/10/2012
			QA Test Lead Facilitates
	* If Applicable QA Project Plan template consist: Return to Milestone Dates Milestone Dates Task # Days Start Date End Date Resources QA Testing Region Test Planning Integrated Test Strategy  — QA Test Workbook Integrated Test Strategy Signoff Function Testing Areas Covered  — QA Test Workbook Function Regression Function Testing Areas Covered Signoff Upload BRS to QC Test Case Creation Function Regression Upload Test Cases to QC Test Case Execution in QA region Execute daily/weekly QA Status Report (Project release report) Setup Daily Defect calls QA Test Execution Function Regression Milestone Test Dates 25% Test Cases Executed 50% Test Cases Executed, 25% Test Cases Passed 75% Test Cases Executed, 50% Test Cases Passed 100% Test Cases Executed, 75% Test Cases Passed 100% Test Cases Passed QA Test Sign off Training plan includes Project ID: Project Name: ISBA-QA Skills Assessment: http://teamsite/dept/ISQA/Docs/ISBA-QA%20Application%20Skills%20Assessment%20Results.xls On-Line Resource for help in Developing a Training Plan: http://managementhelp.org/trng_dev/gen_plan.htm Training Goal Learning Objectives Learning Methods / Activities Training Support Evaluation Overall results or capabilities you hope to attain by implementing your training plan. What you will be able to do as a result of the learning activities in this plan What you will do in order to achieve the learning objectives Identify the  Business Operations Resources,  Etc. Assessment and judgment on quality of evidence in order to conclude whether you achieved the learning objectives or not

Templates for Test Scenarios

Test Scenario includes:

QA Test Workbook

Table of Contents

Project ID & Name:

Integrated Test Strategy

Change Request

Function Testing Areas Covered

Milestones Dates

Training Plan

* If Applicable to the Project*

Required — Must be either Functional or Regression Testing.  Functional testing should link to Test Set Names in Quality Center.  Therefore it is possible to have multiple functional tabs in the QA Test Workbook and will be listed in the Table of Contents Section on this tab.

Conti.. Format of Test Steps…

Format of test steps

Each step can be written very tersely using the following keywords:

login [as ROLE-OR-USER]

Log into the system with a given user or a user of the given type. Usually only stated explicitly when the test case depends on the permissions of a particular role or involves a workflow between different users.

visit LOCATION

Visit a page or screen. For web applications, LOCATION may be a hyperlink. The location should be a well-known starting point (e.g., the Login screen), drilling down to specific pages should be part of the test.

enter FIELD-NAME [as VALUE] [in SCREEN-LOCATION]

Fill in a named form field. VALUE can be a literal value or the name of a variable defined in the “Test Data” section. The FIELD-NAME itself can be a variable name when the UI field for that value is clear from context, e.g., “enter password”.

enter FIELDS

Fill in all fields in a form when their values are clear from context or when their specific values are not important in this test case.

click “LINK-LABEL” [in SCREEN-LOCATION]

Follow a labeled link or press a button. The screen location can be a predefined panel name or English phrase. Predefined panel names are based on GUI class names, master template names, or titles of boxes on the page.

click BUTTON-NAME [in SCREEN-LOCATION]

Press a named button. This step should always be followed by a “see” step to check the results.

see SCREEN-OR-PAGE

The tester should see the named GUI screen or web page. The general correctness of the page should be testable based on the feature description.

verify CONDITION

The tester should see that the condition has been satisfied. This type of step usually follows a “see” step at the end of the test case.

verify CONTENT [is VALUE]

The tester should see the named content on the current page, the correct values should be clear from the test data, or given explicitly. This type of step usually follows a “see” step at the end of the test case.

perform TEST-CASE-NAME

This is like a subroutine call. The tester should perform all the steps of the named test case and then continue on to the next step of this test case.

Every test case must include a verify step at the end so that the expected output is very clear. A test case can have multiple verify steps in the middle or at the end. Having multiple verify steps can be useful if you want a smaller number of long tests rather than a large number of short tests.

Choosing Test Data

In system testing, test data should cover the possible values of each parameter based on the the requirements. Since testing every value is impractical, a few values should be chosen from each equivalence class. An equivalence class is a set of values that should all be treated the same.

Ideally, test cases that check error conditions are written separately from the functional test cases and should have steps to verify the error messages and logs. Realistically, if error test cases are not yet written, it is OK for testers to check for error conditions when performing normal functional test cases. It should be clear which test data, if any, is expected to trigger errors.

Example equivalence classes:

 

Strings

• empty string
• String consisting solely of white space
• String with leading or trailing white space
• syntactically legal: short and long values
• syntactically legal: semantically legal and illegal values
• syntactically illegal value: illegal characters or combinations
• Make sure to test special characters such as #, “, ‘, &, and <
• Make sure to test “Foreign” characters typed on international keyboards

 

Numbers

• empty string, if possible
• 0
• in range positive, small and large
• in range negative, small and large
• out of range positive
• out of range negative
• with leading zeros
• syntactically invalid (e.g., includes letters)

 

Identifiers

• empty string
• syntactically legal value
• syntactically legal: reference to existing ID, invalid reference
• syntactically illegal value

 

Radio buttons

• one item checked
• nothing checked, if possible

 

Checkboxes

• checked
• unchecked

 

Drop down menus

• select each item in turn

 

Scrolling Lists

• select no item, if possible
• select each item in turn
• select combinations of items, if possible
• select all items, if possible

 

File upload

• blank
• 0 byte file
• long file
• short file name
• long file name
• syntactically illegal file name, if possible (e.g., “File With Spaces.tar.gz”)

How to write effective Test cases, procedures and definitions

Writing effective test cases is a skill and that can be achieved by some experience and in-depth study of the application on which test cases are being written.

Some tips on how to write test cases, test case procedures and some basic test case definitions.

What is a test case?
“A test case has components that describes an input, action or event and an expected response, to determine if a feature of an application is working correctly.”

There are levels in which each test case will fall in order to avoid duplication efforts.
Level 1: In this level you will write the basic test cases from the available specification and user documentation.

Level 2: This is the practical stage in which writing test cases depend on actual functional and system flow of the application.
Level 3: This is the stage in which you will group some test cases and write a test procedure. Test procedure is nothing but a group of small test cases maximum of 10.
Level 4: Automation of the project. This will minimize human interaction with system and thus QA can focus on current updated functionalities to test rather than remaining busy with regression testing.

So you can observe a systematic growth from no testable item to a Automation suit.

Why we write test cases?
The basic objective of writing test cases is to validate the testing coverage of the application. If you are working in any CMMi company then you will strictly follow test cases standards. So writing test cases brings some sort of standardization and minimizes the ad-hoc approach in testing.

Fields in test cases:

Test case id:
Unit to test: What to be verified?
Assumptions:
Test data: Variables and their values
Steps to be executed:
Expected result:
Actual result:
Pass/Fail:
Comments:

So here is a basic format of test case statement:

Verify
Using [tool name, tag name, dialog, etc]
With [conditions]
To [what is returned, shown, demonstrated]

Verify: Used as the first word of the test case statement.
Using: To identify what is being tested. You can use ‘entering’ or ‘selecting’ here instead of using depending on the situation.

For any application basically you will cover all the types of test cases including functional, negative and boundary value test cases.

Keep in mind while writing test cases that all your test cases should be simple and easy to understand. Don’t write explanations like essays. Be to the point.

*How to write test cases?*

unique-test-case-id: Test Case Title

Purpose:	Short sentence or two about the aspect of the system is being tested. If this gets too long, break the test case up or put more information into the feature descriptions.
Prereq:	Assumptions that must be met before the test case can be run. E.g., “logged in”, “guest login allowed”, “user testuser exists”.
Test Data:	List of variables and their possible values used in the test case. You can list specific values or describe value ranges. The test case should be performed once for each combinationof values. These values are written in set notation, one per line. E.g.: loginID = {Valid loginID, invalid loginID, valid email, invalid email, empty} password = {valid, invalid, empty}
Steps:	Steps to carry out the test. See step formating rules below. visit LoginPage enter userID enter password click login see the terms of use page click agree radio button at page bottom click submit button see PersonalPage verify that welcome message is correct username
Notes and Questions:	NOTE QUESTION

Tips to design test data before executing your test cases..

“Tester should check and update the test data before execution of any test case.”

Tips on how to prepare test environment so that any important test case will not be missed by improper test data and incomplete test environment setup.

What do I mean by test data?

If you are writing test case then you need input data for any kind of test. Tester may provide this input data at the time of executing the test cases or application may pick the required input data from the predefined data locations. The test data may be any kind of input to application, any kind of file that is loaded by the application or entries read from the database tables. It may be in any format like xml test data, system test data, SQL test data or stress test data.

Preparing proper test data is part of the test setup. Generally testers call it as testbed preparation. In testbed all software and hardware requirements are set using the predefined data values.

If you don’t have the systematic approach for building test data while writing and executing test cases then there are chances of missing some important test cases. Tester can’t justify any bug saying that test data was not available or was incomplete. It’s every testers responsibility to create his/her own test data according to testing needs. Don’t even rely on the test data created by other tester or standard production test data, which might not have been updated for months! Always create fresh set of your own test data according to your test needs.

Sometime it’s not possible to create complete new set of test data for each and every build. In such cases you can use standard production data. But remember to add/insert your own data sets in this available database. One good way to design test data is use the existing sample test data or testbed and append your new test case data each time you get same module for testing. This way you can build comprehensive data set.

How to keep your data intact for any test environment?

Many times more than one tester is responsible for testing some builds. In this case more than one tester will be having access to common test data and each tester will try to manipulate that common data according to his/her own needs. Best way to keep your valuable input data collection intact is to keep personal copies of the same data. It may be of any format like inputs to be provided to the application, input files such as word file, excel file or other photo files.

Check if your data is not corrupted:
Filing a bug without proper troubleshooting  is bad a practice. Before executing any test case on existing data make sure that data is not corrupted and application can read the data source.

How to prepare data considering performance test cases?

Performance tests require very large data set. Particularly if application fetching or updating data from DB tables then large data volume play important role while testing such application for performance. Sometimes creating data manually will not detect some subtle bugs that may only be caught by actual data created by application under test. If you want real time data, which is impossible to create manually, then ask your manager to make it available from live environment.

I generally ask to my manager if he can make live environment data available for testing. This data will be useful to ensure smooth functioning of application for all valid inputs.

Take example of my search engine project ‘statistics testing’. To check history of user searches and clicks on advertiser campaigns large data was processed for several years which was practically impossible to manipulate manually for several dates spread over many years. So there is no other option than using live server data backup for testing. (But first make sure your client is allowing you to use this data)

What is the ideal test data?

Test data can be said to be ideal if for the minimum size of data set all the application errors get identified. Try to prepare test data that will incorporate all application functionality, but not exceeding cost and time constraint for preparing test data and running tests.

How to prepare test data that will ensure complete test coverage?

Design your test data considering following categories:
Test data set examples:
1) No data: Run your test cases on blank or default data. See if proper error messages are generated.

2) Valid data set: Create it to check if application is functioning as per requirements and valid input data is properly saved in database or files.

3) Invalid data set: Prepare invalid data set to check application behavior for negative values, alphanumeric string inputs.

4) Illegal data format: Make one data set of illegal data format. System should not accept data in invalid or illegal format. Also check proper error messages are generated.

5) Boundary Condition data set: Data set containing out of range data. Identify application boundary cases and prepare data set that will cover lower as well as upper boundary conditions.

6) Data set for performance, load and stress testing: This data set should be large in volume.

This way creating separate data sets for each test condition will ensure complete test coverage.

Conclusion:

Preparing proper test data is a core part of “project test environment setup”. Tester cannot pass the bug responsibility saying that complete data was not available for testing. Tester should create his/her own test data additional to the existing standard production data. Your test data set should be ideal in terms of cost and time. Use the tips provided in this article to categorize test data to ensure complete functional test cases coverage.

Be creative, use your own skill and judgments to create different data sets instead of relying on standard production data while testing.

Details required for Test Plan

Before that let us know What is test plan?

A software project test plan is a document that describes the objectives, scope, approach, and focus of a software testing effort. The process of preparing a test plan is a useful way to think through the efforts needed to validate the acceptability of a software product. The completed document will help people outside the test group understand the ‘why’ and ‘how’ of product validation. It should be thorough enough to be useful but not so overly detailed that no one outside the test group will read it. The following are some of the items that might be included in a test plan, depending on the particular project:

• Title
• Identification of software including version/release numbers
• Revision history of document including authors, dates, approvals
• Table of Contents
• Purpose of document, intended audience
• Objective of testing effort
• Software product overview
• Relevant related document list, such as requirements, design documents, other test plans, etc.
• Relevant standards or legal requirements
• Traceability requirements
• Relevant naming conventions and identifier conventions
• Overall software project organization and personnel/contact-info/responsibilities
• Test organization and personnel/contact-info/responsibilities
• Assumptions and dependencies
• Project risk analysis
• Testing priorities and focus
• Scope and limitations of testing
• Test outline – a decomposition of the test approach by test type, feature, functionality, process, system, module, etc. as applicable
• Outline of data input equivalence classes, boundary value analysis, error classes
• Test environment – hardware, operating systems, other required software, data configurations, interfaces to other systems
• Test environment validity analysis – differences between the test and production systems and their impact on test validity.
• Test environment setup and configuration issues
• Software migration processes
• Software CM processes
• Test data setup requirements
• Database setup requirements
• Outline of system-logging/error-logging/other capabilities, and tools such as screen capture software, that will be used to help describe and report bugs
• Discussion of any specialized software or hardware tools that will be used by testers to help track the cause or source of bugs
• Test automation – justification and overview
• Test tools to be used, including versions, patches, etc.
• Test script/test code maintenance processes and version control
• Problem tracking and resolution – tools and processes
• Project test metrics to be used
• Reporting requirements and testing deliverables
• Software entrance and exit criteria
• Initial sanity testing period and criteria
• Test suspension and restart criteria
• Personnel allocation
• Personnel pre-training needs
• Test site/location
• Outside test organizations to be utilized and their purpose, responsibilties, deliverables, contact persons, and coordination issues
• Relevant proprietary, classified, security, and licensing issues.
• Open issues
• Appendix – glossary, acronyms, etc.

 

 

 

 

 

 

 

 

 

 

 

 

 

Test Plan

 

 

Version 1.0

 

 

 

 

 

 

Revision History

 

DATE	Version	AUTHOR	DESCRIPTION
31/05/12	1.0	abhinay	Sample Test Plan

 

 

 

 

 

Table of Contents

 

1.    Introduction……………………………………………………………………………………………………… 4

1.1.     Test Plan Objectives…………………………………………………………………………………….. 4

2.    Scope…………………………………………………………………………………………………………….. 4

2.1.     Data Entry…………………………………………………………………………………………………. 4

2.2.     ReportsFile Transfer…………………………………………………………………………………….. 4

2.3.     File Transfer……………………………………………………………………………………………….. 4

2.4.  Security……………………………………………………………………………………………………….. 4

3.    Test Strategy……………………………………………………………………………………………………. 5

3.1. System Test………………………………………………………………………………………………….. 5

3.2. Performance Test…………………………………………………………………………………………… 5

3.3. Security Test…………………………………………………………………………………………………. 5

3.4. Automated Test……………………………………………………………………………………………… 5

3.5. Stress and Volume Test…………………………………………………………………………………… 5

3.6. Recovery Test……………………………………………………………………………………………….. 5

3.7. Documentation Test………………………………………………………………………………………… 5

3.8. Beta Test………………………………………………………………………………………………………. 5

3.9. User Acceptance Test……………………………………………………………………………………… 5

4.    Environment Requirements………………………………………………………………………………….. 6

4.1. Data Entry workstations…………………………………………………………………………………… 6

4.2 MainFrame…………………………………………………………………………………………………….. 6

5.    Test Schedule…………………………………………………………………………………………………… 6

6.    Control Procedures……………………………………………………………………………………………. 6

6.1 Reviews………………………………………………………………………………………………………… 6

6.2 Bug Review meetings……………………………………………………………………………………….. 6

6.3 Change Request……………………………………………………………………………………………… 6

6.4 Defect Reporting…………………………………………………………………………………………….. 6

7.    Functions To Be Tested……………………………………………………………………………………… 7

8.  Resources and Responsibilities……………………………………………………………………………… 7

8.1. Resources…………………………………………………………………………………………………….. 7

8.2.     Responsibilities…………………………………………………………………………………………… 7

9.    Deliverables…………………………………………………………………………………………………….. 8

10.  Suspension / Exit Criteria…………………………………………………………………………………… 9

11.  Resumption Criteria……………………………………………………………………………………………. 9

12.  Dependencies………………………………………………………………………………………………….. 9

12.1    Personnel Dependencies………………………………………………………………………………. 9

12.2    Software Dependencies………………………………………………………………………………… 9

12.3    Hardware Dependancies……………………………………………………………………………….. 9

12.3    Test Data & Database………………………………………………………………………………….. 9

13.  Risks………………………………………………………………………………………………………………. 9

13.1. Schedule…………………………………………………………………………………………………….. 9

13.2. Technical…………………………………………………………………………………………………….. 9

13.3. Management………………………………………………………………………………………………… 9

13.4. Personnel………………………………………………………………………………………………….. 10

13.5 Requirements………………………………………………………………………………………………. 10

14. Tools……………………………………………………………………………………………………………… 10

15.  Documentation……………………………………………………………………………………………….. 10

16. Approvals……………………………………………………………………………………………………….. 10

 

 

 

 

 

 

1.   Introduction

 

The company has outgrown its current payroll system & is developing a new system that will allow for further growth and provide additional features. The software test department has been tasked with testing the new system.

 

The new system will do the following:

• §  Provide the users with menus, directions & error messages to direct him/her on the various options.
• §  Handle the update/addition of employee information.
• §  Print various reports.
• §  Create a payroll file and transfer the file to the mainframe.
• §  Run on the Banyan Vines Network using IBM compatible PCs as data entry terminals

1.1.  Test Plan Objectives

This Test Plan for the new Payroll System supports the following objectives:

• §  Define the activities required to prepare for and conduct System, Beta and User Acceptance testing.
• §  Communicate to all responsible parties the System Test strategy.
• §  Define deliverables and responsible parties.
• §  Communicate to all responsible parties the various Dependencies and Risks

 

2.   Scope

2.1.   Data Entry

The new payroll system should allow the payroll clerks to enter employee information from IBM compatible PC workstations running DOS 3.3 or higher. The system will be menu driven and will provide error messages to help direct the clerks through various options.

2.2.    Reports

The system will allow the payroll clerks to print 3 types of reports. These reports are:

• §  A pay period transaction report
• §  A pay period exception report
• §  A three month history report

2.3.    File Transfer

Once the employee information is entered into the LAN database, the payroll system will allow the clerk to create a payroll file. This file can then be transferred, over the network, to the mainframe.

2.4. Security

Each payroll clerk will need a user id and password to login to the system. The system will require the clerks to change the password every 30 days.

 

 

 

3.        Test Strategy

The test strategy consists of a series of different tests that will fully exercise the payroll system. The primary purpose of these tests is to uncover the systems limitations and measure its full capabilities. A list of the various planned tests and a brief explanation follows below.

3.1. System Test

The System tests will focus on the behavior of the payroll system. User scenarios will be executed against the system as well as screen mapping and error message testing. Overall, the system tests will test the integrated system and verify that it meets the requirements defined in the requirements document.

3.2. Performance Test

Performance test will be conducted to ensure that the payroll system’s response time meet the user expectations and does not exceed the specified performance criteria. During these tests, response times will be measured under heavy stress and/or volume.

3.3. Security Test

Security tests will determine how secure the new payroll system is. The tests will verify that unauthorized user access to confidential data is prevented.

3.4. Automated Test

A suite of automated tests will be developed to test the basic functionality of the payroll system and perform regression testing on areas of the systems that previously had critical/major defects. The tool will also assist us by executing user scenarios thereby emulating several users.

 

3.5. Stress and Volume Test 

We will subject the payroll system to high input conditions and a high volume of data during the peak times. The System will be stress tested using twice (20 users) the number of expected users.

3.6. Recovery Test   

Recovery tests will force the system to fail in a various ways and verify the recovery is properly performed. It is vitally important that all payroll data is recovered after a system failure & no corruption of the data occurred.          

3.7. Documentation Test  

Tests will be conducted to check the accuracy of the user documentation. These tests will ensure that no features are missing, and the contents can be easily understood.

3.8. Beta Test

The Payroll department will beta tests the new payroll system and will report any defects they find. This will subject the system to tests that could not be performed in our test environment.

3.9. User Acceptance Test

Once the payroll system is ready for implementation, the Payroll department will perform User Acceptance Testing. The purpose of these tests is to confirm that the system is developed according to the specified user requirements and is ready for operational use.

 

 

4.   Environment Requirements

4.1. Data Entry workstations

• §  20 IBM compatible PCs (10 will be used by the automation tool to emulate payroll clerks).
• §  286 processor (minimum)
• §  4mb RAM
• §  100 mb Hard Drive
• §  DOS 3.3 or higher
• §  Attached to Banyan Vines network
• §  A Network attached printer
• §  20 user ids and passwords (10 will be used by the automation tool to emulate payroll clerks).

 

4.2 MainFrame

• §  Attached to the Banyan Vines network
• §  Access to a test database (to store payroll information transferred from LAN payroll system)

 

5.   Test Schedule

• §  Ramp up / System familiarization                             01/06/12            –           15/06/12
• §  System Test                                                                                                16/06/12            –           26/06/12
• §  Beta Test                                                                                                    28/06/12            –           08/07/12           
• §  User Acceptance Test                                                                      09/07/12            –           09/07/12

 

6.   Control Procedures

6.1 Reviews

The project team will perform reviews for each Phase. (i.e. Requirements Review, Design Review, Code Review, Test Plan Review, Test Case Review and Final Test Summary Review). A meeting notice, with related documents, will be emailed to each participant.

6.2 Bug Review meetings

Regular weekly meeting will be held to discuss reported defects. The development department will provide status/updates on all defects reported and the test department will provide addition defect information if needed. All member of the project team will participate.

6.3 Change Request

Once testing begins, changes to the payroll system are discouraged. If functional changes are required, these proposed changes will be discussed with the Change Control Board (CCB). The CCB will determine the impact of the change and if/when it should be implemented.

6.4 Defect Reporting

When defects are found, the testers will complete a defect report on the defect tracking system. The defect tracking       Systems is accessible by testers, developers & all members of the project team. When a defect has been fixed or more information is needed, the developer will change the status of the defect to indicate the current state. Once a defect is verified as FIXED by the testers, the testers will close the defect report.

           

 

 

7.   Functions to Be Tested

The following is a list of functions that will be tested:

 

• §  Add/update employee information
• §  Search / Lookup employee information
• §  Escape to return to Main Menu
• §  Security features
• §  Scaling to 700 employee records
• §  Error messages
• §  Report Printing
• §  Creation of payroll file
• §  Transfer of payroll file to the mainframe
• §  Screen mappings (GUI flow). Includes default settings
• §  FICA Calculation
• §  State Tax Calculation
• §  Federal Tax Calculation
• §  Gross pay Calculation
• §  Net pay Calculation
• §  Sick Leave Balance Calculation
• §  Annual Leave Balance Calculation

 

A Requirements Validation Matrix will “map” the test cases back to the requirements. See Deliverables.

8.  Resources and Responsibilities

The Test Lead and Project Manager will determine when system test will start and end. The Test lead will also be responsible for coordinating schedules, equipment, & tools for the testers as well as writing/updating the Test Plan, Weekly Test Status reports and Final Test Summary report. The testers will be responsible for writing the test cases and executing the tests. With the help of the Test Lead, the Payroll Department Manager and Payroll clerks will be responsible for the Beta and User Acceptance tests.

8.1. Resources

The test team will consist of:

• §  A Project Manager
• §  A Test Lead
• §  5 Testers
• §  The Payroll Department Manager
• §  5 Payroll Clerks

 

8.2.  Responsibilities

Project Manager	Responsible for Project schedules and the overall success of the project. Participate on CCB.
Lead Developer	Serve as a primary contact/liaison between the development department and the project team.Participate on CCB.
Test Lead	Ensures the overall success of the test cycles. He/she will coordinate weekly meetings and will communicate the testing status to the project team.Participate on CCB.
Testers	Responsible for performing the actual system testing.
Payroll Department Manager	Serves as Liaison between Payroll department and project team. He/she will help coordinate the Beta and User Acceptance testing efforts. Participate on CCB.
Payroll Clerks	Will assist in performing the Beta and User Acceptance testing.

 

9.     Deliverables

Deliverable	Responsibility	Completion Date
Develop Test cases	Testers	6/11/12
Test Case Review	Test Lead, Dev. Lead, Testers	6/12/12
Develop Automated test suites	Testers	7/01/12
Requirements Validation Matrix	Test Lead	6/16/12
Obtain User ids and Passwords for payroll system/database	Test Lead	5/27/12
Execute manual and automated tests	Testers & Test Lead	8/26/12
Complete Defect Reports	Everyone testing the product	On-going
Document and communicate test status/coverage	Test Lead	Weekly
Execute Beta tests	Payroll Department Clerks	8/18/12
Document and communicate Beta test status/coverage	Payroll Department Manager	8/18/12
Execute User Acceptance tests	Payroll Department Clerks	9/03/12
Document and communicate Acceptance test status/coverage	Payroll Department Manager	9/03/12
Final Test Summary Report	Test Lead	9/05/12

 

 

 

10. Suspension / Exit Criteria

If any defects are found which seriously impact the test progress, the QA manager may choose to

Suspend testing. Criteria that will justify test suspension are:

• §  Hardware/software is not available at the times indicated in the project schedule.
• §  Source code contains one or more critical defects, which seriously prevents or limits testing progress.
• §  Assigned test resources are not available when needed by the test team.

11.   Resumption Criteria

If testing is suspended, resumption will only occur when the problem(s) that caused the suspension has been resolved. When a critical defect is the cause of the suspension, the “FIX” must be verified by the test department before testing is resumed.

12.   Dependencies

12.1  Personnel Dependencies

The test team requires experience testers to develop, perform and validate tests. These

 

The test team will also need the following resources available: Application developers and Payroll Clerks.

12.2  Software Dependencies

The source code must be unit tested and provided within the scheduled time outlined in the Project Schedule.

12.3 Hardware Dependencies

The Mainframe, 10 PCs (with specified hardware/software) as well as the LAN environment need to be available during normal working hours. Any downtime will affect the test schedule.

12.3  Test Data & Database

Test data (mock employee information) & database should also be made available to the testers for use during testing.

13.   Risks

13.1. Schedule

The schedule for each phase is very aggressive and could affect testing. A slip in the schedule in one of the other phases could result in a subsequent slip in the test phase. Close project management is crucial to meeting the forecasted completion date.

13.2. Technical

Since this is a new payroll system, in the event of a failure the old system can be used. We will run our test in parallel with the production system so that there is no downtime of the current system. 

13.3. Management

            Management support is required so when the project falls behind, the test schedule does not

get squeezed to make up for the delay. Management can reduce the risk of delays by supporting the test team throughout the testing phase and assigning people to this project with the required skills set.

 

 

 

 

13.4. Personnel

Due to the aggressive schedule, it is very important to have experienced testers on this project. Unexpected turnovers can impact the schedule. If attrition does happen, all efforts must be made to replace the experienced individual

13.5 Requirements

The test plan and test schedule are based on the current Requirements Document. Any changes to the requirements could affect the test schedule and will need to be approved by the CCB.

14. Tools

The Acme Automated test tool will be used to help test the new payroll system. We have the licensed product onsite and installed. All of the testers have been trained on the use of this test tool.

15. Documentation

The following documentation will be available at the end of the test phase:

• §  Test Plan
• §  Test Cases
• §  Test Case review
• §  Requirements Validation Matrix
• §  Defect reports
• §  Final Test Summary Report

 

 

 

16. Approvals

Name (Print)	Signature	Date
1.
2.
3.
4.
5.

 

Articles with supported links for: Top Resources, Software QA and Testing-related Organizations/Certifications/Conferences, Certification Information for Software QA and Test Engineers, Software QA and Testing Conferences, QA and Testing-related Magazines/Publications, General Software QA and Testing Resources, Agile and XP Testing Resources, Test Automation Resources, Mobile Testing Resources, Web QA and Testing Resources, Web Security Testing Resources, Web Usability Resources…

Top Resources

Stickyminds.com – Comprehensive software testing resource site associated with ‘Better Software’ Magazine, with articles, news, information on software testing and quality engineering, books, tools, conferences, message boards.

Cem Kaner’s software testing site – Cem Kaner’s site contains a large selection of his articles about software testing, legal issues, test management, and more (see the ‘Publications’ section of the site). Also see his ‘badsoftware.com’ website , a consumer and legal-issues orientation to software quality issues.

IT Metrics and Productivity Institute Archives – Large collection of software engineering articles by various authors; most of the articles are related to software QA; site maintained by Computer Aid, Inc. Articles cover such subjects as project management, process, metrics, estimation, software maintenance, IT governance, etc.

Software QA and Testing-related Organizations/Certifications/Conferences

SEI – Software Engineering Institute web site; info about SEI technical programs, publications, bibliographies, some online documents, SEI courses and training, links to related sites.

EOQ-SG – European Organization for Quality – Software Group, an independent not-for-profit organization founded in 1983. It is comprised of more than 30 national quality organizations and other institutions, enterprises and specialists.

American Society for Quality – American Society for Quality (formerly the American Society for Quality Control) web site; geared to quality issues in general, not just Software QA. ASQ is the largest quality organization in the world, with more than 100,000 members. Provides a wide variety of general quality-related certifications, as well as the CSQE (Certified Software Quality Engineer).

Association for Software Testing – A nonprofit professional organization dedicated to advancing the understanding and practice of software testing. For scholars, students, and software development practitioners. Sponsors conferences, publications, web sites, newsletter.

QAI Global Institute – QAI is a global workforce development and consulting organization addressing the education and ‘Operational Excellence’ needs of information technology, information technology enabled organizations and knowledge intensive organizations.
Information on associated QAI local chapters around the world can be found via their QAI Chapters site.

SPIN – Software Process Improvement Network, for those interested in improving software engineering practices. Organized into regional groups called “SPINs” that meet and share their experiences initiating and sustaining software process improvement programs. Annual meeting at the Software Engineering Process Group (SEPG) Conference, which is co-sponsored by the SEI and a regional SPIN. Web site lists links to regional SPINs worldwide.

Society for Software Quality – Has chapters in San Diego and Washington DC area; each with monthly meetings.

Northern Virginia Test Automation Interest Group – Monthly meetings on software test automation in the Washington DC area.

SQuAD – Software Quality Association of Denver – software QA monthly meetings and an annual conference, in Denver, Colorado.

TCQAA – Twin Cities Quality Assurance Association of Minneapolis and St. Paul, Minnesota. Has monthly meetings.

Southern California Quality Assurance Association – Software QA organization with chapters in the U.S. in the Los Angeles area including in Los Angeles, Orange County, San Fernando Valley, and Inland Empire.

SQSQAG – Seattle Area Software Quality Assurance Group

VANQ – Vancouver Canada area association for software quality assurance and testing.

KWSQA – Software QA/Testing group with monthly meetings in Kitchener-Waterloo region (Ontario, Canada).

Certification Information for Software QA and Test Engineers:

ISTQB Certified Tester – The International Software Testing Qualifications Board, based in Belgium, was initially a part of the European Organization for Quality – Software Group. Certifications and testing are administered by ISTQB organizations in each of a number of countries around the world. Multiple certification levels are available, including: Foundation, Advanced, and Expert.

CSQE – ASQ (American Society for Quality) CSQE (Certified Software Quality Engineer) program – information on requirements, outline of required ‘Body of Knowledge’, listing of study references and more.

ISEB Software Testing Certifications – The British Computer Society Information Systems Examinations Board (ISEB) maintains a program of several levels of testing certifications.

ASTQB – The American Software Testing Qualifications Board is a branch of the ISTQB Certifications are based on experience and a written test. Two primary levels are available: Foundation and Advanced. The ‘Advanced’ level has multiple types of certifications.

CSQA/CSTE – QAI Global Institute’s program for CSQA (Certified Software Quality Analyst), CSTE (Certified Software Tester), and Certified Software Project Manager (CSPM), Certified Associate in Software Quality (CASQ), and many other certifications.

Software QA and Testing Conferences:

Conferences Listing – North American and International conferences listing maintained by Zephyr/D Software Inc.

Conferences Listing – Danny Faught’s Conferences Listing page (worldwide).

Conference/Events Listing – Worldwide testing conferences listing along with other testing-related events, seminars, etc at Test Events web site.

QA and Testing-related Magazines/Publications

Tea Time With Testers Magazine – Free online testing magazine available as PDF file; web site includes back issues.

Professional Tester Magazine – Bi-monthly testing publication from Professional Tester Inc. Free access to PDF version, archive, individual articles and news and events agenda.

STQA Magazine – Software Test and QA Magazine from BZ Media. No access to articles without registering on site.

Software Quality Professional Magazine – Published by the American Society for Quality; web site includes table of contents and abstracts of all articles, and full text of selected articles.

Better Software Magazine – Web site has full text of each print issue’s featured article. Published by Software Quality Engineering, Inc.

IT Metrics and Productivity Journal – Free on-line publication from Computer Aid, Inc. covering best practices in software development, maintenance, and management, process, metrics, estimation, IT governance, etc. See site’s ‘archive’ link for large collection of past articles.

Methods and Tools – Software Methods and Tools e-newsletter web site by Martinig and Associates; regular articles are included on process improvement, testing, modeling, management, etc. Site includes current issues and past issues with full text of all articles; as well as extensive additional information and resources.

General Software QA and Testing Resources

(Note: also see the ‘Books’ section for a listing of books on Software QA, Testing, and related subjects.)

Thoughtworks Testing Portal – Testing articles and blogs from Thoughtworks.

Testing Reflections – Site run by Antony Marcano that aggregates software testing-related blog posts from a wide variety of sources.

Google testers’ blog – Public blog site for Google’s testers. (Although the site does not seem to have been updated or added to since late 2011, there are many useful blog posts.)

The Value of Checklists and the Danger of Scripts – Presentation at CAST 2008 by Cem Kaner. Summary of the presentation available from his blog, and a link to the pdf file of the presentation.

Software Engineering: An Idea Whose Time Has Come and Gone? – An interesting 2009 article from ‘IEEE Software’ by Tom DeMarco in which he indicates, among other things, that his early ideas and advice regarding quantified work, project planning, and metrics for software projects – such as those in his 1982 book ‘Controlling Software Projects: Management, Measurement, and Estimation’, may have been wrong.

Exploring Exploratory Testing – Article by Cem Kaner and Andy Tinkham from 2003 about the exploratory testing approach to software testing; includes discussions of questioning strategies and heuristics.

Exploratory Testing Explained – Article by James Bach on exploratory testing; includes attributes of a software project and tester that impact decisions on testing approaches, exploratory testing examples, etc.

Pass vs. Fail vs. Is There a Problem Here – Interesting blog article on testing vs checking and the issues in ‘pass/fail’ type testing and reporting, from 2009 in Michael Bolton’s DevelopSense blog.

How to write near-perfect software – An old but still highly relevant long article in Fast Company about how software was developed for the U.S. Space Shuttle. “The group’s most important creation is not the perfect software they write — it’s the process they invented that writes the perfect software.”

G. Weinberg’s Collection of QA Articles – Jerry Weinberg’s site has a section with a collection of some of his QA-related articles; site also includes sections with articles on thinking, leadership, and software projects, and more.

Tom Gilb Web Site – Site includes a large collection of articles, papers, course slides, etc concerning risk, metrics, and other QA-related software engineering topics.

Software Negligence and Testing Coverage – Article by Cem Kaner contains an old but still very informative list of 101 types of testing coverage measures; shows the complexities in any discussion of ‘testing coverage’. Selected quotes of interest from the article: “Even if you achieve complete coverage for a given population of tests (such as, all lines of code tested), you have not done complete, or even adequate, testing.” and “The decision as to whether to try for 1%, 10%, 50% or 100% coverage against any given population is non-obvious. It involves tradeoffs based on thoughtful judgment.”

What is a Test Architect? – Discussion re test architects at the blog site of Microsoft’s Alan Page.

RBCS Testing articles – RBCS Consulting Services web site’s collection of software testing articles on a wide variety of testing-related subjects.

STORM – Software Testing Online Resources/MTSU – a well-organized site with listings of many links to software QA and testing-related web sites.

Comp.software.testing Usenet News Group – Via Google Groups web site (formerly the Deja News site), can be used to search through past postings; postings go back to 1995.

Errors in Scientific Software – Article titled ‘The T experiments: errors in scientific software’ by Les Hatton; old but still alarming article from 1997.

Software Certifications and Standards impacting Embedded Software – Article from Dr Dobb’s Portal from September 2006, discussing standards such as DO-178B “Software Considerations in Airborne Systems and Equipment Certification”, IEC 61508 for safety-related systems, FIPS 140-2 re “Security Requirements for Cryptographic Modules”, HIPPAA, etc.

Certifications – Cem Kaner’s perspective on software testing certifications from a 2007 article – includes a discussion of a proposed ‘Open Certification Process’; section 3 of the paper has a long discussion of “Project Manager’s Perspective: Problems With the Current Certification System”

‘Good programmer’ definitions/discussions – Since testers and developers often need to work closely together, and since many testers also do some programming, it is helpful to get some perspective on ‘what is a good programmer’. Also see the discussion in the old Joel on Software forum..

WSR Consulting Group publications – Good collection of QA and Testing related articles from WSR consulting, a computer crisis/litigation consulting company. The articles have an emphasis on proper management of problem projects and engineering-customer relationships for software projects.

Practical Software and Systems Measurement – Web site with extensive information on software development metrics, sponsored by U.S. government. Site contains articles, reports, examples, and a free PC-based software tool to assiste in project-specific metrics development.

Software Estimation – December 2005 interview about software metrics and estimation from the IT Metrics and Productivity Journal .

Software Project Estimation – Good introductory article covering the basic issues of software project estimation, from the Software Productivity Centre in Vancouver.

Testing Education Articles – Collection of articles on software testing and the teaching of software testing from the Florida Institute of Technology funded by the U.S. National Science Foundation.

‘Software Experts’ site – Software engineering site oriented to microcontroller/embedded system environments, by Eberhard De Wille and Dana Vede. Site has sections on design, coding, refactoring, process, and a large section on testing.

ITIL – ‘IT Infrastructure Library’ – a set of best-practices guides on the management and provision of operational IT Services. From the British Office of Gevernment Commerce and the itSMF, the ‘IT Service Management Forum’ a UK-based organization comprised of 1000 companies and government organizations worldwide. There is a related ISO/IEC 20000 Standard against which organizations can be assesses and certified. An online organizational ITIL Service Management Self Assessment is available. There is a good summary of the ITIL approach in Wikipedia.

Big Ball of Mud – Outstanding essay on the ‘de-facto standard software architecture’, by Briane Foote and Joseph Yoder of the U. of Illinois at Urbana-Champaign. The ‘Big Ball of Mud’ architecture is defined as ‘a casually, even haphazardly, structured system. Its organization, if one can call it that, is dictated more by expediency than design….The overall structure of the system may never have been well defined. If it was, it may have eroded beyond recognition.’ They discuss why this architecture is so popular, advantages and disadvantages, and what can be done to improve such systems.

The Frameworks Quagmire – Old but still relevant article by Sarah A. Sheard summarizing and exploring the conflicts and complexities among various software process standards existing in the late 1990’s – CMMi, SW-CMM, SDCE, Trillium, IEEE, FAA-iCMM, EIA-632, NATO-AQAP, MIL-STD-498, ISO/IEC-12207, ISO-9000, etc.

Satisfice.com Web Site – James Bach’s Satisfice.com Web Site with a great collection of his articles on various aspects of software testing.

DevelopSense Web Site – Web site of Michael Bolton, who collaborates with James Bach, has a large collection of articles and blog postings with interesting perspectives on software testing.

Bret Pettichord’s Web Site – Web site of Bret Pettichord with articles and links to various test and QA-related info including his Watir open source web testing framework.

BetaSoft Web Site – Wide variety of QA, testing, and automated testing discussion forums, sections for jobs and resumes, other resources.

SQATester.com – QA and Testing information, discussion forums, other resources.

Search Software Quality – TechTarget’s software QA and testing site with a collection of articles, tutorials, blogs, and news.

Illustrative Risks to the Public in the Use of Computer Systems – Enormous list of software, system, and related problems compiled by Peter Neumann/SRI International. Organized by categories such as space, defense, medical, stock market, elections, insurance, cryptography, etc. Includes related book list, other information. (Also see ‘Risks Digest’ listed below.)

Process Improvement Case Study Featuring Reviews and Inspections – Article titled ‘Process Improvement: Case Study of an Improvement Program Featuring Reviews and Inspections’ in Software Quality Professional magazine.

Will Bugs Eat Up the U.S. Lead in Software? – Bloomberg/Business Week (International Edition) article comparing present state of U.S. software industry to U.S. automobile industry in the 1970’s, when Japan took away huge market share with better and cheaper products by adopting Deming’s and Juran’s quality approaches. Discusses a possible repeat with the current US software industry losing software development market share to countries such as India. Entire issue and cover story reports on software problems – discussion of past problems, problems with software engineering, and possible fixes. Also see similar Infoweek article – ‘The Big Picture: Killer Apps And Dead Bodies’.

ARIANE 5 Flight 501 Failure Report by the Inquiry Board – A rare and instructive detailed public analysis of a major software failure – the 1996 launch failure of the new Ariane 5 rocket. This is the official report of the inquiry board appointed by the French National Center for Space Studies and the European Space Agency. Also see the article ‘Design by Contract: The Lessons of Ariane’ which includes a discussion of the code reuse issues brought to light by the Ariane 5 failure.

Eiffel FAQ – FAQ site for a programming approach, based on the ideas of Bertrand Meyer, with the goal of improving software component reusability, extendibility and reliability using assertions, preconditions, and postconditions.

Internationalization Testing – Article at Oracle/Sun’s web site; includes guidelines and good check list of questions for testing an internationalized software product.

Risks Digest – Digest of the ‘Forum on Risks to the Public in Computers and Related Systems’. Includes latest issue and archives covering software and system problems, vulnerabilities, disasters; based on the comp.risks newsgroup.

SEI Capability Maturity Models – SEI’s CMMI web site, with info and documentation downloads on the CMMI for Development and the CMMI for Acquisition models.

Construx Software Resources – Site with many useful resources such as CxOne, a lightweight, tailorable, modular, and scalable software engineering framework, estimation info and resources, various checklists, and Steve McConnell’s ‘Software Survival Guide’ website.

CM FAQ – Configuration Management FAQ edited by David Eaton; includes ‘What is CM?’, ‘How should a CM system relate to process enforcement?’, CM books and other resources, etc. Not updated in recent years but still a useful FAQ.

SR/Institute’s Software Quality Hot List – Extensive collection of links to many QA and testing-related articles, resources, etc.

Uniform Computer Information Transactions Act (UCITA) – Text of controversial proposed U.S. laws (formerly Uniform Commercial Code Article 2B) concerning software quality. This would essentially implement new laws in all 50 states in the U.S. Additional info on UCITA controversies at Ralph Nader’s CPT (Consumer Project on Technology) web site and Cem Kaner’s BadSoftware.com web site. . As of 2010, only the states of Maryland and Virginia had enacted UCITA since it was first proposed in 1999. In 2009 the American Law Institute proposed an alternative version for software contracts but it resulted in similar controversy to UCITA.

FDA Medical Device Software Validation Guidelines – U.S. Food and Drug Administration’s 2002 guidelines for medical device software validation.

Negotiating Testing Resources – Excellent article by Cem Kaner about testing project planning and budgeting; from a 1996 software quality conference – old but still relevant.

Software Engineering Resources – Large collection of useful information and links to many other sites and resources, all related to the SW engineering process including project planning and management, metrics, risk analysis, programming methods, OO SW engineering, testing, QA, CM. From R.S. Pressman, author of the book ‘Software Engineering, A Practitioner’s Approach’.

Software Test Coverage Analysis article – Article containing a good discussion of test coverage analysis from Bullseye Testing Technology, maker of “C-Cover Test Coverage Analyzer” tool.

Object-Oriented Concepts – Basics of object-oriented programming concepts, from Oracle’s (formerly Sun’s) Java site. Good quick intro.

CMMI (Capability Maturity Model Integration) – A suite of process improvement models for product and service development and maintenance. The suite includes the CMMI-SW model, and there is a ‘staged’ and ‘continuous’ version. Each of the CMMI models can be coordinated with other CMMI models to enable enterprise-wide process improvement. CMMI-SW builds on the previous SW-CMM model which was ‘sunsetted’.

TMMi – The TMMi is being developed as a method for assessing and rating organizations’ testing process capabilities. The TMMi foundation was formed from the perspective that CMMi was mainly about software development and less about QA process. There is a TMMi Reference Model and a set of requirements for building assessment models.

The comp.software.testing FAQ – The comp.software.testing FAQ; maintained by Raymond Rivest; resource for testing-related conferences, mailing lists, books, periodicals, organizations, and links to other sites. Not maintained in recent years but still useful.

Agile and XP Testing Resources:

Manifesto for Agile Software Development – The origin of the ‘Agile’ approach and the twelve guiding principles of agile software development.

Agile Testing Articles – Large collection of articles related to Agile testing at Agile Alliance web site.

Agile Methodologies – Martin Fowler’s online discussion of ‘agile’ methodologies (XP, Scrum, Crystal, FDD, DSDM, etc.) includes summaries of various approaches as well as reference information, and factors to consider in choosing these approaches.

Perils and Pitfalls of Agile Adoption – Article by Matt Heuser at InformIT site, includes discussion of risks such as that agile methods are easy to misunderstand, that it’s easy to think you’re doing Agile right, and be wrong, and that agile methods make value (or lack of value) visible.

Agile Testing – What is it? Can it work? – PDF version of an article by Bret Pettichord that summarizes considerations and issues in testing in agile environments.

An Uncomfortable Truth about Agile Testing – Article by Jeff Patton on the StickyMinds site about some of the potential difficulties of testing on an Agile project.

XP Resources – Large collection of resources from Ron Jeffries about ‘Extreme Programming’ including a discussion of how QA fits into the XP approach, XP Magazine archives with articles such as ‘Test-First Design’, ‘Incremental Requirements’, ‘Extreme Programming and the CMM’, and more. Also see ‘The Rules and Practices of Extreme Programming ‘ at the www.extremeprogramming.org web site.

XP in a Safety-Critical Environment – Interesting article by Mary and Tom Poppendieck concerning the applicability of XP practices in safety-critical software development.

Scrum – web site of ScrumAlliance.org which describes the basics of the Scrum agile approach, a team-based agile approach to iteratively, incrementally develop software with rapidly changing requirements; has lots of articles and other resources.

 

Test Automation Resources:

Seven Steps to Test Automation Success – Good introductory article on how to approach automated testing; by Bret Pettichord.

NoVaTAIG Test Automation Resources – Test automation presentations, articles and resources from the Northern Virginia Test Automation Interest Group monthly meeting summaries (‘Recent past meetings’ link on site main page).

Architectures of Test Automation – Long article on test automation by Cem Kaner, includes discussions on GUI regression testing, maintainability, a classification scheme for test automation, an automation evaluation scheme, ‘test automation’ vs ‘computer assisted testing’ and more.

Test Automation Snake Oil – Old but still relevant article by James Bach about how to approach test automation.

Test Tools for Free – Short article by Danny Faught discussing some free test tools, along with the basics of ‘freeware’, and information on a testing freeware newsletter.
(Note: Many free web testing tools are also included among the web testing tools listed in the Softwareqatest.com Web Test Tools page. )

Effective Performance Testing articles – Extensive collection of how-to and other information on performance testing at Scott Barber’s web site.

Evaluating and Choosing the Right Tool – Elisabeth Hendrickson describes a five-step process for comparing, evaluating, and choosing the right test tool; from the Stickyminds.com web site.

JUnit.org – Site for test/development engineers using JUnit or one of the other XUnit testing frameworks. Has many useful articles and resources on automated Java regression testing and on ‘agile’ testing processes in general.

Java GUI Testing – Short discussion of automated Java GUI testing issues, includes interesting discussion of methods of identifying a component in a GUI hierarchy for use in developing automated test scripts.

See the ‘Tools’ section for test tool listings and the ‘Web Tools’ section for web site testing tools.

See the Softwareqatest.com Bookstore section on Automation for books on test automation.

 

Mobile Testing Resources:

Test Your Mobile Web Apps with WebDriver – From the Open Source at Google blog – discussion of how to write automated tests to test a site when viewed from an Android or iOS browser. The WebDriver web testing framework includes a touch API that allows a test to interact with a web page through finger taps, flicks, finger scrolls, and long presses. It can rotate the display and provides an API to interact with HTML5 features such as local storage, session storage and application cache.

Mobile Wireless Test Automation – Site with a collection of information by Julian Harty on practical experiences in automating aspects of software testing for mobile wireless applications; also has information on effective mobile manual testing. Includes testing techniques for iPhone, Android, SMS; and information on common tools, app testing over WiFi, more. Also has links to some presentations and tutorials and lists of other resources.

Mobile Application Security Testing – Whitepaper from Foundstone/McAfee about mobile app security testing.

Mobile App security Articles – Articles and whitepapers on mobile app security from Denim Group’s web site.

MobiForge – Testing – MobiForge mobile development community web site section on mobile software testing.

Mobile Application Testing blog – Mobile app testing blog by Anurag Khode

Mobile Web Development Resources – Mobile web development resources including some testing resources and standards resources, etc.

See the ‘Mobile Web/App Testing Tools’ section of the Web Test Tools List page for mobile testing tools.

 

Web QA and Testing Resources:

Why Load Testing Ajax is Hard – Article by Patrick Lightbody on the Ajaxian blog site from December 2008 on the challenges of load testing sites incorporating Ajax.

Web Site Testing Checklist – More of a web site development checklist, but it is extensive and still useful as a way to generate ideas for testing a web site.

Performance Testing Guidance for Web Applications – Online 18-chapter guide for an end-to-end approach for implementing web performance testing. Part of the Performance Testing Guidance Project web site. Covers: managing and conducting performance testing in both Agile and structured environments; load testing, stress testing, and other types of performance related testing; identifying objectives, designing tests, executing tests, analyzing results, and reporting. Published 2007 but still useful.

Web Site Performance Testing – A collection of useful information on various aspects of performance testing, from Scott Barber’s web site. Topics include: “Pinpointing and Exploiting Specific Performance Bottlenecks”, “Common Performance Testing Challenges”, “How Fast is Fast Enough”, and “Introduction to Performance Testing”. Although some of the information is not specifically oriented to web performance testing, it is still highly applicable.

Load Testing Of Web Sites – Article from IEEE Internet Computing about web load testing; useful overview from 2002.

Keynote Systems Resources Page – Useful collection of articles and information on web site performance testing. Also see the related site performance indices which lists a variety of business, consumer, government, and other web sites along with their ‘performance index’.

Evaluating Web Sites for Accessibility – Article on the World Wide Web Consortium web site’s ‘Web Accessibility Initiative’ section on how to assess and test web sites for accessibility issues.

Handling and Avoiding Web Page Errors – Three part series from Microsoft site; covers sources of common Web page errors, how to handle run-time script errors, and techniques for avoiding preventable errors. Old but still useful.

 

Web Security Testing Resources:

StopBadware.org – Web site security guidelines and information from the StopBadware site, based at Harvard University’s Berkman Center for Internet & Society.

OWASP – The Open Web Application Security Project (OWASP) is dedicated to finding and fighting the causes of insecure software. Everything available in site is free and open source. ‘How To’ section includes ‘Guide to Building Secure Web Applications and Web Services’, ‘Testing Guide’, ‘Code Review Guide’. Also security news, articles such as ‘How to Write Insecure Code’, tools, code, filters, downloads, and more.

Computer Audit FAQ – Good introductory information from IsecT Ltd. on ‘Computer Audit’, which refers to the analysis of computer systems and networks by examining the effectiveness of their technical and procedural controls (information security control systems) to minimise risks. Also has links to other resources, and some articles such as ‘Strategic Approach to Information Security Management’

SANS Security Resources – Web site of SANS (SysAdmin, Audit, Network, and Security Institute), a cooperative research and education organization for sysadmins, security professionals, and network administrators for sharing lessons learned and solutions. Includes and Instrusion Detection FAQ; more than 1500 white papers on security; webcasts; security trends, top security risks, and much more are freely available.

CVE – Searchable, downloadable, and on-the-web ‘Common Vulnerabilities and Exposures’ list hosted by Mitre Corp. CVE goal is to standardize the names for all publicly known vulnerabilities and security exposures, so that security information can be efficiently shared and handled. Many security test tools are utilizing or planning on utilizing this standardized naming/numbering system.

Common Attack Pattern and Enumeration – CAPEC is a publicly available, community-developed list of common attack patterns (descriptions of common methods for exploiting software systems), with a comprehensive schema and classification taxonomy. By MITRE Corporation.

W3 Security Resources – Large collection of information and resources on web security, including an FAQ, hosted by the W3C Consortium (the folks who set web standards/protocols, etc.)

Microsoft Security Site – Microsoft’s web site for discussion of security issues for MS products, including their web server products. Includes security self-assessment information, home and business security information, etc.

Security Focus.Com – Site from Symantec for news, forums, resources, vulnerability info, conference info, tools, etc. related to computer security including web and internet security issues. Search vulnerability database by keywords, date, vendor, version, etc.

Computer Emergency Response Team site – CERT’s internet security web site; includes web server security information; hosted by the Software Engineering Institute at Carnegie Mellon University.

See the ‘Web Tools’ section for web security test tool listings.

See the Softwareqatest.com Bookstore section on Security Testing for books on software security testing.

 

---

Web Usability Resources

Prioritizing Web Usability – PDF chapter from 2006 book on Web Usability by Jakob Nielsen and Hoa Loranger.

Usability.gov – Web site with a large collection of web usability resources, information, and guidelines. Although the site was developed by the U.S. federal government for use by various federal agencies, the site is a resource available to anyone.

User Interface Engineering – Web site of User Interface Engineering Inc., founded by Jared M. Spool. Many articles on web site and product usability, such as ‘Web Application Form Design’, ‘Seven Common Usability Testing Mistakes’, ‘5 Things to Know about Users’, and more.

Useit.com – Jakob Nielsen’s web usability website with such articles as ‘How Users Read on the Web”, ‘Costs of User Testing’, and ‘Differences between Print Design and Web Design’.

UIWizards.com resources list – Jeff Johnson’s UIWizards.com listing of usability design and testing resources

What is SEI? CMM? CMMI? ISO? IEEE? ANSI? Will it help?

• SEI = ‘Software Engineering Institute’ at Carnegie-Mellon University; initiated by the U.S. Defense Department to help improve software development processes.

 

• CMM = ‘Capability Maturity Model’, now called the CMMI (‘Capability Maturity Model Integration’), developed by the SEI. It’s a model of 5 levels of process ‘maturity’ that determine effectiveness in delivering quality software. It is geared to large organizations such as large U.S. Defense Department contractors. However, many of the QA processes involved are appropriate to any organization, and if reasonably applied can be helpful. Organizations can receive CMMI ratings by undergoing assessments by qualified auditors.

       Level 1 - characterized by chaos, periodic panics, and heroic
                 efforts required by individuals to successfully
                 complete projects.  Few if any processes in place;
                 successes may not be repeatable.

       Level 2 - software project tracking, requirements management,
                 realistic planning, and configuration management
                 processes are in place; successful practices can
                 be repeated.

       Level 3 - standard software development and maintenance processes
                 are integrated throughout an organization; a Software
                 Engineering Process Group is in place to oversee
                 software processes, and training programs are used to
                 ensure understanding and compliance.

       Level 4 - metrics are used to track productivity, processes,
                 and products.  Project performance is predictable,
                 and quality is consistently high.

       Level 5 - the focus is on continuous process improvement. The
                 impact of new processes and technologies can be
                 predicted and effectively implemented when required.


      Perspective on CMM ratings:  During 1997-2001, 1018 organizations
      were assessed.  Of those, 27% were rated at Level 1, 39% at 2,
      23% at 3, 6% at 4, and  5% at 5.  (For ratings during the period 
      1992-96, 62% were at Level 1, 23% at 2, 13% at 3, 2% at 4, and 
      0.4% at 5.)  The median size of organizations was 100 software 
      engineering/maintenance personnel; 32% of organizations were 
      U.S. federal contractors or agencies.  For those rated at 
      Level 1, the most problematical key process area was in 
      Software Quality Assurance.

• ISO = ‘International Organisation for Standardization’ – The ISO 9001:2008 standard (which provides some clarifications of the previous standard 9001:2000) concerns quality systems that are assessed by outside auditors, and it applies to many kinds of production and manufacturing organizations, not just software. It covers documentation, design, development, production, testing, installation, servicing, and other processes. The full set of standards consists of: (a)Q9001-2008 – Quality Management Systems: Requirements; (b)Q9000-2005 – Quality Management Systems: Fundamentals and Vocabulary; (c)Q9004-2009 – Quality Management Systems: Guidelines for Performance Improvements. To be ISO 9001 certified, a third-party auditor assesses an organization, and certification is typically good for about 3 years, after which a complete reassessment is required. Note that ISO certification does not necessarily indicate quality products – it indicates only that documented processes are followed. There are also other software-related ISO standards such as ISO/IEC 25010:2011 which includes a ‘quality in use model’ composed of five characteristics and a ‘product quality model’ that covers eight main characteristics of software. Also see http://www.iso.org/ for the latest information. In the U.S. the standards can be purchased via the ASQ web site at http://asq.org/quality-press/
  ISO 9126 is a standard for the evaluation of software quality and defines six high level quality characteristics that can be used in software evaluation. It includes functionality, reliability, usability, efficiency, maintainability, and portability.

 

• IEEE = ‘Institute of Electrical and Electronics Engineers’ – among other things, creates standards such as ‘IEEE Standard for Software Test Documentation’ (IEEE/ANSI Standard 829), ‘IEEE Standard of Software Unit Testing (IEEE/ANSI Standard 1008), ‘IEEE Standard for Software Quality Assurance Plans’ (IEEE/ANSI Standard 730), and others.

 

• ANSI = ‘American National Standards Institute’, the primary industrial standards body in the U.S.; publishes some software-related standards in conjunction with the IEEE and ASQ (American Society for Quality).

 

• Other software development/IT management process assessment methods besides CMMI and ISO 9000 include SPICE, Trillium, TickIT, Bootstrap, ITIL, MOF, and CobiT.

 

What is Quality Assurance ?, When should QA testing start in a project ? Why?, What is Software Testing ?, What is Software Quality ?, What is Software Verification and Validation ?,What is Test Plan ?, What is Test Case ?, What is Good Software Coding ?, What is a Good Design ?, Who is a Good Test Engineer ?, What is Walkthrough ?, What is Software Life Cycle ?, What is Software Inspection ?, What are the benefits of Automated Testing ?, What do you imagine are the main problems of working in a geographically distributed team ?, What are the common problems in Software Development Process ? in simple saying…

What is Quality Assurance ?

Quality assurance makes sure the project will be completed based on the previously agreed specifications, standards and functionality required without defects and possible problems. It monitors and tries to improve the development process from the beginning of the project to ensure this. It is oriented to “prevention”.

When should QA testing start in a project ? Why?

QA is involved in the project from the beginning. This helps the teams communicate and understand the problems and concerns, also gives time to set up the testing environment and configuration. On the other hand, actual testing starts after the test plans are written, reviewed and approved based on the design documentation.

What is Software Testing ?

Software testing is oriented to “detection”. It’s examining a system or an application under controlled conditions. It’s intentionally making things go wrong when they should not and things happen when they should not.

What is Software Quality ?

Quality software is reasonably bug free, delivered on time and within budget, meets requirements and/or expectations and is maintainable.

What is Software Verification and Validation ?

Verification is preventing mechanism to detect possible failures before the testing begin. It involves reviews, meetings, evaluating documents, plans, code, inspections, specifications etc. Validation occurs after verification and it’s the actual testing to find defects against the functionality or the specifications.

What is Test Plan ?

Test Plan is a document that describes the objectives, scope, approach and focus of a software testing effort.

What is Test Case ?

A test case is a document that describes an input, action or event and an expected response, to determine if a feature of an application is working correctly. A test case should contain particulars such as test case identifier, test case name, objective, test conditions/setup, input data requirements, steps and expected results.

What is Good Software Coding ?

Good code is bug free, readable, expandable in the future, easily maintainable and works according to the requirements.

What is a Good Design ?

In a good design, the overall structure is clear, understandable, easily modifiable and maintainable. Works correctly when implemented and functionality can be traced back to customer and end user requirements.

Who is a Good Test Engineer ?

Good test engineer has the ability to think the unthinkable, has the test to break attitute, strong desire to quality and attention to detail.

What is Walkthrough ?

Walkthrough is a quick and informal meeting for evaluation purposes.

What is Software Life Cycle ?

The software life cycle begins when an application is first conceived and ends when it is no longer in use. It includes aspects such as initial concept, requirements analysis, functional design, internal design, documentation planning, test planning, coding, document preparation, integration, testing, maintenance, updates, retesting, phase-out and other aspects.

What is Software Inspection ?

The purpose of inspection is trying to find defects and problems mostly in documents such as test plans, specifications, test cases, coding etc. It helps to find the problems and report it but not to fix it. It is one of the most cost effective methods of software quality. Many people can join the inspections but normally one moderator, one reader and one note taker are mandatory.

What are the benefits of Automated Testing ?

It’s very valuable for long term and on going projects. You can automize some or all of the tests which needs to be run from time to time repeatedly or diffucult to test manually. It saves time and effort, makes testing possible out of working hours and nights. They can be used by different people and many times in the future. By this way, you also standardize the testing process and you can depend on the results.

What do you imagine are the main problems of working in a geographically distributed team ?

The main problem is the communication. To know the team members, sharing as much information as possible whenever you need is very valuable to solve the problems and concerns. On the other hand, increasing the wired communication as much as possible, seting up meetings help to reduce the miscommunication problems.

What are the common problems in Software Development Process ?

Poor requirements, unrealistic schedule, inadequate testing, miscommunication and additional requirement changes after development begin.

Accessibility testing

Accessibility testing is the technique of making sure that your product is accessibility compliant. There could be many reasons why your product needs to be accessibility compliant as stated above.Typical accessibility problems can be classified into following four groups, each of them with different access difficulties and issues:

Visual impairments

Such as blindness, low or restricted vision, or color blindness. User with visual impairments uses assistive technology software that reads content loud. User with weak vision can also make text larger with browser setting or magnificent setting of operating system.

Motor skills Such as the inability to use a keyboard or mouse, or to make fine movements.

Hearing impairments Such as reduced or total loss of hearing

Cognitive abilities Such as reading difficulties, dyslexia or memory loss.

Development team can make sure that their product is partially accessibility compliant by code inspection and Unit testing. Test team needs to certify that product is accessibility compliant during the functional testing phase. In most cases, accessibility checklist is used to certify the accessibility compliance. This checklist can have information on what should be tested, how it should be tested and status of product for different access related problems. Template of this checklist is available here.

For accessibility testing to succeed, test team should plan a separate cycle for accessibility testing. Management should make sure that test team have information on what to test and all the tools that they need to test accessibility are available to them.

Typical test cases for accessibility might look similar to the following examples –

* Make sure that all functions are available via keyboard only (do not use mouse)
* Make sure that information is visible when display setting is changed to High Contrast modes.
* Make sure that screen reading tools can read all the text available and every picture/Image have corresponding alternate text associated with it.
* Make sure that product defined keyboard actions do not affect accessibility keyboard shortcuts.
* And many more..

There are many tools in the market to assist you in your accessibility testing. Any single tool cannot certify that your product is accessibility compliant. You will always need more than one tool to check accessibility compliance of your product. Broadly, tools related to accessibility can be divided into two categories. Inspectors or web checkers

This category of tool allows developer or tester to know exactly what information is being provided to an assistive technology. For example, tools like Inspect Object can be used to get information on what all information is given to the assistive technology. Assistive Technologies (AT)

This category of tools is what a person with disability will use. To make sure that product is accessibility compliant, tools like screen readers, screen magnifiers etc. are used. Testing with an assistive technology has to be performed manually to understand how the AT will interact with the product and documentation. More information on the tools is present in tool section of this website for you to explore.

Some tips that can be used for Accessibility testing .

* When using a screen reader, be sure to include tests for everything the user would be doing, such as install and un-install of the product.
* If a function cannot be performed using an Assistive Technology, then it may be considered accessible if it has a command line interface to perform that function.

Most of the time on windows platform, accessibility is built in your product using Microsoft Active Accessibility (MSAA). You can get more information about MSAA on this page.

What is MSAA?

MSAA is the abbreviation of Microsoft Active Accessibility. MSAA is a set of dynamic link libraries (DLLs) that provide COM interface and APIs. It is incorporated into the Microsoft Windows operating system and provide methods for exposing information about UI elements.

MSAA is used by assistive technologies like screen readers. These tools get information from the MSAA and gives to the user. MSAA gives information in the form of objects. Every UI element is treated as UI object and information like Name, Value, Role, State, Keyboard Shortcut, etc. is given to the tools for assistive technology. MSAA also supports events to capture state changes in the UI objects, for example object focus changes.

Today most screen readers expect that application have implemented MSAA. Implementing MSAA is probably one of the easiest way to ensure that assistive technologies will work with your product.

MSAA Software Development Kit contains following tools:

Inspect Objects Displays object information for user interface objects

Event Watcher Displays events fired by application when navigating the user interface

Accessible Explorer Displays object properties and relationship hierarchy

Development team can use these tools to find accessibility related defects in the development phase. Test team can validate that product is accessibility compliant using these tools and some other

Ten quick tests to check your website for accessibility

The DDA (Disability Discrimination Act) says that websites must be made accessible to blind and disabled people. So how can you check that your website actually has good accessibility? There are a number of basic tests you can make to address some of the main accessibility issues. The following list includes guidelines that provide a good start in increasing accessibility to disabled people:
1. Check informational images for alternative text
In Internet Explorer place the cursor over an informational image, for example, the organisation logo. Does a yellow box appear with a brief, accurate description of the image? For users whose browsers don’t support images, this alternative text is what they’ll see (or hear) in place of the image.

2. Check decorative images for alternative text
Place the cursor over a decorative image that doesn’t have any function other than to look nice. Does a yellow box appear with a description of the image? It shouldn’t. This image serves no purpose so there’s no reason for users whose browsers don’t support images to know that it’s here.Be careful though as this isn’t a foolproof accessibility test. If a yellow box doesn’t appear, this could mean one of two things:* The alternative text of the image is assigned a null value (alt=””), which means that it will be ignored by browsers that don’t support images. This is the ideal scenario.
* The alternative text of the image is simply not set at all, which means that users whose browsers do not support images will be alerted to its existence but will be unable to find out what purpose it carries – something which is very frustrating! This is certainly not the desired outcome.

3. ‘Listen’ to video or audio content with the volume turned off
If you turn your speakers off, you’re clearly unable to listen to, or follow, any audio content. This situation is faced by a deaf person on a daily basis. Ensure your website supplies subtitles or written transcripts so that this type of content is accessible to hearing impaired users.

4. Check that forms are accessible
Usually there’s prompt text next to each item in a form. For example, a contact form might have the prompt text ‘name’, ‘e-mail’ and ‘comments’, each one next to a box where site users will enter their details. When you click on the prompt text, does a flashing cursor appear in the box next to that text? If not, your forms may not be accessible.

 

5. Ensure that text can be resized
Can the text size on your website be adjusted? If not, then your website may not be accessible to web users with poor visibility. To check in Internet Explorer go to ‘View > Text size > Largest’. Alternatively, scroll with the wheel of your mouse whilst holding down the control key.

 

6. Check your website in the Lynx browser
The Lynx browser is a text-only browser and doesn’t support many of the features that other browsers such as Internet Explorer have. You can check how your site looks in this browser with the Lynx Viewer. If your website makes sense and can be navigated through the Lynx browser, then it’ll be fulfilling many of the web accessibility guidelines.

 

7. Check you can access all areas of your website without the use of a mouse
Can you navigate through your website using just tab, shift-tab and return in Internet Explorer? If not, then neither can keyboard- and voice-only users.

 

8. Check there’s a site map
Can you find a site map? If not, then neither can people who are lost on your website.

 

9. Ensure link text makes sense out of context
Visually impaired Internet users can browse web pages by tabbing from one link to the next. Does all the link text on your website make sense out of context? ‘Click here’ and ‘more’ are two common examples of non-descriptive link text that can cause a website to suffer poor accessibility.

 

10. Check your web pages with an automated program
Two accessibility programs available for free on the Internet are WebXACT and Wave. They’re unable to provide you with all the information that you need, as most accessibility checks must be done by humans, but they can tell you some of the areas where your site might be going wrong.

Conti…. FRS, SRS

Functional Specification Standard

Introduction

In general terms, the functional specification states what the proposed system is to do, whereas design is how the system is to be constructed to meet the functional specification. However in writing it, some consideration of design issues must take place, to ensure a realistic system is specified.

The functional specification should be clear, consistent, precise and unambiguous. The user requirement may mean that the user interface should be included in this document for some projects, whereas for others this will be done at the design stage either within a document or developed via a prototype.

It is important that there is a draft functional specification before the design stage on any project is started and that the functional specification is agreed and issued normally within a week of the final quality review. There must be a milestone on the project plan for the issue of the functional specification. The functional specification must be kept up-to-date, as this is the communication with the world outside the development staff.

The following should be used as a standard for a functional specification with some mandatory sections. The layout itself is at the discretion of the author except for Chapter 1. The document should have a standard front page, document authorisation page containing the title, issue, author and quality controller and contents page. Use diagrams where appropriate.

Do not be afraid of examples! Use them copiously throughout, as a brief, concrete example often illustrates a point much more succinctly than a normative explanation. Also remember to keep the examples interesting, as this is a useful way of keeping the reader’s interest – this is just as important in a functional specification as in any other type of document.

1.  Introduction

An introductory sentence or two about the project as this is probably the first document written on the project.

1.1.      Summary

A few sentences summarizing the project: what it is, who it is for (customer or internal), is it a bespoke project, a product, a demo.

1.2.      Requirements

This section should state the requirements the functional specification is attempting to fulfil. This may be an understanding of a customer’s requirement or a statement given as an internal starting point, e.g. produce a comprehensive mail tool in minimum time. Normally requirements are by their nature unstructured with high and low level statements intermingled. This section should refer to a separate requirements document if it exists. If there is anything else clarifying the requirement such as faxes these should also be referred to and probably a copy put into an Appendix.

1.3.      Numbers

This section should detail the number of users expected to use the system, how often, expected number of transactions (per minute/hour/day), peak usage times etc.

The question that should be asked of project stakeholders up-front is: “What numbers are we looking at?”

Capacity/response time needs have to be outlined so that we don’t come up with a slow/tiny system, or don’t totally over-do it and come up with a n-tier EJB solution costing £500k, when the system will only ever have 20 simultaneous users.

Such information will make a big difference to the architecture, i.e. the eventual design specification. This is why it is vital to establish these figures early in the project.

These figures are such an overarching issue that they do not belong in any one section. In fact the issue is expanded upon in several sections, such as User Community, Performance and Expandability.

1.4.      Existing System

This section should include an explanation of the system we are replacing, even if it’s an old manual system.

What problems does the current system have? Which of these problems do we solve?

What useful functions of the current system will we not provide (Constraints)?

Depending on the depth of analysis required, this section may also describe the root causes of each problem. “Root cause” analysis is a systematic way of uncovering the underlying cause of an identified problem:

“It’s amazing how much people do know about the problem behind the problem; it’s just that no-one – by which we usually mean management – had taken the time to ask them before. So, ask them and then ask them again.”
Source: Managing Software Requirements: A Unified Approach by Dean Leffingwell, Don Widrig – Chapter 4, “The Five Steps in Problem Analysis”

1.5.      Terminology

This section should contain all words or phrases having a special meaning for this project with a clear, concise, unambiguous statement on their meaning.

1.6.      References

List any document references with numbers, remembering to include issue numbers and/or dates so that the actual version is identified and refer to them as ref[n] in the rest of the document.

2.  Functional Description

The rest of the document may be divided into individual sections or chapters depending on the size and complexity of the system. Avoid forward references as the flow of the document is lost; consider re-ordering of the document in such circumstances.

Whereas requirements tend to be unstructured, the functions provided to fulfil the requirements must be structured. All statements as to functionality, should be written clearly using consistent terminology such that a test could be written to ensure the final system, performs as described and also that a design should fall naturally with no interpretation being necessary. It should be possible to draw up a table of functions within full system and product tests and incorporate a test for each function. To this end all functional statements should be numbered.

It may be that basic functionality could be identified such that some items are mandatory whereas some are highly desirable which should be clear from the requirements. If this is so, then identify these in this specification.

The functions should be grouped where possible under sub-headings to make an easily readable and understandable system.

All the following underlined headings must be included somewhere in the document, not necessarily in the order given here. If it is not relevant or we are not addressing it for this system, then say so.

Use Cases
Most likely these will be kept in a separate document or CASE tool, referenced from the functional specification. Development of the use cases and functional specification should happen in parallel, where information from one feeds the other incrementally.

Always avoid repetition. The amount of detail in the rest of the functional specification will depend on the number of use cases that have been written.

Although important, use cases do not capture all functional requirements: this is why we need an encompassing functional specification. The availability of a separate document also discourages use case authors from putting too much detail in the use case (e.g. functional requirements instead of usage scenario text) or the wrong detail (e.g. boundary conditions), which are both common mistakes.

(Note this is a similar approach to the Unified Process “Supplementary Spec” which captures additional detail that should be kept separate from the use case).

Where the functional specification references a use case, always use the unique use case name (e.g. “Perform Order Entry”). Depending on the size of the system being modelled, you might also need to include the package name.

Similarly, if the use case references an item in the functional spec, always use the section and number of the functional item (e.g. “User Community, item 1.2.3.4”). If possible (given the constraints of the word processor or CASE tool being used) provide a hyperlink that takes the reader directly to the referenced item.

User Community
Identification of who the system is aimed at. There may be more than one group of people and each group may have slightly different requirements. Are we providing different functions to fulfil these or not?

These groups of people are normally identified as use case roles (i.e. actors), and the functions assigned to each role as individual use cases. Where this information does not fit into the use case model, it should be captured in the main functional specification instead.

Administration Functions
How will the system be administered? Are there separate functions for an administrator? Is there any security built in to stop others using administrative functions? Passwords?

Error Handling
How errors should be handled should be stated. Identify the different types and reasons for the classification.

Security
Security considerations are an important part of any project. This section should detail possibilities of abuse of the system.

Along with error handling, the specification has to handle “the negative path”. There is no point in having a system that does lots of good things if it also does lots of bad things.

Help
what type of help is to be provided?

Printing
Ensure any printing to be provided is stated.

Interfaces

User
This could be a chapter in its own right if it is a full definition. If it is deferred to the design specification stage, this should be stated.

Software
we may be interfacing to existing software. This should be stated, e.g. toolkits, back ends of existing packages. State versions. Do interface documents exist?

Boundary Conditions
It should be clear what are the extremes to be taken into consideration. These items may have come up. This will vary with different systems but it could be items such as number of users, size of forms, number of forms.

Constraints
All other constraints not specified under particular headings. For example: economic, political, technical, system, environmental, scheduling constraints.

Platforms
We should list which platforms we will be supporting. Name a reference platform or platforms plus appropriate operating system versions.

Internationalisation
Is this to be included in the product now or in the future?

Performance

Capacity

Response times

Portability
although we may only be supporting one platform initially, we almost certainly will want to be able to port developments to other platforms. This should be stated here.

Expandability
State the likely expansion requirements. Some of the items may have been considered earlier in the document. These should be referred to from this section and any additional items put in.

 

Customisation
Are we allowing the user to customise the system? If so, what are we going to provide?

Support & Maintenance
are any functions to be included to make maintenance and support easier, e.g. internal monitoring of traffic flows.

Configuration Management
how are we proposing to manage the various software versions?

Documentation
List the documents that will be produced. This could refer to the project plan if that exists and contains such a list, otherwise it should be stated here.

For seeing sample Document click on below  FRS

FRS

making of HLD, LLD.

                                                   The High Level Design

 

The High Level Design Document is a pretty important document for a project, covering at a high level the overall design of the solution. If one were to try and present a very succinct summary of the High Level Document, it could be something like this:
– Detailed use case scenarios of key process flows of the application
– The class model and relationships
– The sequence diagrams which outline key use case scenarios
– The data/object model with relational table design
– User interface style and design
Another definition of the High Level Design Document would be:
High-level design is the transitional step between WHAT [requirements for sub-systems] the system does, and HOW [architecture and interfaces] the system will be implemented to meet the system requirements. This process includes the decomposition of system requirements into alternative project architectures and then the evaluation of these project architectures for optimum performance, functionality, cost, and other issues [technical and non-technical]. Stakeholder involvement is critical for this activity. In this step, internal and external interfaces are identified along with the needed industry standards. These interfaces are then managed throughout the development process.

 Examples of a High Level Document available:

1.1 BackgroundA well-written background should not cover more than a single page. As it is the opening section of your design document, the background must refer to the following questions:

• What the new system is
• The social and technological environment in which the system will function
• Its advantages over older systems
• Who the potential users are, and how they will benefit from it

Although some of these points were probably discussed at length elsewhere, their brief mention in the background is invaluable to the clarity and effectiveness of your document.1.2 Design Goals

This section should outline the technical manifestation of the system requirements, and list the goals your design is intended to achieve.

2. Architecture

2.1 Introduction

Should contain details such as:

• The type of the system (distributed, client-server, etc.)
• What platform(s) the system will run on
• The major inputs and outputs
• What user interfaces the system will have and in what form (web, Windows GUI, etc.)
• The distances between components – on different PCs, on a LAN, on the web
• A rough estimate of the number of instances of each part (modules, threads, processes, clients, etc.)

A block diagram of the modules and the relationships between them can be very useful here. Try to point out the dynamic aspects even though this view is mostly static: include arrows to indicate flow of data and/or control, multiple boxes to indicate multiple instances of a thread or a module, etc.2.2 Data

This section describes the persistent data, and/or any other data important enough to justify a separate section. Such a section will not be needed when there isn’t a central database, or any other bulk of permanent storage.

2.2.1 Introduction

Should explain the need for a database, the considerations which led to the choice of a particular type of database; should contain a short description of the data stored in there, an estimate of the size and frequency of updates, some special considerations like security requirements, recovery, interfacing with external systems, report generation, etc.

2.2.2 Schema

Should give a list of tables and columns in each table, a description of each of the following – data type, size, number of records, what data it keeps, which parts of the software read it and why, which parts of the software write it and when, an estimate of the update frequency.

2.2.3 File and Data Formats

Most systems deal with external data stored in files, the majority of these notably configuration files and imported or exported data files. The files should be listed, as well as which module reads/writes them, at what instances and for what purpose. This section should give the name, or a detailed description, of the file formats.

2.3 Communication

Whenever modules communicate, be it using TCP/IP or some other protocol, this section should indicate the following:

• Which modules do so, at what instances and for what purpose.
• Who listens and who connects.
• Which protocol is used for communication. If relevant, give a detailed description of both the protocol and the format of the exchanged data. Furthermore, explain how the protocol may accommodate future changes.

2.4 Code

2.4.1 Introduction

Should mention any special considerations that led to this specific design, or that should be kept in mind while implementing it. It has to include general comments about the code in addition to notes that do not fit in any specific sub- section.

2.4.2 Modules

A general description of each module and where it fits in the global picture. There should be a description of what each module is responsible for, what inputs it takes, where it takes these from, what format they are in, and what method it uses to take these inputs (TCP, read from a file, a pipe, direct function call…) The same description should appear for outputs: what outputs each module produces, where these outputs go, in what format, using what method. This section should also specify the following: what other modules each module uses and for what purpose; what non-trivial algorithm each module uses; what non-trivial data structures it uses and for what purpose.

2.4.1.n Internal Functions

This section should be repeated for each module description – a comprehensive listing of the module non-trivial internal functions (functions that are not part of any interface). Be sure to refer to each item mentioned below:

• General description of the function and what it is used for.
• The name of the function.
• The return type.
• Ranges of return values and their meanings.
• Parameter names, types, whether the parameter is input, output or both and under what circumstances it is read or written.
• Assumptions on the parameter values.
• Assumptions on other conditions, such as global data or system state.
• Input validations that the function performs.
• Side effects of the function.
• Exceptions the function might throw and under what conditions.
• Non-trivial algorithms used.
• Non-trivial data structures used and for what purpose.
• Other non-trivial functions that the function calls.
• If the software has a layer structure, or some other inner partitioning, then to which part or layer this function belongs (this information should be evident from the naming convention).

 2.4.3 Interfaces

 

Every module must expose one or more interfaces. If these interfaces are not trivial and clear from the module descriptions, they should be listed and described in an orderly fashion at this point. This description should detail the purpose of each interface and in what context it can be used. It should also mention what kind of modules or objects expose it and which ones will be using it. If using the interface requires obtaining a reference to it, there should be an explanation of how such a reference is to be obtained and what information will be needed in order to do so. Following the description of the interface, there should come a list of the interface functions, referring to the very same items mentioned above (see section 2.4.1.n Internal Functions).

It is advisable to add here a description of the scheme that will determine how the interfaces will evolve in future versions of the system in order to answer new requirements and modifications. If backward and forward compatibility is one of the requirements, explain how this scheme supports it.

3 Operations

This section should describe issues related to how the system is supposed to operate once it has been implemented.

3.1 User types

Should list the various types of users that will use the system (e.g., administrators, operators, managers, employees, customers…) Should also describe the user interface each user type would need to employ, and what prior knowledge and experience each type would require. Furthermore, this section should specify for each authenticated type/user which operations they may be allowed to perform, and which parts of the system, or data, would remain closed to them. Another important clarification should be the number of users the system may support and how many of them would be able to work concurrently.

3.2 Scenarios

Should describe a few typical scenarios of how the system works. For example, what happens in the system while a user logs in, gives a short sequence of typical commands and logs out – what data flows from which module to where, what triggers which actions, etc. If the system is not very simple, this description can make the difference between utterly confused and fully comprehending readers.

3.3 Installation

Yet another section that most designs ignore. It should explain how the system is installed, who can perform the installation (i.e., does it require some special skills, or can it be done by anyone?), what resources are needed for a successful installation, what medium would be used for the installation (e.g., a CD, download modules over the Internet, etc.), what user interface would be used during the installation (e.g., a Unix shell script, a Windows-like install UI, etc.), whether or not the installation would require a serial number. An important detail to include here is whether or not you allow several installations to co-exist on the same machine or LAN.

3.4 Licensing

Most commercial software is protected by a licensing scheme. If this is the case with your system, you should describe the scheme you are going to use, particularly the way a license will be validated (by some internal consistency check or by an online query to a central server), at what times (during installation, at program start up, at regular intervals) and what should happen when validation fails.

3.5 Upgrades

This section should describe the way the system would be upgraded to newer versions. It should also relate to the following questions:

• How will these upgrades be distributed (Internet, CD)?
• How (if at all) will an appropriate license be distributed and verified?
• Who will be authorized to install the upgraded version (i.e., which of the system’s user types mentioned in section 3.1)?
• How complicated will it be to reinstall the previous version in case an upgrade causes problems?
• How can a partial upgrade be installed? (e.g., in the case of a distributed system where it is unreasonable to expect the server and all the clients to be upgraded at the same time)
• How will the design handle version conflicts (between a server and a client, between two modules, between stored data and a module or between imported data and a module)?
• How will the design ensure that the users do not experience data loss or downtime during an upgrade?

If time-limited (“try before you buy”) or demo versions are planned, they require special attention. You must consider all possible scenarios, such as upgrading from a demo to a full version, attempting to install a demo over a full version, the coexistence of a demo and a full version on the same machine, etc.3.6 Uninstall

This section should describe the way the uninstall would be done, who would be authorized to perform it, what resources would be needed, what data would be left behind, and what would happen to that data when re-installing on a computer after an uninstall. Don’t forget to consider the implications of your licensing scheme.

4 Development

[Some companies require that this chapter be included in the design; others expect this information to appear in a separate document.]

This section lays out a plan for the development process. It can be quite complex if the project has modules that cannot be developed or tested before others are completed, or if it depends on the availability of external resources like developers, machines, access to special services or real-world objects, etc.

For example, you may choose to start by developing a mock-up version of some module, in order to allow the development and testing of another. Once this is done, you can develop the two modules in parallel using two independent teams. Another common technique is to manually prepare a data file that will be used as the input to some module until the UI is mature enough to supply real data.

The design should describe the necessary resources – number of developers, their required skills, the hardware, environment and development tools required for the development process. It should also say when and for how long each resource would be needed. This is the place to include development time estimates.

5 Miscellanea / Appendices

5.1 Conformance with standards

Must contain the lists of both the standards the system should conform to and the references as to where these standards may be obtained.

5.2 Interoperability with other systems

Must contain the list of the external systems that the new system should interact with in addition to a description of the way in which it should do so.

5.3 Expandability

If applicable, this section should explain how a third party would be able to extend the system. This may be done by writing plug-ins or scripts, or by writing some instructions in a configuration file.

5.4 Debugging

Most of the development time will no doubt be spent on debugging. The larger the system, the more essential it becomes to have some built-in mechanism to help debug the complete system. It can be a mechanism of log files, or special functions in each interface, or a special mode in the UI. This section should describe your choice of mechanisms, the ways in which they would be used, and what the programmers should do in order to make their module part of the game.

5.5 Security

Most software systems today face one or more security threats: spoofing, identity theft, password stealing, eavesdropping, sniffing, spamming, data theft, web site defacing, denial of service attacks, password breaking, fraud, forgery, hacking, viruses, worms, trojans – just to name a few.

This section should describe the security threats you foresee and intend to deal with. It should specify your assumptions regarding the environment (whether the computer is behind a firewall, who has physical access to it, etc.) and the means you plan to employ in order to protect the system (authentication, data encryption, input validation, internal sanity checks), etc.

5.6 Open Issues

There are always some issues left open. Sometimes information needed for making certain design decisions is not available in time or even at all. Sometimes decisions are delayed for a more convenient time. This section should list all the open issues in the design, and, if possible, point out what is required in order to resolve each one.

5.7 Glossary

List all the technical terms, concepts and acronyms that appear in the document or that are relevant to it, for the sake of the uninformed reader. The explanation of each term/concept/acronym should not exceed 4 lines. People will thank you for not having to spend hours on looking up unfamiliar terminology.

5.8 Bibliography

If applicable, list documents, publications, books and other information sources that your readers may find useful when trying to understand the full implications of your design.

 

 Cover Page

Team Name, Logo (optional), Deliverable name (e.g. “Web Calendar Design Document”), Date, Course-Section.Credits Page

Clearly describes each group member’s contribution in producing the deliverable.

Design Overview

This is the first place other developers/maintainers will look to learn how your project works. It must provide a comprehensive outline of the entire design. The design lead is responsible for writing the overview. Write this section as HTML and place it in the javadoc overview file.

1. Identifying info: project name, team name, authors, version, date, etc.
2. Design Overview. This crucial section provides an overall view of your design. Use your best technical writing to communicate to other designers the framework of your design solution. It must clarify the essence of your design to someone who is not familiar with it. Try to limit this to one page.
3. Design Issues. There are many tradeoffs in design, a few of which we discussed in class. Describe the key design issues raised in your project, what decisions (or tradefoffs) you made, and your rationale. (You can think of this section as being the background that lead to your Overview above).
4. Tools: Compilers and other development tools used, and their version numbers.
5. Libraries: Indicate what pre-existing libraries are used in your design, why you selected them, any licensing restrictions that may apply.
6. External References: Indicate where associated project documents can be found: specification document, prototype, structure chart diagram, design patterns, coding standards, etc. If these references are in digital form provide the file name or URL. MAKE SURE THESE ARE ACCURATE when the finished deliverable is submitted.

Class Diagram or Structure Chart

This diagram is the core of your design. Think of it as the blueprint of your software design. This diagram shows the relationship among all the classes in your design. You need not show the classes that create and operate the GUI. It’s recommended that you use a design tool such as Rational Rose to facilitate ease of modification (as this chart changes a lot during the design phase). Quality hand illustration is an acceptable alternative. The chart may be quite large. If it spans multiple pages be sure the page ordering or hierarchy is clearly indicated through some page numbering scheme or connector labels. You may create your finished chart on a single large piece of poster board if you desire. (One advantage of this is that it can be hung on the wall of your team meeting space.) Alternatively, place an image of your class diagram on your team web page.

Note: If you are reusing code don’t draw the entire structure of the reused code.
Note: If significant portions of your solution are functions (i.e., complex methods) rather than classes consider showing their decomposition with a structure chart.

Class Skeletons or Module Headers

You must implement and compile (not execute) the class definition for each class in your class diagram. (Don’t forget to count compile defects – see QA plan). The purpose and interface for the class is documented as Javadoc comments and must adhere to Javadoc style. The Javadocs can be placed on your team web page.

Each class in your system is to be assigned to an individual author who designs the class and writes the class definition. Each team member is responsible for at least one prominent class in your system.

A class definition consists of:

1. A Class Header
   • Identifying info: Class name, @author, @version, date, etc.
   • Descriptive Header
2. Method headers
   • Method (Operation) name – must correspond exactly with the name shown on the class diagram.
   • Purpose – Describe the purpose of this operation. Must be a concise, clear English sentence using terms from the problem domain.
   • Parameters and return types – name, type, and description.
   • Exceptions thrown.
   • (Design By Contract teams) Pre- and Post- conditions (except for getter/setter methods). Recommended: use custom tags
   • (Error-Checking teams) Document all error handling.

Interaction Diagrams (as appropriate)Also known as sequence diagrams or event trace diagrams. They provide specific details about the order in which events occur and how the classes interact.  They show the sequence in which objects pass messages to each other. Use some standard diagramming notation such as UML.  Rational Rose or similar tool recommended. How do you decide appropriateness? Use your judgments to determine which Use Cases are complex enough that you need to clarify the interations. As a guideline, an interaction diagram is probably appropriate if the Use Case has more than two objects involved, or has more than four steps.

Data Structures

Internal

Document how you have decided to implement your data abstractions. Provide a javadoc comment for each composite data structure in your solution. (I.E., array, vector, etc). This implementation should be private, so you will need to used the -private flag on the javadoc command. Be sure for each major data structure there is a rationale provided in “Design Issues” which explains your implementation choice.
Example:

/** the cipher key is an array of 26 characters,
* one to correspond to each letter of the alphabet.
*/
private char [] Cipher Key;

External

If your design includes external data files, show the exact format/contents of each file using javadoc in the appropriate class in your design.

Appendix

1. FTR Review Summary report
2. Compilation Listings showing a successful compile of all modules. If you are using JDK, use the -verbose flag on javac and include a screen capture of the output. Also include the output from the javadoc command.
3. QA Checklist, (with items checked off), signed by your QA person. You must provide an explanation for any items not checked off. If an item is missing or obviously unsatisfactory that has been checked off, the entire document will be returned for correction and a late penalty will accrue.

BRIEF ABOUT S.T.L.C, S.D.L.C

                                                  Software Test Life Cycle:

 

Software Testing Life Cycle (STLC) identifies what test activities to carry out to accomplish quality assurance process in a software development project.

 

There are different kinds of software development life cycle (SDLC) such as Waterfall, Spiral, Agile, and many others. Software testing has its own life cycle that intersects with every stage of the SDLC either it is Waterfall, Spiral or Agile. However, STLC varies from one to another based on size of project, test team, test in Scope/out of scope, and code release date (how frequently). So, knowledge about some of the major phase in STLC, quality assurance activities during phases and role of a tester, makes you always ready to accomplish your task with a mark. This picture describes one of widely used STLC phases.

 

Generally STLC for a single test cycle consists of phases:

1) Planning,

2) Analysis,

3) Design,

4) Initial Testing,

5) Testing Cycles,

6) Final Testing and Implementation,

7) Post release.

 

 

Planning: Some QA activities during planning phase.

High level test plan 

• Identify review process, Metrics
• Bug reporting procedures
• Acceptance criteria for QA
• schedule

 

Analysis Phase: Some QA activities during analysis phase.

• Develop Test Case format , Validation Matrix
• Develop, and plan Test Cycles matrices and time lines
• Begin writing Test Cases based on Functional Validation matrix
• Map baseline data to test cases to business requirements
• Identify Automation, Manual and Types of testing ,
• Test environment, automation system setup.

 

Design: Some QA activities during design phase.

• Test -planning review and verify.
• Review matrix (coverage).
• Continue working on Test Cases.(update, new )
• Finalize test case selection for each cycle for manual run and automation.

 

Initial Testing:

• Complete all plans, Test Case, scripting
• Unit test (Automated?)

 

Test Cycle:

• Test Cycle 1, run first set of test
• Report bugs – Triage(bug verification)- Bug fixes – Regression
• Add test cases as required
• Test Cycle 2, 3 …

 

Final Testing and Implementation:

• Code Freeze
• Run Test cases for including performance level.
• Communicate defect tracking metrics.
• Regression
• Documents.

 

Post Release:

• Evaluation meeting – lesson learned
• Prepare final Defect Report and metrics. Develop strategies to prevent similar problems in future project.
• Milestones for improvements
• Environment clean-up. Clean (tag and archive tests and data for that release) restore test machines to baseline for next test cycle

S.T.L.C  STRUCHERAL VIEW

 

 

 

V-model, Agile methodology, Agile manifesto

                                                           The V Model

The V model is a modified version of the Waterfall method. As opposed to the Waterfall method, this one was not designed in a linear axis; instead the stages turn back upwards after the coding phase is done so that it makes a V shape and hence the name – V Model. It was put forth by Paul E. Brook in 1986. Let’s look at the different stages, test processes, techniques, advantages and disadvantages of this method.

 

About The Cyclic Phases

The V Model is in contrast to the Waterfall method in more than one way. This developmental process is balanced and relies on the verification from the previous steps before proceeding forward. So the cycle of the model has been divided into several phases and each one is supposed to yield a predefined product.

When the product from one phase has reached completion, it will then form the basis for the next phase. This signifies the importance of verification in this model. Similar to the waterfall model, one progresses to the next step when the previous one has been completed. However in the V Model, the product from every phase needs to be checked and approved before moving forward.

This way of verification continues for all the stages and with subsequent phases, one receives a new base of an approved product which instills more confident in the project.

 

 

The Various Stages of the V model

1.         Requirement Analysis

This is the first step in the verification process. It is in here that the project and its function is decided. So a lot of brainstorming and documentation reveals what all will be required to produce that program or product. During this stage the employees are not going to discuss how it is going to be built; it is going to be a generalized discussion and a user requirement document is put forth. This document will carry information regarding the function of the system, performance, security, data, interface etc.

This document is required by the business analysts to convey the function of the system to the users. So it will merely be a guideline.

2.         System Design

Like the name of the phase suggests, here the possible design of the product is formulated. It is formulated after keeping in mind the requirement notes. While following the documents, if there is something that doesn’t fit right in the design, then the user is made aware of it and changes are accordingly planned. Diagrams and data dictionary is also produced here.

3.         Architecture Design

The architecture design, also known as the computer architecture design or the software design should realize the modules and the functionality of the modules which have to be incorporated.

4.         Module Design

In the module design, the architectural design is again broken up into sub units so that they can be studied and explained separately. The units are called modules. The modules can separately be decoded by the programmer.

The Validation Phases of the V model

1.         Unit Testing

A unit in the programming system is the smallest part which can be tested. In this phase each of these units are tested.

2.         Integration Testing or Interface Testing

In this phase the separate entities will be tested together to find out the flaws in the interfaces.

3.         System Testing

After the previous stage of interface testing, in this phase it is checked if the system meets the requirements that have been specified for this integrated product.

4.         Acceptance Testing

In the acceptance test, the integrated product is put against the requirement documents to see if it fulfills all the requirements.

5.         Release Testing

It is in here that judgment has to be made if the product or software which is created is suitable for the organization.

Advantages of the V Model

The biggest advantage of using the V Model is that unlike the Waterfall model and the aorta life cycle method, every stage is tested.

Disadvantages of the V Model

It assumes that the requirements do not change.

The design is not authenticated.

The Requirements are not verified.

At each stage there is a potential of errors. The first testing is done after the design of modules which is very late and costs a lot.

Aorta Life cycle, RAD, Incremental Build model,Evolutionary Development model.

                                                 Aorta Life Cycle Model

The aorta life cycle model is a software development method according to the waterfall model, but after each cycle is a feedback to the customer site.

 

History of the Aorta Life Cycle Model

In the eighties and nineties, many life cycle models were introduced. What these models have in common is that they have a shorter development cycle. By strict application of the ‘waterfall’ model, the customer only at the time of delivering what has become the final application. When using the other life cycle models are developed several cycles, where a customer gets feedback after each cycle. Any errors are thus more easily detected and the customer can directly indicate how he wants it. It is also important to make certain that each life cycle is completed without error. If an error is detected in a subsequent life cycle costs a lot of money to correct the error. Each previous cycle will be repeated for any new errors.

 

The Phases of the Aorta Life Cycle Model

The Aorta life cycle model includes nine steps to reach the final model to come. We start with the Orientation phase. In the orientation

Aorta Life Cycle ModelPhase is roughly mapped out the wishes of the user of the application are. Then comes the planning stage. During the planning phase, a project plan for the rest of the project. After these two phases beginning the concrete work, consisting of five key stages. In the model below, these phases appear along with their main product. The first letters of these phases form the word “Aorta”. This is the beating heart of a project. Finally, even the last two stages. The evaluation phase project to look back at the experience in identifying and then comes the final phase, the maintenance phase. In this final phase of maintenance on the delivered application. This is often distinction in a warranty and a maintenance period.Pros and Cons of the Aorta Life Cycle ModelAdvantages of the aorta life cycle modelIf in the beginning of the project failures are detected, it takes less effort (and therefore time and money) for this error. In the waterfall model phases to be properly sealed first before proceeding to the next stage. It assumes that the phases are correct before proceeding to the next phase.

In the waterfall model lay the emphasis on documentation. In the newer software development methodologies makes it less documentation. This means that when new people in the project, and people leave it is difficult to transfer knowledge. This disadvantage is not the waterfall method.

It is a clear and simple method. It is very obvious when certain steps are finished.

One can use this method of milestones.

The waterfall model is well known. Many people have experienced, so there might be easy to work.

Disadvantages of the Aorta Life Cycle Model

There are some disadvantages of this way to develop software.

Many software projects are dependent on external factors. The client is a very important external factor. Often the requirements during the course of the project change, because the client wants something different. It is a disadvantage that the waterfall method assumes that the requirements will not change during the project. Where the construction phase a requirement changed, a whole number of phases made again.

It is very difficult to time and cost estimate. The phases are very large, it is therefore very difficult to estimate how much each step cost.

In a number of new methods are almost all aspects of a software development project included. You can think of planning techniques, project management methods and how the project should be organized.

For many software projects, different people at different stages of the project. For example: the designers and builders. They all have a different view of the project as designers look at the project differently than the builders. Conversely, the builders often different from the design of the designers look than the designers themselves. Frequently, the design will be adjusted again. Here is the waterfall method is not made for that.

Within the project the team members often specialized. One team member will be only the first phase involved the design, while the only builders in construction helping to build the project. This can lead to waste of different sources. The main source is the time. An example: the designers are working on perfecting the design. The builders are in principle already start building, but because they work with the waterfall model, they should wait until the first phase is complete. This typical example of a waste of time.

When frequent portions of the software product to be delivered this gives the customer confidence but also the software development team.

Testing is done only in one of the last phases of the project. In many other software development methods will be tested once a specific sub-application is complete and is at last an integration test.

This is what you should know about the aorta life cycle model.

SPIRAL, PROTOTYPE, FISH BONE, SASHIMI

                                                             Spiral Model

The Spiral Life Cycle Model is a type of iterative software development model which is generally implemented in high risk projects. It was first proposed by Boehm. In this system development method, we combine the features of both, waterfall model and prototype model. In Spiral model we can arrange all the activities in the form of a spiral.Each loop in a spiral represents a development phase (and we can have any number of loops according to the project). Each loop has four sections or quadrants:

1. To determine the objectives, alternatives and constraints. We try to understand the product objectives, alternatives in design and constraints imposed because of cost, technology, schedule, etc.

2. Risk analysis and evaluation of alternatives. Here we try to find which other approaches can be implemented in order to fulfill the identified constraints. Operational and technical issues are addressed here. Risk mitigation is in focus in this phase. And evaluation of all these factors determines future action.

3. Execution of that phase of development. In this phase we develop the planned product. Testing is also done. In order to do development, waterfall or incremental approach can be implemented.

4. Planning the next phase.Here we review the progress and judge it considering all parameters. Issues which need to be resolved are identified in this phase and necessary steps are taken.Subsequent loops of spiral model involve similar phases. Analysis and engineering efforts are applied in this model. Large, expensive or complicated projects use this type of life cycle. If at any point of time one feels the risk involved in the project is a lot more than anticipated, one can abort it. Reviews at different phases can be done by an in-house person or by an external client.

 

Why spiral model is called Meta model?

Spiral model is also called as meta-model because in a way it comprises of other models of SDLC. Both waterfall and prototype models are used in it. Here we do software development systematically over the loops (adhering to waterfall approach) and at the same time we make a prototype and show it to user after completion of various phases (just in case of prototype model). This way we are able to reduce risks as well as follow systematic approach.Now let’s discuss the advantages and disadvantages of Spiral Model in detail.

 

 

Advantages of Spiral Model

1)      Spiral Life Cycle Model is one of the most flexible SDLC models in place. Development phases can be determined by the project manager, according to the complexity of the project.2)     Project monitoringis very easy and effective. Each phase, as well as each loop, requires a review from concerned people. This makes the model more transparent.3)      Risk management is one of the in-built features of the model, which makes it extra attractive compared to other models.4)     Changes can be introduced later in the life cycle as well. And coping with these changes isn’t a very big headache for the project manager.5)      Project estimates in terms of schedule, cost etc become more and more realistic as the project moves forward and loops in spiral get completed.

6)      It is suitable for high risk projects, where business needs may be unstable.

7)      A highly customized product can be developed using this.

Disadvantages of Spiral Model

1)      Cost involved in this model is usually high.

2)       It is a complicated approach especially for projects with a clear SRS.

3)       Skills required, evaluating and reviewing project from time to time, need expertise.

4)       Rules and protocols should be followed properly to effectively implement this model. Doing so, through-out the span of project is tough.

5)      Due to various customizations allowed from the client, using the same prototype in other projects, in future, is difficult.

6)       It is not suitable for low risk projects.

7)       Meeting budgetary and scheduling requirements is tough if this development process is followed.

8)       Amount of documentation required in intermediate stages makes management of project very complex affair.
I hope after reading the benefits (pros) and drawbacks (cons) of Spiral model, you will be a better judge to which model suits your requirements best.

Waterfall Model in Software Testing

                                            All About the Waterfall Model

The waterfall model is a model which was developed for software development; that is to create software. It is called as such because the model develops systematically from one phase to other in a downward fashion, like a waterfall.

The most probable phases through which it progresses downwards are

•           Definition Study/Analysis

•           Basic Design

•           Technical Design/Detailed Design

•           Construction

•           Testing

•           Integration

•           Management and

•           Maintenance

Waterfall Model

Before the advent of this method, the software development in the computer companies suffered from a haphazard integrated software network like cluttered knitting. However with this method they hoped to bring clarity in their projects.

About the Phases

As said earlier the waterfall model has been structured on multiple phases especially to help out the software construction companies to develop an organized system of construction. By following this method, the project will be divided into many stages thus easing out the whole process. For example you start with Phase I and according to this model, one only progresses to the next Phase once the previous one has been completed. This way one moves progressively to the final stage and once that point is reached, you cannot turn back; similar to the water in a waterfall.

Brief Description of the Phases of Waterfall Model

• Definition Study / Analysis: During this phase research is being conducted which includes brainstorming about the software, what it is going to be and what purpose is it going to fulfill.

•Basic Design: If the first phase gets successfully completed and a well thought out plan for the software development has been laid then the next step involves formulating the basic design of the software on paper.

•Technical Design / Detail Design:  After the basic design gets approved, then a more elaborated technical design can be planned. Here the functions of each of the part are decided and the engineering units are placed for example modules, programs etc.

•Construction / Implementation: In this phase the source code of the programs is written.

•Testing: At this phase, the whole design and its construction is put under a test to check its functionality. If there are any errors then they will surface at this point of the process.

•Integration: in the phase of Integration, the company puts it in use after the system has been successfully tested.

•Management and Maintenance: Maintenance and management is needed to ensure that the system will continue to perform as desired. Through the above mentioned steps it is clearly shown that the Waterfall model was meant to function in a systematic way that takes the production of the software from the basic step going downwards towards detailing just like a Waterfall which begins at the top of the cliff and goes downwards but not backwards.

History of the Waterfall Model

The history of the Waterfall model is somewhat disrupted. It is often said or believed that the model was first put forth by Winston Royce in 1970 in one of his articles; whereas he did not even used the word “waterfall.” In fact Royce later presented this model to depict a failure or a flaw in a non-working model. So later on, this term was mostly used in writing about something that is often wrongly done in the process of software development – like a common malpractice.

Royce was more of the opinion that a successful model should have the allowance of repetition or to go back and forth between phases which the waterfall model does not do. He examined the first draft of this model and documented that a recurrent method should be developed in this model. He felt the need of progressing only after a feedback from the previous stage has been received. This is known as the Iterative model.

As opposed to the Waterfall model, the Iterative model is more practical and has room for maneuver. Followers of the Iterative method perceive the Waterfall model as inappropriate.

Advantages of the Waterfall Model

Let’s look at some of the advantages of this model,

•The project requires the fulfillment of one phase, before proceeding to the next. Therefore if there is a fault in this software it will be detected during one of the initial phases and will be sealed off for correction.

•A lot of emphasis is laid on paperwork in this method as compared to the newer methods. When new workers enter the project, it is easier for them to carry on the work from where it had been left. The newer methods don’t document their developmental process which makes it difficult for a newer member of the team to understand what step is going to follow next. The Waterfall Model is a straight forward method and lets one know easily what stage is in progress.

•The Waterfall method is also well known amongst the software developers therefore it is easy to use. It is easier to develop various software through this method in short span of time.

Disadvantages of the Waterfall Model

There are many disadvantages to the model as well. Let’s have a look at those,

• Many software projects are dependent upon external factors; out of which the client for which the software is being designed is the biggest factor. It happens a lot of times, that the client changes the requirement of the project, thereby influencing an alteration in the normal plan of construction and hence the functionality as well. The Waterfall Model doesn’t work well in a situation like this as it assumes no alteration to occur once the process has started according to plan.

If, for instance, this happens in a Waterfall Model, then a number of steps would go to waste, and there would arise a need to start everything all over again. Of course this also brings about the aspect of time and money which will all go to waste. Therefore this method will not at all prove to be cost effective. It is not even easy to take out the cost estimate of each step, as each of the phases is quite big.

There are many other software developmental models which include many of the same aspects of the Waterfall model. But unlike the Waterfall model, these methods are not largely affected by the outside sources. In the waterfall model, there are many different people working in the different phases of the project like the designers and builders and each carries his own opinion regarding his area of expertise. The design, therefore, is bound to be influenced; however in the Waterfall model, there is no room for that.

•           The other negative aspect of this model is that a huge amount of time is also wasted. For example if we study any software development process, we know that Phase II cannot be executed until Phase I has been successfully completed; so while the designers are still designing the software, time of the builders is completely wasted.

•           Another disadvantage of this method is that the testing period comes quite late in the developmental process; whereas in various other developmental programs the designs would be tested a lot sooner to find the flaw at a time when a lot of time and money has not been wasted.

•           Elaborate documentation during the Waterfall method has its advantages, but it is not without the disadvantages as well. It takes a lot of effort and time, which is why it is not suitable for smaller projects.

Modified Waterfall Models

Due to the various disadvantages a lot of other modified versions of this model have also been put forth and some of these are mentioned below:

•Royce Model

Although not documented as the “Waterfall Model,” Winston W. Royce is credited with the first formal description of what we now know as the Waterfall Model in 1970. In his original definition, the model consisted of the following steps:  Requirements specification, Design, Construction (or coding in today’s terms), integration, testing and debugging, installation, and maintenance.

•SASHIMI MODEL
Sashimi is a true modified version of the Waterfall model. The phases are somewhat the same as in the Waterfall Model; only this time the phases are overlapping each other which present many advantages. For example the time won’t be wasted because before Phase I would be completed, Phase II would already be underway. Moreover, since they overlap, so one can return to the previous step if desired.

•Aorta life cycle model: The difference in this model is that they rely a lot on the feedback which comes from other phases before progressing onto the next.

• V waterfall model:
The V Waterfall Model relies on a linear software developmental program which stresses on balanced development more than anything else.

HISTORY OF TESTING

Before we get into deep let us know the history of testing……..

Usage of the term “bug” to describe a defect has been a part of engineering jargon for many decades, perhaps even from the times of Tomas Edison. In software testing it always refers to September 9th, 1945, when the first real bug traced an error in the Harvard Mark II, an electromechanical computer. This bug was carefully removed and taped to the log book (see picture:  The First “Computer Bug” ). This history is usually connected with the name Grace Murray Hopper, who described this event in the computer log.

Let us start the history of testing from the 50s, when the first modern programming language was designed: FORTRAN, the “FORmula TRANslator“, invented by John W. Backus, and the first FORTRAN compiler was delivered in April 1957.

The history of computers starts form Analytical Engine created in 1842 by Charles Babbage enlisted the help of Lady Ada Lovelace as a translator. Ada, called herself ‘an Analyst’

Charles Babbage, (1791 – 1871) was an English mathematician, philosopher, inventor and mechanical engineer who originated the concept of a programmable computer.

The separation of debugging from testing was initially introduced by Glenford J. Myers in 1979. Although his attention was on breakage testing (“a successful test is one that finds a bug”) it illustrated the desire of the software engineering community to separate fundamental development activities, such as debugging, from that of verification. Dave Gelperin and William C. Hetzel classified in 1988 the phases and goals in software testing in the following stages

* Until 1956 – Debugging oriented

* 1957–1978 – Demonstration oriented

* 1979–1982 – Destruction oriented

* 1983–1987 – Evaluation oriented

* 1988–2000 – Prevention oriented

1950 – 1960     1960 – 1970      1970 – 1980     1980 – 1990      1990 – 2000    2000 – 2010

 

1950-1960

1. 1953 Dr Edward Deming published: Management’s responsibility for the Use of Statistical Techniques in Industry. He outlined 14 quality principles in this book.

2. 1954 – The first truly mass-produced computer IBM 650 was marketed.

3. 1955 – Grace Hopper created Flow-matic, the first high-level language.

4. 1955 the first computer user group, called SHARE was formed.

5. Until 1957 it was the debugging oriented period, when there was no clear difference between testing and debugging.

 

1960-1970

1. 1960 – Digital Equipment Corporation (DEC) marketed the PDP-1,considered the first commercial minicomputer

2. 1960 – Block structure for better organization of code in the programs was introduced in Algol.

3. January 1, 1961 Computer Programming Fundamentals,Mcgraw-hill Inc; 1 edition by Herbert Leeds and Jerry Weinberg, describes software testing

4. 1962 The first computer science departments established at Purdue and Stanford.

5. 1962 Douglas Engelbart invented the computer mouse. (The keyboard was first invented and patented in 1868 by Christopher Latham Sholes.)

6. 1963 Adam, Carl – wrote his dissertation with topic: Petri Nets.

7. 1964 The American National Standards Institute (ANSI) officially adopted the ASCII (American Standard Code for Information Interchange) character code.

8. 1966 Book:”Computer Programming Fundamentals” by Herbert D. Leeds and Gerald M. Weinberg

9. 1967 Herm Schiller creates the first software code coverage monitor, called Memmap, at IBM Poughkeepsie. It supports 360/370 Assembler language. [Richard Bender]

10. 1968 The first introduction of the term software engineering and structured programming.

11. 1969 Edgar F. Codd introduced the concept of the relational system.

12. 1969 The first automatic teller machine (ATM) was put in service.

13. 1969 Richard Bender and Earl Pottorff created the first static and dynamic analysis tools using data flow analysis for improved test coverage. It increases code based coverage by 25% over the statement and branch coverage criteria in Memmap. (search for article called “How Do You Know When You Are Done Testing” that addresses this.) Note: this work was given the first Outstanding Invention Award ever handed out by IBM for breakthroughs in software engineering.

1970-1980

1. 1971 -The IEEE Computer Society was founded.

2. 1971 Milt Bryce first applied the term “methodology” to systems development.

3. 1972 Alan Kay developed Smalltalk the first object-oriented programming language.

4. 1972 Dennis Ritchie and Brian Kernighan developed C language.

5. 1973 The first computer user groups was founded in Boston.(disbanded in 1996)

6. 1973 ( or 1970?) Elmendorf, William R. introduced cause – effect graphs in functional testing.Elmendorf is also the person who first created equivalence class testing with boundary analysis.

7. 1973 Gruenberger, F., introduced the triangle testing problem in his article: Program testing, the historical perspective.

8. 1974, 5 April first software related standard: “MIL-S-52779 Software Quality Program Requirements” was issued.

9. 1974 The first international computer chess tournament is won by the Russian KAISSA program.

10. 1975 November Hamlet, R.G., Compiler-based Systematic Testing.

11. 1976 Software reliability : principles and practices by Glenford J. Myers   “The goal of the testers is to make the program fail. If his test case makes the program or system fail, then he is successful; if his test case does not make the program fail, then he is unsuccessful.”

“A good test case is a test case that has a high probability of detecting an undiscovered error, not a test case that show that the program works correctly.” – Glenford Myers.

12. 1976 Fagan, Michael E, published his article ” Design and Code Inspections to reduce errors in program development.” IBM System Journal Vol. 15, No.3, 1976 pp.182-211. (developed Code Inspection process)

13. 1976 December – The cyclomatic complexity metric for measure complexity of a routine, originally described by Tom McCabe.

14. 1977 Atari 2600 a video game console was released.

15. 1977 Requirements Based Testing was introduced.

16. 1978 CompuServe pioneered the wide use of e-mail.

17. 1978 Hayes developed the Smartmodem for the first personal computers, it took the market in 1981.

18. 1979 Philip Crosby,published his book “Quality is free” in McGraw-Hill Publishing.

19. 1979. – The separation of debugging from testing was initially introduced by Glenford J. Myers In his book “The Art of Software Testing” he provided definition of software testing widely used now and the first clear explanation of equivalence classes, boundaries and other testing principles .

1980-1990

1. 1980 Epson MX-80 became the best-selling dot-matrix printer.

2. 1982 William Edwards Deming offers a theory of management based on his famous 14 (quality principles) Points for Management.

3. 1983 Boris Beizer, “Software Testing Techniques” 1st edition.

4. 1983 Lotus 1-2-3 spreadsheet for DOS was released.

5. 1984 Tetris game was created.

6. 1985, July Commodore finally released the Amiga 1000 personal computer at a retail price of $1295

7. 1985 Excel spreadsheet application launched in by the Microsoft Corporation.Excel in the best friend of a tester.

8. 1986 Apple Macintosh Plus with 1MB of RAM was introduced.

9. 1986 – The particulars of the Six Sigma methodology were first formulated by Bill Smith at Motorola

10. 1987 ISO 9000 quality standards were released.

11. 1987 The Zachman Framework for descriptive represntations of an enterprise IT environment.

12. 1988 Eudora was the first non-mainframe e-mail client.

13. 1988 B. W. Boehm introduced a spiral model for software development.

14. 1988 ISO/IEC 12207- “Software Life Cycle Processes” was proposed and published in August 1995.

15. 1988 Dave Gelperin and William C. Hetzel classified the phases and goals in software testing.

16. 1989 WordPerfect Corporation released the WordPerfect 5.1 for DOS.

1990-2000

1. In the early 1990s, continuous quality improvement (CQI) methods were implemented.

2. From early 1990’s Bug Tracking and Version Control tools become popular.

3. 1991, June – publication of ISO 9000-3: “Quality management and quality assurance standards”

Part 3: Guidelines for the application of ISO 9001 to the development, supply and maintenance of software.

4. 1991- Linus Torvalds wrote his own unix kernel for the popular now Linux operating system.

5. 1992, October, IBM introduced the first ThinkPad model 700.

6. 1993 Software Quality Automation, Inc., Woburn, Mass., has unveiled SQA

TeamTest, a GUI testing tool implemented on a team/workgroup model.
Rational purchased SQA TeamTest v6.1

IBM acquired Rational corporation on December 2002.

7. 1993 HP LaserJet 4L was introduced.

8. 1993 Mosley, Daniel J., introduced Decision table method

9. 1994, October 13, Marc Andreessen launched Web browser called Mosaic Netscape 0.9.

10. 1994, 5 December DoD issued MIL-STD-498, software Development and Documentation.

11. 1995, July – Microsoft released the Windows 95 operating system.

12. 1997 UML (Unified Modeling Language) was introduced by James Rumbaugh,Grady Booch and Ivar Jacobson.

13. 1998 Rational unified process (RUP) was introduced.

14. 1998 K. Zambelich published article “Totally Data-Driven Automated Testing“

15. 1999, May How (and how not) to implement Data Driven Automation using Rational Robot , by Carl Nagle SAS Institute, Inc

16. 1999 -Poston, Robert developed a specification-based test generation tool

2000-2010

1. 2000, March First Keyword Driven Automation was implemented using Rational Robot

2. Rational Unified Process (RUP) Methodology (Develop iteratively, with risk as the primary iteration driver)