Thursday, July 31, 2008

Quality By Software Engineering Perspective

Attributes of a Good Quality Software:

1) A good quality software product:
Is the one which meets all the requirements when viewed through customer’s perspective. Hence the prime definition of software quality is meeting customer needs or understanding customer requirements, expectations and exceeding these expectations. If using the software product satisfies the customer, then it is a good quality product otherwise it is not.

2) Customer needs are not always translated into his requirements:
Although the software creation team tries their level best to accurately capture the customer needs into customer requirements; in spite of all these efforts, sometimes the developers fail in their endeavor due to 1) Lack of accurate communication of the exact needs by the customer 2) Misunderstanding of the requirements as captured by the developers.

3) A Software Free from Bugs or Defects may not always be a Quality product:
Quality of a software product is not merely the absence of bugs / defects in it. Even if it has been developed without any bugs; however if it is difficult to learn and tiresome to operate, then it can’t be called a quality product.

4) A Software may be termed as a Quality product in spite of presence of some Bugs or Defects in it:
Acceptance of few known bugs in the software is a subjective term depending upon the nature of the bugs. If the software product is released to the customer very well within the constraints of budget, time & resources, & it contains some known open bugs, which will not cause any loss to the customer; then also it can be termed as a quality product. However for critical products aim of ensuring 100% or zero defect approach becomes necessary.

5) Maintaining quality of a software product is not only the responsibility of software testers:
Producing right quality software product is the responsibility of everyone involved in the chain including the customer. Role of a software tester is to identify the deviations and report them to the development team effectively.

Software Tester has a prime role to validate the correctness, reliability, usability and interoperability of the product and report the deviations.

Software tester may not be able to validate certain important quality related factors like maintainability, reusability, flexibility & portability etc. etc.

6) Software tester should endeavor to detect bugs in the software right from the beginning itself:
As the software moves into advanced stages of its development, the cost of fixing the bugs in it also keeps on increasing exponentially.

For example, If during an advanced stage of system testing of software, any bug is found in its basic design itself, this could lead to an extremely costly process of fixing the bug. In addition to this may necessitate amendments in the design & the basic code itself.

Hence detection & rectification of bugs during initial stages of design are quite economical.

7) Software Quality Factors are essential attributes, absence of which can pose threat to the success of the software:
For any software product following are the important quality factors, the priority and importance of which may vary from software product to product. For example, factors like flexibility and reusability gain prime importance, if the software under development needs to be changed quite frequently.

#) Flexibility: An effort required to modify an operational program.

#) Reusability: Extent to which a program can be used in other applications.

#) Correctness: Extent to which the program satisfies the desired requirements.

#) Reliability: Extent to which the program can be expected to perform its intended functions with desired precision.

#) Efficiency: The amount of computing resources and code required by the program to perform a particular function.

#) Integrity: Extent to which unauthorized persons get prevented / controlled from accessing the software or its data.

#) Usability: Effort required in learning, operating, preparing inputs, and interpreting the outputs of a software program.

#) Maintainability: Effort required in locating and fixing an error in an operational program.

#) Testability: Effort required in testing a program to ensure that it performs in accordance with its intended function.

#) Portability: Effort required to transfer the software from hardware of one configuration to the other.

#) Interoperability: Effort required to couple one system with the other.

8) Cost of software quality refers to the total expenditure incurred on firstly preventing the errors, secondly identifying the errors and lastly correcting these errors:
Cost of quality can be brought down if the software developers are extra vigilant in developing their application and before passing it into testing phase & endeavor to ensure that the application is either free of defects or has bare minimum number of defects. However such activities aimed at preventing errors from going into the product involve additional efforts & costs. This calls for added concentration on building efficient process of development and keeping on continuously improving it by identifying weaknesses therein. The benefits of such extra efforts may not be immediate, however in the long run cost of software quality ought to come down significantly.

9) Cost of fixing a software bug is possible to be brought down, if the bug is able to be detected during the early stages of development:
As the software moves into advanced stages of its development, the cost of fixing the bugs in it also keeps on increasing exponentially. Hence practice of life cycle testing becomes necessary.

10) Apart from testing of the software product, repeated inspections, design reviews & code walkthroughs are effective quality control measures.

An Introduction to Software Testing

Software testing is the process used to assess the quality of computer software. Quality is not limited to, the process of executing a program or application with the intent of finding software bugs.


Software Faults:The software faults occur through the following process. A programmer makes an error (mistake), which results in a defect (fault, bug) in the software source code. If this defect is executed, in certain situations the system will produce wrong results, causing a failure. Not all defects will necessarily result in failures. A defect can turn into a failure when the environment is changed. Examples of these changes in environment include the software being run on a new hardware platform, alterations in source data or interacting with different software.


Software Quality:Quality is not an absolute term; it is value to some person. With that in mind, testing can never completely establish the correctness of computer software. Functional dimensions of quality like Usability, Scalability, Performance, Compatibility, Reliability are highly subjective terms having significant value to one person but may be intolerable to the other.

Complexities in Software Testing:A problem with software testing is that testing all combinations of inputs and preconditions is not feasible when testing anything other than a simple product. This means that the number of defects in a software product can be very large and defects that occur infrequently are difficult to find during testing.

Approach to effective Software Testing:An effective approach to the process of testing of complex products involves "Investigation" or "Questioning" a product in order to evaluate it. Here "Questions" or "Investigations" are the activities which a tester tries to perform on the software product, and the product answers with its behavior in reaction to the probing done by the tester.

Software testing is successful only if used in association with "Verification" and "Validation"

"Verification" asks a question: Have we built the "Software - Right" (i.e. Does it match the specifications)?

"Validation" asks a question: Have we built the "Right – Software" (i.e. Is this what the customer wants)?


Objectives of Software Testing:
1) Testing is a process of executing a software program with the intention of finding an error.
2) A good test case is one that has a high probability of finding an as yet undiscovered error.
3) A successful test is one that uncovers an as yet undiscovered error.


Outcome of Software Testing:
Testing should systematically uncover different classes of errors in a minimum amount of time and with a minimum amount of effort. A secondary benefit of testing is that it demonstrates that the software appears to be working as stated in the specifications. The data collected through testing can also provide an indication of the software's reliability and quality. But, testing cannot show the absence of defect -- it can only show that software defects are present.

What are the CMM Level of Companies

Various Capability Maturity Levels – CMM Levels for Companies

The Capability Maturity Model defines following levels for the organizations depending upon the processes being followed by them.


CMM Level 0 - Companies:
CMM – Level 0 companies are the ones, which do not have any structured Processes, Tracking Mechanisms & Plans. It is left to the developer or any person responsible for Quality to ensure that the product meets the expectations.

CMM Level 1 - Companies: Work is Performed Informally

CMM – Level 1 companies are the ones, where the teams put in extra work hard to achieve the results. Such companies do not have any structured Tracking Mechanisms & defined Standards. The software development work is performed informally but it is not properly documented.

CMM Level 2 - Companies: Work is Planned and Tracked

CMM – Level 2 companies are the ones, where some planned processes within the teams exist and the team can repeat them or follow these processes for all projects being handled by them. However these process are not standardized across the organization. All the teams within the organization do not follow the same standard.

CMM Level 3 - Companies: Work is Well-Defined

CMM – Level 3 companies are the ones, where the processes are well defined and are followed throughout the organization.

CMM Level 4 - Companies: Work is Quantitatively Controlled

CMM – Level 4 companies are the ones, where the processes are well defined and are followed throughout the organization. In such companies, Goals to be achieved are well defined and the actual output is measured. Such companies have proper mechanism to collect the Metrics, hence future performance can predicted.

CMM Level 5 - Companies: Work is based upon Continuous Improvement

CMM – Level 5 companies are the ones, which have well defined processes which are properly measured. Such organizations have good understanding of IT projects which have good effect on the Organizational goals. Level – 5 organizations are able to continuously improve their processes based upon such understanding.

Introduction to CMM & CMMI

It is the prime requirement of any Quality Software that it should reasonably be bug-free, delivered on time and within budgetary constraints. It should meet the defined requirements or expectations, and should be maintainable. Therefore to produce error free and high quality software there arises a great need that certain internationally recognized standards be followed.

Quality Standards: Various Quality Standards are available

ISO 9001: 2000 - is Quality Management System Certification. To achieve this, an organization is expected to satisfy the requirements defined in CL. 1 to 8 of ISO 9001: 2000

Six Sigma - is a process improvement methodology focused on reduction in variation of the processes around the mean. Its objective is to make the process defect free.

CMM - CMM means - Capability Maturity Model. It is a standard for assessing and improving processes related to software development. The software community had developed it in the year 1986 under the leadership from SEI – Software Engineering Institute. It is a process capability maturity model related to software applications & it helps in defining and understanding of the processes followed by the organization. It provides guidance to the measurement of software process maturity and helps process improvement programs. CMM had been devised as a tool for objectively assessing the ability of government contractors' processes to perform a contracted software project.

CMM emphasizes on the appropriateness of the Process followed for developing a software product. CMM aims to ensure that the process is capable to produce error free product. Process driven companies are more successful as compared to people driven companies. Hence a company needs to have a good process for software development for being successful.

Advent of CMMI: In CMM, the entire emphasis had been on the software practices. However software was becoming a major factor in the systems which were being built that it had become virtually impossible to logically separate the two disciplines of systems & practices. Hence SEI redirected all its effort toward the integration of system and software practices which led to the birth of CMMI which stands for Capability Maturity Model Integration.

CMMI has now superceded CMM. The older term CMM has been renamed now to Software Engineering CMM (SE-CMM).

Prior to going deep into CMM or CMMI, lets understand what a software process is.

A Software Process can be defined as set of activities, methods, practices and transformations which people employ to develop and maintain software and the associated products. The quality of a software product is essentially determined by the quality of the processes employed to develop and maintain it.


Five Maturity Levels of SEI CMM: Continuous process improvement is based on many small but evolutionary steps. CMM organizes these steps into 5 maturity levels. Each maturity level comprises of a set of process goals which, upon getting satisfied, stabilizes an important component of the software process. Organizing the goals into different levels helps the organization to prioritize their improvement actions. The five maturity levels of CMM are as under.

1) Initial: The software process is characterized as ad hoc, and occasionally even chaotic. Few processes are defined, and success depends on individual effort and heroics.

2) Repeatable: Basic project management processes are established to track cost, schedule, and functionality. The necessary process discipline is in place to repeat earlier successes on projects with similar applications.

3) Defined: The software process for both management and engineering activities is documented, standardized, and integrated into a standard software process for the organization. All projects use an approved, tailored version of the organization's standard software process for developing and maintaining software.

4) Managed: Detailed measures of the software process and product quality are collected. Both the software process and products are quantitatively understood and controlled.

5) Optimizing: Continuous process improvement is enabled by quantitative feedback from the process and from piloting innovative ideas and technologies.

Detailed Information is available on http://www.sei.cmu.edu/cmmi/

Classification Among Software Testing

Broad Differentiation between Levels of Software Testing versus Types of Testing versus Techniques of Testing are:

Levels of Testing: Various levels of testing are
  • Unit Testing
  • Integration Testing
  • System Testing
Types of Testing : classified on the basis of intent of testing
  • Acceptance Testing
  • Performance Testing
  • Load Testing
  • Regression Testing
Techniques of Testing: are broadly categorized as under
  • Black box Testing
  • White box Testing