Reliability testing is more costly compared to other types of testing. Thus while doing reliability testing, proper management and planning is required. This plan includes testing process to be implemented, data about its environment, test schedule, test points etc. Problems in designing test cases.
Some common problems that occur when designing test cases include:
- Test cases can be designed simply by selecting only valid input values for each field in the software. When changes are made in a particular module, the previous values may not actually test the new features introduced after the older version of software.
- There may be some critical runs in the software which are not handled by any existing test case. Therefore, it is necessary to ensure that all possible types of test cases are considered through careful test case selection.
Test plans have two main functions. One is to determine the sample size and testing time needed to estimate model parameters. The other is to demonstrate that you have met specified reliability requirements. A test plan includes:
- The number of units you need to test.
- A stopping rule: the amount of time you must test each unit or the number of failures that must occur.
- Success criterion: the number of failures allowed while the test still passes (for example, all units are tested for the specified amount of time and there are no failures).
There is tremendous value in developing a comprehensive test plan that outlines all of the reliability testing activities. This can aid in comparing against marketing specs, helps to satisfy customer requirements and expectations, and is helpful for test labs that may be utilized to help perform some of the tests.
Test plans often point out overlooked areas such as how to test a product, how to fixture, what constitutes a failure, and many other contingency plans necessary for successful test activities.
The objective of a reliability test plan is to tie all of the different reliability test activities together to assure that all of the testing needs are satisfied and that there is no duplication between tests.
Reliability Test Requirements
Reliability test requirements can follow from any analysis for which the first estimate of failure probability, failure mode or effect needs to be justified. Evidence can be generated with some level of confidence by testing. With software-based systems, the probability is a mix of software and hardware-based failures. Testing reliability requirements is problematic for several reasons. A single test is in most cases insufficient to generate enough statistical data. Multiple tests or long-duration tests are usually very expensive. Some tests are simply impractical, and environmental conditions can be hard to predict over a systems life-cycle.
Reliability engineering is used to design a realistic and affordable test program that provides empirical evidence that the system meets its reliability requirements. Statistical confidence levels are used to address some of these concerns. A certain parameter is expressed along with a corresponding confidence level: for example, an MTBF of 1000 hours at 90% confidence level. From this specification, the reliability engineer can, for example, design a test with explicit criteria for the number of hours and number of failures until the requirement is met or failed. Different sorts of tests are possible.
The combination of required reliability level and required confidence level greatly affects the development cost and the risk to both the customer and producer. Care is needed to select the best combination of requirements – e.g. cost-effectiveness. Reliability testing may be performed at various levels, such as component, subsystem and system. Also, many factors must be addressed during testing and operation, such as extreme temperature and humidity, shock, vibration, or other environmental factors (like loss of signal, cooling or power; or other catastrophes such as fire, floods, excessive heat, physical or security violations or other myriad forms of damage or degradation). For systems that must last many years, accelerated life tests may be needed.
