How reliable is a product behaviour when it is used? Reliability testing is used to determine whether the product works reliably over a period of time and/or conditions. These failures then can be used to improve the product to overcome its reliability issues.
Consider a new bus which has to undergo reliability testing to ensure whether the vehicle is reliable enough to be used for cross country journeys across the Australian outback.
During the tests, the vehicle keeps breaking down and has problems primarily with its engine overheating. This unreliability needs to be overcome to make sure the vehicle can perform under these conditions for prolonged periods.
Another example could be an organisation that makes elevators. The organisation has finished building their latest style of elevator. The elevator undergoes a series of reliability tests to ensure that they work as expected.
By running a reliability test of the elevator moving up and down for a set period of time, any of the components used in its manufacture which are suspect, could be rapidly determined in a reliability test and these inadequacies corrected before the elevator is marketed.
Reliability testing checks whether the product is reliable.
A Swiss clock manufacturer has to ensure that their clocks are accurate and hence reliable. Their reputation of not only building accurate but reliable clocks is world renowned.
To ensure that this reputation remains intact, every clock produced is checked against a master clock whose reliability and accuracy has already been assured. Any clocks that deviate from the accuracy and reliability of the master clock are either repaired or discarded.
How secure is a product's behaviour when it is used? Security testing is used to determine whether the product works securely over a period of time and/or conditions. Any failures can then be used to improve the product to overcome its security deficiencies or mechanisms put in place to reduce the impact of these deficiencies.
Consider several new cars whose marketing stipulates that they are secure, are tested by an independent authority. For the testing, a reformed car thief is used and the time taken for this reformed offender to break into the cars noted.
Four of the cars security systems can be overcome within 30 seconds. The manufacturers are told that their security systems are weak and need to be readdressed before the independent authority can issue a security rating.
Security testing checks the security of a product.
A major bank develops an on-line banking system. Two days before the system is scheduled to go live, one of the directors asks the development team on whether the system passed all the testing. The development team assured the director that all the tests that they ran proved that the system was secure.
Unimpressed by this, the director authorises a third party security specialist organisation to carry out penetration tests. Within minutes, the security testers have opened up several security holes caused by poor development.
The director decides to postpone the launch of the online banking facility and orders corrections to be made to the issues highlighted by the security testers. Once the corrections are complete, the security testers test the system again. This time the testing reveals that the system is secure and the online banking system is ready to be used by the general public.