Software QA Testing

Manual testing is always required.
Automated software testing accelerates the overall
testing effort, but it doesn’t eliminate the need for
manual testing. A certain amount of manual testing
will always be required to ensure that new and
modified features are functioning properly.
Effective testing is best achieved through a
combination of manual and automated tests.
Automation makes sense for validation of existing
system functionality where few defects are expected
and speed is a top priority. But new and complex
systems generally require manual testing by a
knowledgeable tester.
The business case for automated testing varies
from one project to the next, and isn’t always clear.
Automation requires a significant upfront investment
in tools and script development, and usually only
pays back after 4–6 iterations of testing. For systems
with limited testing needs, or involving only minor
changes, this initial investment may not be justified.
Regression effort analysis
Reality check
Testing is easy
...and other myths
Regression automation effort analysis
0
50
100
150
200
250
300
350
400
3 6912
Manual
Each test cycle takes
26 effort days for
500 test cases
After initial automation,
each test cycle requires
minimal additional effort
‘Payback’ point
One-time fixed at 6 test runs
costs for initial
automation
Effort days
Test cycles
Automated
9
Checklist
• Do you use automated testing in the right
situations, such as:
- Applications driven by a graphical user
interface?
- Regression testing of existing functions?
- Tests that are highly repetitive or
data-driven, with predictable results?
• Do you use manual testing in the right
situations, such as:
- Testing new or extensively modified
functions?
- Functions that are highly complex and
would require
a massive effort to develop automated
tests?
Performance problems can and should be identified and fixed throughout the development life cycle. Many people think the sole purpose of testing is to find defects that could crash the system or produce incorrect results. But performance problems often have an even greater impact on the organisation – frustrating users, eroding productivity, and in some cases bringing the entire operation to a grinding halt. To avoid costly delays and reworks, start looking for performance issues early in the development cycle. Bottleneck analysis, code-profiling and performance modelling can help identify and resolve defects as early as the design phase – preventing performance issues from becoming performance problems. Software development lifecycle and main performance management activities and outcomes Most organisations test performance by trying to replicate the production environment; however, this kind of test environment is expensive to procure and maintain. Moreover, demand for such an environment typically exceeds supply – particularly near the end of a project – creating a major bottleneck in the schedule. Development phases Performance management Software development life cycle and main performance management activities and outcomes Establish performance acceptance criteria Establish performance risk management plan Establish performance test scenarios and models Establish architectural guidelines for performance Identify and optimise ine cient database queries Identify and optimise ine cient code Establish baseline and benchmark for test executed Identify bottleneck suspects Monitor application performance Gather data for capacity management Gather past workload distribution statistics Gather client’s architectural standards, regional impacts, security model Identify architectural constraints Capture architectural views De ne key metrics to monitor For each sub-system build a prototype and instrument prototype for performance metrics and evaluate results Instrument critical code-paths with performance monitors Unit testing and code pro ling activities have taken place Tune application, database, and middle ware Monitor system performance continually Monitor user activity Monitor system availability Performance acceptance criteria Modeling and engineering Code optimisation Performance testing Monitoring and capacity planning Requirements and analysis Design and construction Testing Production Objectives Activities Checklist • Are you making a good first impression with your users? • Are performance requirements objectively defined, or are they inferred based on current system performance? • Are performance defects identified early, or do they often cause last-minute surprises and delays? • Is performance testing only performed in a full replica of the production environment?