Despite a push for more testing of applications, online or off, there aren’t any standardized systems available. There are common techniques, but there lacks a unifying by-the-book approach which yields reliable results. A lack of standards should, however, be no reason not to test your web application. Here is a general overview of the five basic approaches to testing.
I wrote this article a while ago, if you feel it doesn’t reflect current trends, please leave a comment. I’ve since become a fan of assisted manual testing.
Monkeys in a closet
I’ll start with this one as it is the most common, and the most misguided approach. It is achieved by deploying your application and letting many test subjects use the site and look for problems. The idea is to get more eyeballs and cursors on the site weeding out defects. In practice, this does indeed produce many defect reports but usually fails in detecting severe issues.
The trouble starts with the individuals hired as testers. Given the pay range for many eye-balls tests, you won’t have people qualified or trained for this type of work. Sticking a bunch of these people together in a room is not going to get you good results. Parts of the site remain untested, and much effort is wasted with redundant permutations. Furthermore, you are likely to create resent from other teams with poor defect reports.
Let the programmers do it
First in line to ensure proper operations are the people responsible for developing the web application. These are the programmers who put integrate the various modules bits and write custom code for your unique features. They know the system well and are in the best position to perform certain kinds of tests.
A few common problems arise here. Many coders lack sufficient training in testing techniques. Furthermore, a few teams may have a work ethic putting testing beneath them. Some don’t understand why they should be testing the website, after all, it is just a bunch of modules written by others.
Web applications, even smaller interactive websites, need to be treated like any software development. Software testing is fortunately well documented. Coders should be comfortable with unit tests and have a readily available test environment. They can’t do all the testing though; programmers may not be suitable for use-case or usability testing. Let them focus on unit and functional tests.
Manual feature testing
Despite automation, you can’t avoid manual testing. The standard approach is to write a test plan that a person can execute. There are numerous references to creating test plans, though many lack substantially on the actual art of writing a good test case.
It is this lack of good test cases which is the pitfall of manual testing. Team members without substantial experience in writing test cases are about the same as the monkey scenario. Proper test writing requires balancing of what requires light testing and what requires in-depth testing. It is not possible to test everything. A sound test designer must have a method to choosing his test executions. Unfortunately, there aren’t many, if any, well documented and complete methods.
Even without clear tactics, there is no way you can avoid manual testing and expect to produce a high-quality site. Finding somebody capable of writing good test cases is your biggest problem.
Record and play automation
It sounds appealing to run through the website once, record it, and then play it back again for any future release. When something breaks the test framework tells you. In theory, the automation tools require little training since the tester doesn’t do much more than use the website as a regular user.
The tester sits down, takes five to ten minutes to record the script, then another two to twenty hours manually tweaking it! If you don’t tweak it, even the slightest UI code change can cause it to break. Worse, a decision to revamp the interface can break your entire library of recorded scripts!
Be willing to accept that the recording feature of any testing tool is a minimal first step. Don’t let the sales people mislead you into thinking it is the holy grail of testing. The tester needs to manage the recorded script and write additional code manually. Many tools are severely lacking in this area.
Scripted automation
Rather than rely on record and playback techniques you should incorporate a real programming mentality into your automation effort. Have your testers manage a series of testing scripts which are appropriately implemented as modules and functions. Handcrafted scripts allow for dynamic testing, such that every run of the script tests the application slightly differently than the last. In a layered system, these scripts remain usable even after UI changes.
A fully automated test system that goes through all the primary use cases of the website is a boon to product quality. You can safely deploy changes without breaking anything critical.
Be aware though that automating testing is much work, both in initial programming and maintenance. It also cannot cover the same depth and breadth as a trained manual tester in the same time frame. It’s to best to combine automation and manual testing.
A critical point here is that script writing is programming. This leaves a task of finding a qualified programmer who is also capable of writing quality test cases.
