Client: Answer Consulting
Project: ParentMail Upgrade
Background
ParentMail is a website used by schools to facilitate communication between the school and parents. ParentMail has a registered 3.5 million user base. A new version, ParentMail 2, was commissioned in 2011 to consolidate features that had been added over the year and make it easier to cater for individuals with differing. Answer Consulting were commissioned by PeopleWiz (owners of ParentMail) to develop the new website, and Autism Works were commissioned by Answer Consulting to carry out independent software testing.
Autism Works has been involved in two phases - the release of an original feature-enhanced site and the implementation on overhauled registration functions.
The Task
Autism Works was brought into testing of this first phase to introduce a more robust structure around the software testing and to free the Programming Team to focus on development rather than testing.
As is usual when initially looking at the system with “fresh eyes”, a number of defects were found at an early phase, largely because the issues had not been detected during component/integration testing, and/or regression bugs arose after the component was originally written.
Due to the pressing launch date, a further issue was that the website was being redeployed with updates on a frequent basis and sometimes several times in the same day. This meant that test data had to be created from scratch after every rebuild, and manually creating this data each time would have been heavily time-consuming.
Our Testing
Autism Works overcame this problem by creating a series of data preparation scripts that automated the steps, with minimal manual intervention. This made it possible to rapidly deploy automated test scripts in a minimal amount of time. It is estimated that this process saved Answer Consulting at least 30 minutes per deployment by handing this task to Autism Works. These scripts were shared with Answer to support their own component level testing.
Testing was primarily based on a set of regression test cases that had been agreed between Answer Consulting and PeopleWiz as the basis for acceptance testing. Autism Works supplemented these tests with its own tests in areas that, in the opinion of their Test Analysts, were believed to be problematic or high-risk. After three weeks of intensive testing, 150 bugs and issues were raised, including 19 rated as critical or blocker and 52 rated as major. Combined with bug fixing on Answer’s part, there was a noticeable improvement in the stability of the system.
Self-registration
Following Phase One testing, PeopleWiz took the decision to overhaul the system for registering users. The aim was to replace a labour-intensive system with a system where some basic data would be imported from an external data source. This new feature brought a number of consequential changes to existing features in the system, which the automated scripts highlighted.
During the next release Autism Works had a longer time-scale for testing and a longer period to make preparations. This enabled Autism Works to plan the testing in a structured manner, with a written set of step-by-step test scripts and a coverage matrix to ensure that all new functions, rules and requirements were tested at least once. The scripts were also designed to cover as many different combinations of the data likely to arise in the live environment. In total, 35 scripts were developed covering 165 test conditions.
The expected results were based on the change documentation agreed between Answer and PeopleWiz. Where necessary, this was supplemented by the Test Analysts examining the website, but only where this was compatible with system requirements. System behaviour was queried with Answer where there were doubts. During this preparation period, 48 bugs and issues were raised, of which 3 were critical or blockers and 14 were major.
Outcome
The formal testing phase took place on the 21st-22nd December. The results were variable, with some areas found to be largely error-free whilst other areas were discovered to have bugs in unexpected places, some of them causing significant problems. 20 bugs and issues were raised during these two days. None were critical or blockers but 9 were major.
The major issues raised were then fixed over the following few working days, with Autism Works re-testing and confirming these fixes.
During this post-test run period, 55 further bugs and issues were raised, including 21 major bugs, but the major issues raised here were also fixed and re-tested. Overall, based on the formal test run and the subsequent testing, the self-registration functions could be shown to be fit for beta deployment.
The regression tests were also run during this period, to check that no regression bugs had been introduced into the rest of the system. No significant regression issues were found during this testing.
Based on this testing, Answer Consulting was able to recommend deployment to the beta testing environment. This took place in late January and, so far, the system has been seen to operate reliably in the live environment.
And finally…
“Our engagement with Autism Works was based upon critical business need. With only a matter of a few weeks ahead of the launch of a major new service to 3.5 million registered users, we recognised the need to introduce a more robust structure around our software testing. The project was in-flight, the launch day set and so we needed an organisation that would step up with credible practices and a competent team. There was no time for any altruistic justification; Autism Works had to prove themselves from day one.
The commitment and professionalism from the whole team was instantly evident and they truly exceeded expectation. Significantly, we launched the service to plan and the quality and performance of the software has proved to be of the highest standards. We will continue to engage with Autism Works because they deliver; the fact that we are supporting a socially innovative business simply makes it that bit more rewarding.”
Gary Parlett
Managing Director
Answer Consulting