eHealth is a rapidly growing industry that connects and improves the quality of life of patients around the world. After seeing an emerging opportunity for digital health, an existing organisation in Australia saw the opportunity to be in the forefront of this space.

Our customer established their digital ehealth company, to build custom solutions for health and aged care clients, effectively integrating the right solutions into their business process. Our client works closely with the Government and industry to bring the technology, solutions, and analytics to healthcare workers, hospitals, pharmacies, and health funds.

Our client quickly grew as they acquired several digital health businesses and the talent that established them. This meant the new company went from a group of start-ups to one of Australia’s largest eHealth companies, with hundreds of people spread across the world in its first year alone.

Our client’s rapid growth from several acquisitions meant that different testing and QA practices were used, which were now resulting in time and monetary inefficiencies. The company knew it could be implementing quality practices in a better way, and had experienced some significant quality issues in production, but it did not have the skills and know-how to improve testing in-house.

Our client sought an external quality partner to objectively assess its quality practices and identify areas for improvement. That would also need to guide their staff to better ways of working for testing and QA.

We were engaged by our client to conduct a whole-of-lifecycle quality assessment across the product lifecycle, to identify how team practices could be improved to uplift product quality and improve user experience. The review included an assessment of QA of requirements (user stories), QA and testing during development, independent local and offshore testing, practices used by Operations and Support teams, and the Agile and DevOps methodologies that were in used across the organisation.

The assessment included interviews with our staff that were responsible for requirements, development, testing, management and support. This include testers, business analyst (BAs), developers, product owners, operations, and business and product managers. It also included attendance to team meetings including agile ceremonies, and a review of key test and QA documentation.

Following the interviews and reviews, we proposed a wide range of recommendations for uplifting product quality and delivery most of which were implemented throughout the year.

The following provides a summary of 10 key improvements we recommended:

• Create an enterprise/organisational test strategy and policy that defines agreed practices for testing and QA throughout the organisation, across all types and levels of testing
• Shift quality left by conducting product quality risk assessments during release test planning, to ensure development and testing focuses on quality characteristics that matter the most to customers
• Shift quality left by commencing non-functional testing earlier and continuously, and making non-functional requirements and testing visible throughout the sprint
• Optimise the regression test suite to reduce the number of regression tests down to a more manageable number, and commence automation of the test suite
• Make automated unit tests visible to balance manual system testing effort, and introduce automated API testing database backup and recovery procedures and ensure customers are backing up their databases regularly, and introduce usability testing to improve customer experience
• Conduct regular exploratory testing to expand the likelihood of detecting defects, capturing test ideas for future reuse reduce separation between development and operations team, and between onshore and offshore teams, to production incident fix turnaround times and improve customer experience
• Improve structure to enable shorter release cycles
• Improve production monitoring and alerting to allow production incidents to be automatically identified and self-healed

Our client was enthused by the opportunity to uplift quality and accelerate delivery, and assigned their most senior QA Lead responsibility for implementing the improvements.

We were then re-engaged by our client at the end of the year, to review whether our recommendations had led to improvements in product quality and user experience, and whether any new improvements were required.

We found our client had implemented significant improvements throughout their lifecycle that resulted in a significant and measurable improvements in product quality:

• The organisation experienced no critical defects in production environments following each new release
• There was a significant improvement in software quality, as evidenced by the very substantial reduction in the number of defects detected before and after production release. The number of defects detected in-sprint reduced by 87%, the number detected during internal (alpha) UAT reduced by 94%, and the number detected during external (beta) UAT reduced by 85%
• The size of the regression test suite was reduced by 75%, with the updated test suite achieving full feature coverage in a far shorter time period, and with many tests now automated
• Quality risk assessments were being conducted during release test planning, making non-functional requirements visible and with testers taking a lead role in the planning.
• The connection between development and operations team, and between onshore and offshore teams, has vastly improved, with offshore testers visible and allocated to product teams, and regularly attending Agile ceremonies
• Release cycles and production incident fix timelines had both shortened, enabling faster time-to-market
• Different agile teams were achieving greater consistency and efficiency in their processes