At a time when CIOs are crying out for more funding, a report has revealed Australia’s large organisations are bleeding software development projects at an average rate of $86.7 million each year.

The inaugural Planit Testing Index, commissioned by independent, Australian software testing organisation, Planit, surveyed 131 large organisations in Australia (mostly corporations in the finance, insurance and telecommunications sectors, plus government organisations). Analysis was conducted on both the total number of projects commenced by respondents, as well as their most important project over the past two years.

Chris Carter, Planit’s managing director, is also the president of the Australian/New Zealand Testing Board (ANZTB) and secretary of the International Software Testing Qualifications Board (ISTQB). Carter says, according to the Planit report, more than half of all Australian software projects run over time and over budget.

“On average, companies start 38 projects annually, less than half of which (42 per cent) are completed on time and on budget, while six per cent are cancelled altogether,” says Carter. “At an average cost of $113,733 per week, projects not completed within the set timeframes are causing companies major budget blowouts.”

According to the Planit Testing Index, successful projects depend largely on management buy-in, quality staff and well-defined requirements. Of those projects completed on time and on budget, 78 per cent of respondents reported good or very good levels of management buy-in, 65 per cent rated the quality of staff as good or very good and 50 per cent felt the requirements definition was good or very good.

Carter agrees quality staff play a major factor in a project’s success, but says finding that staff is a real issue for companies as the strong economy and low unemployment rates combine to create a skills shortage.

“Organisations must invest large amounts of time and money into recruiting and retaining staff, or risk losing them to competitors,” he says. “While more than two thirds (69 per cent) of project work for the surveyed organisations was carried out by in-house resources, the skills shortage is forcing companies to also look at third parties as an alternative to employing staff on a full-time basis.”

The study revealed those companies with advanced testing techniques and methods successfully completed 60 per cent of their projects. This starkly contrasts with those organisations which have no planned or documented testing process and undertake testing in an ad hoc way; the latter group reported successful completion of just 29 per cent of their projects.

The Planit Testing Index showed that:

1. There is a direct link between the proportion of budget spent on test execution and a project’s success – companies spending less than 10 per cent on test execution completed only 13 per cent of their projects on time and on budget.
2. More than a quarter (26 per cent) of respondents still view testing as a “necessary evil”, while 11 per cent see it as either a cost to be minimised (6 per cent) or a low priority (5 per cent)
3. Almost three quarters of companies surveyed said they will increase their use of structured testing processes over the next 12 months, while 64 per cent will increase their use of testing tools
4. Companies are spending 19 per cent of their software project budgets on test execution
5. The average budget for “most important projects” is $15.9 million.

“The Planit Testing Index clearly shows that companies which undertake testing of their software projects, and plan for that testing from the beginning of a project, are more successful than those that don’t” says Carter. “The Index also clearly shows we are losing millions of dollars each year through unsuccessful projects – yet still there are development teams and entire companies out there who treat testing as an after thought, if at all.

“Is this situation caused by developers afraid to expose their work to the cold light of day? By CIOs who would rather spend their budget on developing software – good or bad – rather than perfecting it? Or by boards and senior management who, ignorant of what a software project needs to be successful, will not release enough budget to cover testing? I believe until we start addressing the issue of testing and systems assurance we are crippling software projects in Australia and limiting our chances at ever being known as the clever country.”

The Planit Testing Index will be conducted on an ongoing basis.