A common request made by product owners and project managers is for test automation tooling which manual test staff can use to build up tests. However, in some circumstances, there are potential difficulties with using this approach.
Coding and customisation is often unavoidable
Unless you’re operating in the following environment, you will very likely require a level of coding or customisation to any record and playback automation:
- Your developers have uniquely identified all UI elements with clear simple IDs which infrequently or never change.
- Your developers are only using standard UI components which can all be detected and handled without issue.
- You have no dynamic elements to your UI.
- Your test verifications are limited to simple black box frontend checks.
Common causes for the need to code or interact technically with record and playback automation are:
- UI complexities exist in the systems under test which require customised code to carry out interactions.
- Test verification complexities which are not supported out of the box by the tool. For example, validating the content of outputted files in the file system, listening for and verifying messages captured between integrated system components, and validating data in backend systems and databases.
- To carry out verifications in a codeless automation approach, it is very common to employ regular expressions to parse screen contents. This is a technical concept which non-technical resources may struggle to master.
- Interactions are required with legacy systems not supported by the too. For example, mainframe terminal software.
It is worth noting that many of the premium record and playback tools can be very difficult to customise, and may require assistance from the vendor at additional cost.
The brittle nature of recorded automation
Record and playback automation is notoriously brittle and has a far higher ongoing maintenance cost over well-developed coded test automation. Over time, the maintenance of brittle automation will drastically affect your return on investment.
The reasons that many record and playback tools create brittle automation include:
- Recorded automation will find the easiest way to identify elements rather than the most robust and intelligent way. In some cases, premium tools give you a UI-based tool to select the method of element locator. However, this again requires a technical understanding of how applications under test are developed.
- Recorded automation often doesn’t consider reusability and records steps or code procedurally. Maintenance tends to be a re-record of entire test cases rather than simple tweaks to reusable modules.
- For record and playback tools which store a repository of element locators, the recording method often records these repository items in an unmanageable fashion which easily gets out of control as the code base expands. On the other hand, a well-developed approach will build up an understandable and easily managed object repository.
- For record and playback tools which don’t store a repository of elements or modularise scripts, a huge amount of waste is incurred as locators for elements are stored in each and every test instance, which greatly increases the maintenance cost and can result in large scale re-recording of tests.
- Most record and playback tools are generating code in the backend rather than having a specialist develop it. This automatically generated code is not developed in a best practice fashion and is not easily maintained, which may require it to be completely replaced with new recordings rather than minor tweaks.
White box system knowledge requirement for robust automation
Unless the tests being developed are simple navigation tests or only require very simple black box verifications of the UI frontend, there is likely going to be a need for a technical resource who can analyse the system at a white box level. Some examples of this might be:
- Understanding how applications are developed to identify the best locators for UI elements to improve longevity and robustness of tests.
- Understanding how system components integrate so that intelligent verifications can be conducted at various system layers.
- Having the necessary knowledge and understanding of how to connect to or communicate with various system components to carry out the required verifications.
Technical complexities of automation tool user interfaces
Even the premium record and playback tools have complex user interfaces with steep learning curves. Expensive training is often required to master their usage.
Although these tools are essentially codeless, most utilise code concepts such as modularisation and reuse. These are best leveraged by technical resources who have previous experience with these concepts.
Source control
Source control is a necessary part of test automation. In some cases, premium test automation tools have source control built into their product. However, in a majority of toolsets, especially the cheaper and open source tools, source control responsibility is passed onto the test automation developer.
Source control itself is a technical endeavour which is quite often botched by experienced developers. Expecting manual testers to carry out this function is a big ask and will require extensive training and support when issues arise.
If the source is controlled poorly by inexperienced users, using a poor practice approach can lead to a high likelihood of lost or corrupted tests. There is also the risk of losing large amounts of work.
The concepts of branching, pulling, pushing, merging, and resolving conflicts is a very foreign concept to non-technical resources. For that reason, it is best handled by resources with technical understanding of code development and source control best practices.
Integration with third party software
Integration with third party systems, such as CI and test management tools, is potentially very difficult for non-technical users if the record and playback tooling doesn’t come with inbuilt support or plugins.
Even for technical resources, integrating record and playback tools with third party systems is potentially prohibitively difficult, as it often relies on dealing with proprietary APIs and is often handed off to the vendor to customise at a high cost.
The value of technical resourcing
So what value does having technical resourcing add to your test automation development? For one, a technical resource will be able to develop robust test automation code.
This will avoid the normal brittleness pitfalls associated with record and playback. Best practice coding techniques can be used to make your automation less brittle and more maintainable, which in turn reduces costs and increases return on investment.
They will be able to identify the best methods to interact with the systems under test. This will make your test automation more robust and increase the longevity of the tests.
They will be able to develop technical verifications to interact with your system under test at various layers. This helps to increase your test coverage exponentially.
Technical resources that understand your system at a white box level will enable your automated tests to verify your application more thoroughly. It will also allow for the use of open source tooling to drastically reduce ongoing licensing and support costs.
Using open source tooling opens up avenues to a multitude of value adding options, such as cloud-based cross-browser and -device test coverage, parallel test execution, and simpler integration with third party systems such as test management and CI tools.
Finally, the knowledge and experience of technical resources in code development and source control will greatly reduce the risk of lost or corrupted tests from poor source control practices.
Start automating today
Test automation has become not only beneficial but an essential part of building and releasing quality software, but less than a third of Australian organisations have integrated automation as a key part of their SDLC. As for the reason why, it’s because automation is hard to get right.
If you haven’t yet realised the full potential of test automation, visit our services section to see how we can help you realise the benefits today.