In the highly competitive digital world of mobile applications the ability to get high quality apps to market quickly can make or break the success of a product or company. With new applications for iOS, Android and Windows platforms being released daily, all battling for media attention and consumer dollars, the pressure to get apps built, tested and deployed has never been greater.

Testing methods that have worked previously for web and desktop applications do not meet the needs of mobile applications as Testing Matrices and fragmentation have become far too complex.

Companies must now carefully consider both usability and functional requirements and test their applications across different devices, models, networks, operating systems, browsers and locations.

This calls for a new approach, one that presents benefits and challenges.

Key Points:

• Pressure to get mobile apps built, tested and launched quickly and effectively has never been higher.
• Testing methods that have worked for web and desktop apps fall short for mobile applications.
• A new approach is required specifically for Mobile Testing.

Challenges 

Mobile applications present many challenges for today’s test teams, many of which differ from traditional projects. Defining the scope of mobile testing must take into consideration the following:

Devices and Operating Systems

With a mass of device makers each producing a multitude of devices released year after year, test teams must carefully consider the various operating systems, screen sizes and hardware specific to each of these devices. In doing so, they need an adequate coverage of handsets to test against including various generations of operating system updates. Testing must pinpoint target users of the app under test, prioritising their chosen devices, which includes being up-to-date with the most popular devices in the market.

Networks

Networks are an important and complex consideration as mobile applications depend on both Wi-Fi and 3/4G connections. Network test scenarios should cater for offline activities as well as interruption to calls, alerts and low battery, testing the data connection and the expected responses. Test teams should also consider the network provider, as each has a different network infrastructure which can also affect performance and user experience.

Application Types

There are 3 types of mobile applications each with its own set of challenges for test teams:

• Native apps are client-side apps downloaded from stores onto the device. These require dedicated testing effort across each platform/language that the app is designed and coded.
• Web apps are mobile optimised webpages, which access websites through a mobile device’s browser. These require tests to be run on different types of browsers on devices connected via a mobile or Wi-Fi network.
• Hybrid apps are a combination of both native and web, which run on the device and are platform dependent. While the code remains the same there are different factors that come into play for testing due to the need to address characteristics of both native and web apps.

Strategies 

Development Methodologies play an important part in defining a mobile test strategy.

Mobile Testers need to be highly skilled and practiced in agile techniques. Agile and mobile development practices tend to go hand-in-hand due to technology advances in the digital space. Flexibility and responding to market changes are key advantages of following the agile framework, therefore enabling faster speed to market.

Risk-based and Attack-based Test Strategies need to be considered for mobile testing, evolving through the lifecycle and focusing on early testing to unearth defects during development.

Exploratory testing provides feedback loops to teams, with testers determining if more or less test attack type testing is needed, constantly assessing risks refining scope and going beyond basic user story and requirements validation.

Effectively leveraging Test Automation as part of a mobile test plan provides the ability to continuously test and run the same test case over multiple platforms and devices. This saves testers valuable time, increases quality and improves the team’s ability to correctly repeat tests without errors in test script execution.

Performance and security testing are also key components to a clearly defined mobile test strategy.

Tests should be executed on real devices to maximise confidence for product release while also leveraging emulators early in the lifecycle or remotely connect to devices and web browsers in the cloud.

Conclusion

Mobile development has fast become mainstream and a top choice for organisations as mobile penetration approaches 100% market coverage. In fact, as mobile sites and apps are increasingly considered as business critical, it stands to reason why mobile testing has become such an important part of the mobile evolution.

Mobile applications are a game changing force for companies across all industries. But for all their benefits they also hold significant challenges and risks, making it crucial that organisations adopt and apply the correct testing techniques and strategies.