Mobile devices are built on a variety of fragmented platforms that include multiple versions of iOS, Android, Windows, BlackBerry, Symbian, Bada, Brew and dozens of others (as of today, there are 17 versions of Android alone published since its beta release at the end of 2007). Mobile devices themselves come in a vast variety of form factors, resolutions, processing capabilities and with a diverse range of sensors and interface options. Within this environment, bristling with differing hardware, software and connectivity capabilities are applications in three distinct flavours: Mobile Web Apps: These are server side apps that bring together the vast capabilities of the web and the functionality of touch-enabled smartphones. These apps are quick to deploy since they don’t need to go through approvals as they don’t depend on a mobile app store or marketplace for distribution. Native Mobile Apps: These are platform-specific applications designed for a device’s OS and firmware. They leverage the device’s sensors and capabilities. Typically, a native app developed for one mobile device will have to be re-worked and adapted for another device. Native apps may enjoy a higher level of security than web apps. Hybrid Mobile Apps: These apps are designed using web technologies (HTML5 being amongst the most popular) but run inside the device’s native container. A hybrid app is powerful because it exposes the device’s capabilities (say a camera or an accelerometer using an abstraction layer) to what is essentially a web app. This is not good news from a business point of view as usability, capability and security of hybrid mobile apps can be different across devices. For businesses this means an exceptionally large burden on testing mobile applications across devices and device versions.
Let’s consider some of the key questions – see figure
- On which devices, platforms and OS versions should my application be tested?
- How quickly can I get to the market?
- Even if I do a comprehensive test now, how do I prepare for future device and OS upgrades?
- What kind of tests should I run?
- Can the test scripts be reused across platforms? Will the scripts run on all the platforms?
- Can I use automation to run the scripts? Which automation tool should I opt for, given that they are as confusing as the mobile device landscape?
- How do I ensure end-user satisfaction on the application after the test?
- How quickly can the tests be done? How do I guarantee security of the application?
- How do I evaluate application performance? How can I also test the performance of the device environment (server side)?
- What will the test cost? It shouldn’t come as a surprise to discover that for adequate coverage with respect to mass markets, an application might have to get tested on 15 to 20 device combinations [Platform-Version-Device Type]. It is evident that testing applications for mobile environments calls for a completely different approach, infrastructure and expertise
In today’s highly fragmented, fast growing and dynamic mobile market it is tough for enterprises to own and operate mobile application testing infrastructure with high utilisation levels to justify the investment. It is best for businesses to work with a partner who can deliver testing as a service.
Such a partner would ideally have the required skills and experience and also the infrastructure to deliver a reliable testing service.
- Creating a comprehensive test strategy (device coverage, functional testing, tools etc).
- Solutions and tools for testing.
- Evaluation of onboard tools in enterprise environment.
- Automation capabilities to ensure best quality in shortest possible time. Scalable test infrastructure.
- Mobility test experts.
- Ready-to-customize business processes and use cases.
- On demand services.
- Total mobility validation.
- A global presence.