QA Tool Belt: Our Testing Devices

Written by: on May 16, 2013

When testing mobile applications, the sheer abundance of devices available on the market can present a nearly paralyzing situation before testing even begins.

What devices should we test on?

Only companies with small armies of testers and voluminous device libraries can hope to put a dent in the mountain of device and OS combinations out in the wild. Those of us in the trenches without such luxuries need to prioritize. After considering platform and OS support requirements, the next step in paring down the choices is to know the target audience. Who are you or your client attempting to reach? Tech-savvy early adopters staking out a place in line for the latest iDevice, or perhaps parents who retain their progeny on permanent IT duty? Whatever the target, it’s likely to help shift your focus toward either latest and greatest or legacy device support.

Knowing these basic criteria helps immensely in picking devices to focus on. While this varies from project to project, below is a list some of the devices we use and the reasons we keep coming back to them for testing.

The best device is the one you have.

The first and perhaps most obvious choice is to use your personal device. Hopefully if you’re testing apps you’ll be at least somewhat up to date with your hardware, and unless you’re testing software that could brick your daily, this is the easiest selection to make. You’ll be most familiar with the device you use every day, you know how long on average it takes to do things and because of this familiarity you’ll be able to spot issues rather quickly.

iPad 1 – iOS 5.1.1

The Apple crowd is often seen as the party of early adopters, quick to abandon old devices. However, this behavior is often aided by carrier subsidized upgrade perks which are largely unavailable for tablets. Even those who upgrade their phones like clockwork will find less incentive to update their tablet. In addition, as these devices still work well for many users they often find a second life as a hand-me-down. Aside from these demographic considerations, this is also a good device for testing UI responsiveness and application speed. Your application will reach its limits quickly on an iPad 1 and can provide early indication of where optimization may be needed.

iPad – 4th Generation

Always test your app to make sure it looks and runs great on the latest and greatest out of Cupertino. Don’t let those fancy Retina displays go to waste.

iPhone 5

The most obvious difference between the iPhone 5 and its predecessors is the extra screen real estate. UIs that have been designed for previous iPhones can sometimes look a little caddywompus on the 5. The anchoring and sizing of UI elements can cause an app’s appearance to differ between phones, and functional items such as how far drawers expand or what data is visible before scrolling may need some revisiting after being viewed on the larger screen.

iPhone 3GS

like the iPad 1, the longest running iPhone sold by Apple provides a good way to test for performance inefficiencies. The 3GS also has support for more iOS versions than any other iPhone, making the 3GS a great device on which to test your minimum supported iOS version.

Google Nexus device family

Co-developed by Google and various OEMs, the Nexus family of devices represent what Android should look and act like according to Google. With Android it can be easy to back yourself into the corner of writing support for specific devices, especially if a representative of your client has a particularly finicky device they are keen on having supported. However, if your app works on these devices, it’s likely to work elsewhere, and if it doesn’t, it is likely not your fault.

Amazon Kindle Fire, 2nd gen

While Amazon is mum about their user demographics, some independent research shows the Kindle demographics trending more towards female and older users than the iPad. In addition, the Kindle attracts more price-conscious users, so apps focused on frugal folks would do well to be deployed in the Amazon Appstore. This device also provides a way to test the Amazon Appstore, one of several departures the Kindle makes from vanilla Android tablets. Aside from the different app store, the Kindle OS is forked from regular Android and includes other differences such as the cloud-enhanced Silk browser and a different UI. One should not expect their app to run flawlessly on a Kindle without testing.

HTC EVO 4G – Android Gingerbread(2.3.3)

As almost half of Android users are still using Gingerbread, it’s a very good idea to keep a device hanging out with good old API level 10. The EVO 4G saw wide distribution during its deployment and is our go-to device for testing apps on an aging Android. In addition, many of these devices are still kicking around, and your app is likely to find a home on an EVO somewhere. This is also a great opportunity to find bugs related to the old version of HTC Sense that will inevitably pop up in production.

Samsung Galaxy Note

This family of comically large phones is gaining wide popularity, and while it may look odd to see someone smaller than LeBron James actually using one to make phone calls, it makes testing Android phone issues a bit more convenient.

With the device landscape ever changing, this list is sure to be obsolete in short order. Even so, I hope the reasons these devices were selected will help alleviate some initial testing paralysis well into the future.

Jesse Calkin

Jesse Calkin

Jesse Calkin is a QA Engineer at Double Encore.
Article


Add your voice to the discussion: