Author's Note: This is part 2 in an ongoing series on the importance of Quality Assurance/Quality Control (QA/QC) on your website. How many posts will there be? I'm not really sure. You keep reading and I'll keep writing. Check out part 1 of my series.
QA/QC is tough. There are so many variables which you must test to ensure that something is going to work the way it's expected. But one thing that makes it infinitely tougher is something that is outside our control and that is the diversity of means by which users access content on the web. Desktops, laptops, tablets, smartphones, not-so-smartphones, web-enabled video game consoles, smart TVs, smartwatches... oh my! Add to that the variety of operating systems that run these devices and affect how a browser functions and renders webpage content. Then there are the browsers themselves: Chrome, Firefox, Safari, Edge, Internet Explorer, Opera, Navigator (please tell me at least one of you out there is still rockin' it Netscape style!).
So many variables for front-end and back-end devs to account for as well as for QA/QC to test against. It's mind boggling. It's also darned near impossible to account for all these variables during QA/QC. However, we must account for more than just the single computer and browser immediately in front of us.
That is where a "device lab" comes into play.
What is a Device Lab?
Per Destiny Montague and Lara Hogan, a device lab is "a highly functional way to provide handsets, tablets, and other Internet-capable devices to those interested in testing their work across a variety of screens and hardware."
No! These are not the devices your co-workers have been assigned by the company or their own personally-owned devices.
Yes! A lab should consist of devices dedicated to the sole purpose of allowing developers to check out how some design renders or how a new piece of code functions and to allow for device-specific QC of the site before releasing it to your client for UAT (User Acceptance Testing).
Sure, there are plenty of emulators out there that can provide an experience that is almost the same as using a physical device. But if you want to know how the website you're developing is truly going to look on Chrome on the most recent Samsung Galaxy s9 running Android Oreo or on a two-year-old iPad Mini on iOS 10.5, your safest bet is to use the actual device. There's simply no better way.
Be Smart About Your Lab
"No better way," however, can come at a cost and this is where you must do a little research. Your device lab does not need every single device running every single operating system and allowing testing on every single browser known to man. Unless, of course, you have an endless wealth of cash and space for storage.
I would advise the rest of us, though, to take a look at the analytics on sites that we are developing (assuming this is not your client's very first website) and create a list of the most common devices, operating systems, and browsers that access the site. You can also look at research from tech firms to find this data for the web in general.
From this research, try to develop a budget-conscious list of devices that make sense for your lab. Then go shopping! You can always start slow and build up from there. This is the approach that we're taking.
Diagram's Device Lab: A Work in Progress
Based on aggregate analytics of our clients and data found on the web when we began revisiting our device lab, we knew we wanted to upgrade with one iOS and one Android smartphone. But we opted to select models that are one generation older than what is currently available. The way we see it, not everyone is an early adopter of the "latest and greatest" so one model older is a pretty fair representation of our clients' current user bases. We applied the same logic to determine a new tablet addition to the lab.
We are also considering adding a modern Windows tablet with the latest version of both Internet Explorer and Edge. To access older versions, we are adding a subscription to BrowserStack so we can login -- even remotely -- to use a "real" version of older browsers (BrowserStack does not use emulators but actual virtual machines). Services such as this or Browserling or Sauce Labs might go against my above claim that real devices are always best. But in instances where a browser or operating system shows less usage, this is a good way to fill in the testing gaps.
Another important aspect of a device lab is how it's displayed and maintained within your office. Our physical display is currently just a sketch on a piece of paper. So more will be shared about this in a future post. Stay tuned!
Into the Future
Going forward, my plan is to revisit the device lab annually and add one new device that will be both a new type of device and, hopefully, a new operating system. Through constant vigilance, you can ensure that your device lab is up to standards with a majority of the web-browsing world and you can attempt to gain the maximum buyback value on deprecated devices that you wish to remove from it.
Got a lab? Tell me about it or, better yet, link to photos with details. I'd love to see them!
NOTE: This was part two of a three part series. Read the rest here:
Part One: QA & You: The Importance of Quality Control to Your Website
Part Three: QA & You: The Testing Handoff
