User Testing is what most companies fail at, and the consequences are huge.
To be completely honest, it has been awhile since I have poked around the User Application Testing (UAT) environment because I assumed things had gotten better since artificial intelligence (AI) has become part of the equation. My current position does not really involve app testing so it was “out of sight, out of mind.” However, a recent personal experience made me realize that it’s time to bring the subject back to the forefront. From my viewpoint, things are not better, they are worse.
If user testing is the most important part of development then user input is the most important aspect of UAT and the most overlooked. Let me explain with a real life experience.
I was installing an application that has an automatic calendar generator similar to the one seen in the image below. I was annoyed because I had to scroll or click the back arrow through about 50 pages to choose my birthyear. While this calendar is fine for viewing dates, it is not ideal for searching for them. I consider this to be a lack of knowledge on the developer’s part. They are choosing aesthetics over basic functionality.
I thought it was just a developer oversight and then it happened to me three times this weekend alone on different apps. Now, I am annoyed by the lack of care that goes into creating apps and for proper user testing. Developers are reusing code to speed up the coding process with little regard to wheter it works or not. It is a “fix it later” mentality that is costing billions of dollars in application development.
Applications are useless if subscribers have a bad user experience during sign up.
Proper testing requires beta testers to install and upload the app as if they were a new user. This step is often overlooked because the developer sets a controlled and ready made environment for quick access to the app. The tester will never see the signing up part of the process and will not report issues with it. It is the cause of many missed installation opportunities and is hard to fix. It can be years before any one even notices or decides to report it. If you already have the application installed, you will not even realize there is an issue. This also happens on pages and forms that are only accessed once. Updates and code changes can effect these and if no one ever tests them out, they go unnoticed until a new user tries to use it. Staff and management are usually using the version already installed and may not experience issues that a new user will. A good stradegy is to gather information from new users about their experience or have staff uninstall and reinstall applications to see the changes in realtime. To ensure that they are seeing things in real time, staff should do the install on a completely new device. This will provide a more realistic view because there will be no cookies or history from the older installation.
Honestly, if it was not for the fact that I needed the app, I would have not continued with the click, click, click to my birth year, and yes there were over fifty on three apps and I don’t have that kind of patience!
UAT is how companies protect themselves from bad development practices but they need to be more viligant and monitor the development team. Here are some ways to assist with proper testing protocols:
Controlled user testing enviroments should include a total application review from installation to removal from a variety of different users.
The data will not be reported the same way from an experienced user, versus a “newbie.”
Don’t let the development team pick the testers as the data can be manipulated.
It is always better if the company provides the users and does not rely on friends or family of the developer or staff involved with the data because this creates data bias. It is more effective to pay for individual user testers who will provide you with realistic feedback.
Creative way to do it: Provide a local testing day, people would love to come to an afternoon at your company for a small payout. In fact, it is a great opportunity for advertising if you throw in a fun and entertaining lunch party for the participants. Participants can meet the team, and this has more advertising power if they share the experience on their social platforms. This is a more creative and realistic way of testing because the users will experience the application as it was intended. I would do this as an additional marketing opportunity before the actual launch day.
Make sure you are provided with the raw data and let your company do the analysis or hire a contractor to do so.
I cannot express how important this step is. Developers will pick and choose the data that puts them into a favourable position and create a really impressive presentation of the “numbers.” They will often leave out the most important data such as failed clicks, forms that don’t work and other technical issues within the application. Raw data will provide you with a more realistic view of the numbers. There is a difference between one thousand over night downloads versus one thousand downloads that the user unistalled five minutes afterwards.
Proper testing should involve the entire process.
This means installing the application, inputting all important information and clicking on every page, link and form that exists. Of course you can provide fake names and data, this is not the crucial part, it is how the application responds to the inputting that matters. These “fake” accounts should be removed before launch as they will provide improper subscriber data and also boost the numbers which is unrealistic and fraudulent at best.
Testing in a live environment is not safe and is the reason that many developers provide a sandbox. While this is obvious, they often forget to do live testing before launch.
In a rush to go live a developer will say we can fix that later, it is not that important. While some bugs are not that important, I think inputting your information during the sign on stage is a crucial aspect that should never be placed into the “fix bug later” category. It is better to wait on going live than potentially losing millions of subscribers because they are frusted before they even get into the application.
It is not just a developer problem, it is also a lack of due dilligence on the part of the application owner. Companies hire developers because they lack the technical knowledge to create the application. This lack of knowledge makes them unable to question what the developer is doing. This is a costly mistake. Developers will try to hijack the product or trap you into thinking that you can only use them to make updates and fix the application even though this is not required. View your contract to ensure this is NOT a requirement. If your application cannot be fixed by an independant developer, this is a RED FLAG. Good clean code should be easy to read and fixed by anyone who is provided the proper security protocols.
You must hire a Product Manager or UAT expert like me to be the “go-between” to ensure you understand what the developer is actually proposing and to make sure all deadlines and features are as they should be. And, you need to listen because they are the ones working for you. A developer is not working for you, they are working for thier own company and do not really care how the application makes you money once they hand it over. Developers only care about providing it to you as fast as possible and ensuring that they have continued income from it. Companies are often taken advantage of and this is the real downfall of application development.
Developers also tend to take on too many contracts at the same time which causes them to over promise and under deliver because they are constantly working on a “who screams the loudest schedule.”
To be honest, I would rather spend 75000 on an independant UAT stradegy than to lose 50000 on missing installations and have my application held hostage. Marketing is all about the numbers. We do not always think about these kinds of numbers, but we should.
Madeline Foster 
You must be logged in to post a comment.