Our mistake
As we were developing our induction cooktop aiming to help users learn to cook through a guided cooking experience, we knew we'd have a small display and control panel to give the key information and control for the steps. However, as we designed the first fully working prototype, we created a design on Adobe XD, handed it off to the developers and they made it. It was great seeing our design functioning and while we were involved with the developer team to ensure the design was coming out as intended, we never really tested it with context.
When the time came around to start our internal testing, we quickly realized that while the interface was easily readable and navigable on our PCs, the usability was very different while we were cooking. We were standing in front of the kitchen counter, our food burning, and with panic slowly rising we had to read the small text and find out what to do next. It was clear the HMI needed some change.
So after further testing (now with a prototype being used as intended) we gathered our data and went to look for an interface redesign. The difference this time was that we did much more realistic testing than before.
Work with what you have
While we wanted to test the experience as practically as possible, we obviously could not develop another working prototype. It is an unnecessary use of resources. So we created a short plan to simulate the entire process. We didn't want to "fake the cooking", we needed it to feel as realistic as possible.
So we created the entire recipe, screen by screen, prototyped it as though it were fully functional, and put the prototype on our phone to become our simulated control panel. We created a small test setup, building a platform to be close to a standard kitchen counter, and placed an induction cooktop on it and the phone with the prototype in the front. We asked our users to use the phone as though it was the control panel and follow the instructions to cook the selected recipe. The moderator was changing the temperature of the induction cooktop according to the step the user was on, simulating the auto-temperature control feature of our device.
This gave us much-needed validation about our redesign and allowed us to make a final few tweaks to the design before handing it off for development. Making sure everything from the lighting conditions, posture, size of the display & text, and actions the users are doing is very important to simulate when you are testing your HMI.
Inclusive Testing
Anthropometrics can play a huge role in the quality of interactions with an HMI. Whether the height difference for viewing the display or pressing a button, the experience can vastly change between people. Testing this early will also show you how accessible your design is for different sizes of people.
Identifying which metrics would make the experience change should be done before your testing to see whether you need to bring in users of different sizes.
Plan, test, and test
Planning is everything when it comes to usability testing. This comes at the very start when you’re speaking to the stakeholders for estimating timelines. Not rushing through or reducing the scope of testing can sometimes cause your findings to be incomplete.
It always helps to do a quick pilot test in similar conditions to test out your processes, setup, etc. Almost like a dress rehearsal of a play. This gives you a dry run to make sure your tasks and questions are easy to understand by someone looking at your HMI for the first time.
Some prerequisites before starting your test:
Your moderator and preferably an observer.
A location and setup that can simulate the environment and tasks the users would realistically do.
A test plan which tasks the participants with realistic scenarios, ideally not “simulated” but performed.
Participants who are from your user groups with a variation of whichever anthropometric you’ve narrowed down on.
In summary
Usability testing benefits teams to get answers quickly and cost-effectively. It can strengthen your design by allowing you to fail faster (catch mistakes or UX issues early).
One of the fundamentals of our UX Team here at Futuristic Labs is to iterate quickly, test, and refine. It becomes so much more crucial when the cost of the mistake gets higher (i.e. longer development time for changes). We ensure that our HMI usability testing is realistic and contextual, providing us with a far more user-centric design.