Continue with remote user testing or return to face to face?
As the restrictions on where we work are lifting, lots of us are reflecting on the ways we used to work and how we will work in the future.
For us in SDS one of the changes we made during the COVID 19 restrictions was to deliver all our user testing remotely, rather than sitting down in an office next to users. In this post we take a brief look at the pros and cons to this change.
Remote user testing
Remote user testing is a method of research that uses applications such as MS Teams and Zoom to record the screen, voice and video of test participants as they interact with online services in their natural environment- at home, in their office, or elsewhere.
The Pros
We have found there are many benefits to delivering user testing remotely:
- The tests can take place in the user’s natural environment. This means you are more likely to capture a realistic and natural experience of how customers use your services. Users could be more inclined to give honest and unbiased feedback whilst sitting at home rather than in a council office.
- Easier to recruit participants. With remote user testing we can be more flexible about when we do the tests and can arrange them at a time that suit the customers. It is also easier for the user to participate as they don’t have to take the time to travel to council buildings to participate.
- Users can also be recruited from a much wider geographical area as there is no need for them to travel. We have found that because of this we have not needed to provide the customers with any incentives, shopping voucher etc, to encourage them to take part.
- Faster and more cost-efficient. We have found we can complete more tests in a shorter amount of time remotely as there is no need for us to travel to councils, set up the testing room, give time between sessions for changeover of users, and no wasted time if users do not make their appointment. This time saving has reduced our costs too.
- Working in a more iterative way. Because remote testing has enabled us to do more tests in a shorter amount of time and increased the flexibility of when we do them it is much easier for us to run some tests report the findings and then run some more tests once the changes have been made.
The Cons
However, there are some disadvantages to delivering user testing remotely:
- Assumes a certain amount of digital knowledge. The very fact that we are asking users to participate over MS Teams or Zoom, turn their cameras on, share their screen etc. means they already have a certain amount of digital knowledge. In some cases we have been able to work around this and speak to those with less digital skills by asking if a family member or friend could set them up to do the test, but this is not always easy for people to find.
- Not able to test experiences of using mobiles. When in a room with someone we can hand them a mobile phone or ask them to complete a task on their own mobile phone and observe them doing it. We haven’t been able to do this yet with remote user testing.
- Missing facial expressions and other emotions. We have found it is not quite as easy over camera to pick up users feelings as they progress through a scenario or task. It can be harder to see their expressions or notice any sighs or see their body language.
Because of these pros and cons we are very happy to carry out user testing in any way you feel it would best suit your customers and your organisation. Please do get in touch and we can plan the best method for you to get the most out of your user testing.