top of page

Usability testing

Testing EdPay features

March 2019

Key info

Project type

​Usability testing, user research​

​

Core team size

6

​​​

My role

  • Recruiting participants

  • Planning and facilitating usability testing.

  • Analysed data.​

​

Tools used

  • ​Pen and paper

  • Screen recorder

  • PowerPoint

Skills

  • Planning

  • Facilitating

  • Recruiting users

  • Training others on user testing.

  • Analysing data

  • Note taking

  • Communication

  • Risk and issue identification

  • Stakeholder management​

Background and problem

The task: Conduct usability testing to understand whether the product features were fit for purpose.

​

For the first time a new piece of payroll software has been developed at EdPay. This will pay 90,000 people, which is the largest payroll in the region.  There was however no usability testing approach and all facilitators and note-takers were new to usability testing.​

Process

Preparation

  • Note-takers, observers and facilitators were provided with training to help them understand best practices of usability testing to reduce the risk of biasing the data

  • Test scenarios were written with input from the product owner. Test scenarios reflected tasks that users would need to do in real life. 

  • Questions about the tasks were also included in the protocol to produce a better understand of the users' needs and wants.

Recruitment​

  • My team invited in users from four different schools from the region who represented the persona groups to test new features in the new interface. They were provided an information sheet and consent form via email. 

Data collection​

  • During the test itself, users were walked through the information sheet and consent form. Consent was obtained before audio recording equipment was turned on.

  • The screen was recorded to help developers to trouble shoot any issues

  • Users were asked to do a series of tasks and asked questions

  • Observations, answers, and themes were noted. Themes noted to help with the insight synthesis.

  • Data was collected from each sessio​​n

  • There was debrief after each session to talk about themes and issues that arose so we could make change to the sessions if required

  • A note taker, facilitator and observer (from the build team) were included in each session. Observers were invited to help them empathise with the user.

Test tasks

Findings

  • We found that users were overall happy with the product but wanted some slight modifications of the features and more features.

Delights and improvements identified during testing

Outcomes

  • Good data to inform enhancements to the product

  • Good rapport with users

  • Identified risks and defects with product and product roll out

  • Some of the feedback from users have been incorporated into the product. Most of the feedback was for enhancements as opposed to critical changes.

What I'd do differently

  • Included role play into the training sessions with my team to improve the quality of the test sessions

  • Asked more "why" questions during the test sessions to help with generating insights

bottom of page