top of page
Engage VR
Usability Test & Redesign

ENGAGE is a software platform that allows businesses and educators to create VR meetings, presentations, courses and events. 

While VR technology is becoming more common, it's still a relatively new way for people to interact. This usability test is designed to see if users with limited VR experience can still navigate ENGAGE effectively.  It also suggests recommendations for improvements that ensure the platform is intuitive, user-friendly, and ultimately leads to a smooth and engaging VR experience.

Goal

A usability test was conducted on Engage, a professional metaverse platform, to assess its ease of use, satisfaction, utility, and memorability on VR headsets and smartphones. The evaluation aimed to simplify the user journey, improve information architecture, develop guidelines for creating virtual spaces, and identify missing features as per user expectations.

Method

The Common Industry Format (CIF), an ISO standard, was employed as a summative evaluation post-establishment of a functional prototype. CIF provides a foundational benchmark for assessing usability without prescribing specific improvements, yielding comprehensive, non-speculative data for empirical decision-making and redesign recommendations.

Additionally, this evaluation employed 'Think-aloud' method where participants verbalize their thoughts and experiences while performing tasks providing valuable qualitative insights into the participants' thought processes, rationale behind their actions, and areas of confusion or frustration as they navigated and interacted with Engage.

Frame 21 (2).webp
Metrics
User Demographics
To create a representative user group using stratified sampling, the demographics of both LinkedIn and the Metaverse have been considered. The 2022 YouGov report, titled ‘Unlocking the Metaverse’ (YouGov, 2022), and the 2022 Digimind report titled ‘LinkedIn: The Essential Figures in 2022 in the UK’(Nasr, 2022) provide insights into their respective audiences.
Group 97.webp
Test Results & Findings
  1. Participants encountered difficulties locating specific digital spaces, taking 407% longer than expert users. They struggled with navigation and determining the appropriate island, leading to frustration.
     

  2. Participants faced challenges in finding meaningful information within virtual spaces. They often resorted to using traditional 2D internet browsers for better efficiency.
     

  3. The onboarding process had a 90% completion rate, but the mobile experience was hindered by small keyboard size and unclear error feedback. Customizing avatars also presented difficulties for users.
     

  4. The low active user base on Engage resulted in limited interaction opportunities. Real-world conversational issues like turn-taking were mirrored on the platform. Non-playable characters (NPCs) caused confusion as participants mistook them for real people.
     

  5. Engage ranked in the bottom 15% on the System Usability Scale (SUS), indicating that participants perceived the platform as relatively difficult to use.
     

  6. The User Experience Questionnaire (UEQ) revealed discrepancies in participants' perceptions across devices. While participants held positive views about Engage on VR headsets, they had extremely negative opinions about the platform on mobile devices.
     

  7. Participants demonstrated consistent task completion times, but there was a significant gap between the performance of new users and expert users (152%), suggesting a steep learning curve for new users.
     

  8. The application suffered from severe shortcomings in perspicuity, with participants rating its ease of understanding and clarity considerably low.

Key Redesign Recommendations
Accessibility Audit
The task was executed following Saleem's persona. To simulate the persona's conditions—specifically, the inability to hear and difficulty with reading English due to differences with British Sign Language (BSL)—the researcher disabled the audio output of the application and set the language preferences to French. Although the researcher studied French for two years as a child and has since forgotten much of it, they could decipher sentences with moderate success and great effort just the way Saleem struggles understanding English.
OverallAudit.webp
Applied standard: Web Content Accessibility Guidelines (WCAG) 2.2
Full Audit.webp
Full Audit Header.webp
bottom of page