Hussman School Usability Test

*Disclaimer: This is a class group project based on UNC Hussman's mobile website features on Feb.2023

01 | Goals

To evaluate the accessibility of the website on a mobile device.

Pre-Experiment Survey

Project

Duration

My Role

To determine how simple it is for users to find the information they need on the website.

02 | Summary

We conducted usability testing on the UNC Hussman school website to assess the usability of the website and its ability to engage the audience with the content and function and to fulfill the user's needs.

What we did:

  • Conduct usability testing on 2 voluntary participants per person

  • Measure user success and failure with the tasks

  • Discover difficulties and negative issues impacting user experience

  • Analyze the results and give recommendations based on testing outcome

Overall Findings:

Positive:

  • Color Scheme

  • Contact Information

03 | Methodology

Informed Consent Form

The pre-experiment survey collected demographic information about each participant including factors like age, gender, race, and affiliation with UNC-CH. It also inquired about their prior experience with the website, specifically whether they were or had been a MEJO major at UNC. Participants were additionally prompted to assess the significance of various accesses, such as academic advising and career services, using a rating scale ranging from 1 to 5.

Free Observation

Participants were instructed to access the website on their mobile phones using the provided link (http://hussman.unc.edu/). In the subsequent free observation phase, each participant was granted approximately 3 minutes to freely explore and acquaint themselves with the website. Participants were ewere asked to think out loud during this exploration, while the test administrator observed and documented their interactions.

Post-Experiment Survey

The post-test survey prompted participants to provide their opinions and evaluations regarding their interaction with the app. This survey featured quantitative questions, requiring participants to rate five app-related statements on a scale of 1 to 5, reflecting their overall experience.

04 | Participant Profile

UX Research | Usability Testing

February 2023 (3 weeks)

UX Researcher

To generate insights on how to improve the website's usability and user-friendliness on a phone.

Overall Task Success Rate

64%

Information Gathering:

  • Test Conducted 2/10-2/19, 2023

  • 6 participants were tested

  • Details, Quotes, and information were tracked through forms created by the testing group

Negative:

  • Violates Hick's Law

  • Not Responsive

  • Confusing Menus

  • Dead Links

  • Useful Information was difficult for users to find

  • Useless Search Function

We conducted a study to collect demographic and behavioral information and prior experience with the app of six volunteer participants.

All sessions were conducted virtually using Zoom. Participants were requested to join via their smartphones, enabling observation and screen recording of their actions. Each test session lasted around 40-45 minutes. The usability testing structure and tasks were influenced by research from Poynter (eye tracking study) and Steve Krug to ensure effectiveness.

The usability test consisted of five parts: Pre Test Survey, Free Observation, Assigned Tasks, Post Test Survey, and Post Test Questions.

Prior to the commencement of the test, all participants were required to review and provide their signature on a consent form. This document elucidated the usability test's objective, procedures, and assured the confidentiality of the test, thereby ensuring their understanding and agreement before their participation.

Usability Tasks

Participants were given ten primary tasks related to the website, aimed at identifying potential enhancements for its functionality and design. They were required to complete these tasks while using the website, with the intention of evaluating its user-friendliness, ease of navigation, information structure, and capacity to engage users. If participants encountered difficulties or were unable to accomplish a task within three minutes, they were prompted to move on to the next task. Inability to complete tasks within the given time frame was regarded as a usability concern.

Post-Experiment Interview

The post-test questions aimed to gather qualitative feedback on the user experience, with each participant receiving 10 questions. These questions covered topics such as overall impressions, preferred features, least favorite aspects, desired additional functions, and the likelihood of recommending the app to friends or family members, among others.

A total of six individuals took part in the study, one of whom was a female aged between 21 and 23. None of them were UNC Media and Journalism students, but all had either an ongoing or completed college degree. Interestingly, none of the six participants had visited the Hussman School's website, though they all possessed experience with other official school websites.

Gender
Affiliation
Race

05 | Limitations

Network Connectivity

Throughout the usability test, instances of network connection disruptions were encountered. Sluggish network speeds and intermittent connectivity led to extended loading periods and occasional disconnections during user utterances via Zoom.

Testing Fatigue

The majority of participants are navigating a website while being observed for the first time, potentially causing them to lose engagement during the usability test due to their limited interest in the website. This disengagement impacted their performance, resulting in incomplete or possibly inaccurate data.

Contact Information

Sample Size

The usability test comprised only 6 participants, a potentially inadequate number to establish a representative sample for a complex, critical, and well-established website. This limited scope might not fully capture the breadth of user opinions and experiences, consequently constraining the generalizability of the test results.

Users struggled to vocalize their thoughts while using the website, potentially diminishing the quality of their feedback. Moreover, external influences like the presence of the testing team might impact the sincerity and accuracy of users' expressed thoughts.

All of the users that were tested found the contact information at the bottom of the screen easily and completed that task very quickly. The contact information can be found at the bottom of most websites, so the placement of this information seemed intuitive for the participants.

"The contact information is easy-to-find and provides multiple ways to get in touch, such as a phone number, email address, and other social media."

— User 1

The color scheme of the website was said to be very well put-together and shows a good representation of UNC in the form of a website. The colors are also well-contrasted so that it allows the user to see information clearly.

Recruiting Bias

The participant pool predominantly comprises recent college graduates and students, with 50% from UNC-Chapel Hill, thus lacking diversity and broader representation for comprehensive website testing. Consequently, potential biases in the results might not accurately mirror the experiences of all potential users, including groups like current high school students.

06 | Positive Findings

1

User Feedback Limitations

Color Scheme

07 | MINOR PROBLEMS

2

"I really love the blue and white color scheme used on the website. I especially like how the blue call-to-action buttons stand out against the white background, making them easy to find and click."

— User 3

The "Diversity Program" button lacks a functional link, failing to open a new page or trigger any subsequent action upon tapping. Several participants highlighted the issue of dead links, finding it perplexing for users.

Dead Links / Buttons

1

“I was really confused with some of the links that cannot be clicked.”

— User 6

08 | MAJOR PROBLEMS

Mobile Layout Incompatibility

1

Course Offerings

09 | Catastrophic Problems

1

  • 6/6 participants were able to eventually locate a course list, but all expressed confusion and/or frustration.

  • Only 1/6 navigated to the full MEJO catalog, and the majority navigated to the academic worksheet or the core/concentration/conceptional/capstone section.

"I would say that was one of the more confusing ones and least efficient."

— User 4

"I think a prospective student needs to understand what the difference is between a core, concentration, conceptional, and capstone."

— User 3

Scholarships

2

  • 5/6 participants were unable to locate a specific scholarship 4/5 of these participants were unable to locate the scholarship list at all.

  • The 2 participants who did locate the full list expressed frustration about having to click through the individual scholarships.

"I wish it was more like a drop-down, quick description than taking me to a whole other link."

— User 5

Global Programs

3

The application deadline information extends beyond the boundaries of the mobile device layout, necessitating users to scroll in order to access the complete details.

  • 5/6 participants failed to locate the full spreadsheet of global programs:

    • 2/5 of these participants failed to find the global programs page at all.

    • 3/5 navigated to the global programs page, but didn't open the full program list.

  • Participants had to click on many options and drop-down menus before they could locate the global programs page.

"I just can't find the section of study abroad programs."

— User 2

10 | Finding Summary

  Based on post-experiment survey

Overall ability to navigate the site

4 / 5

Overall impression

3 / 5

Interface

3.67 / 5

Site aesthetic

3 / 5

Engagement of the site

3.5 / 5

11 | Recommendations

Change the home page design

Participants observed that the content showcased on the home page lacks strong relevance to the website's primary objective. A suggested improvement involves initially presenting an outline of departments, degrees, and social media links, followed by news and events. Enhancing font boldness for improved readability is also recommended.

Create a study abroad portal

Participants identified the absence of a study abroad portal on the website, causing challenges in locating school-provided opportunities and hindering their information-seeking process. A recommended solution involves incorporating a study abroad portal that comprehensively presents all available options and opportunities for user convenience.

Create a course offering tab

Numerous participants voiced difficulty in locating a comprehensive course list. The absence of a clear display of all school course offerings on the website was noted. To address this, consider implementing a dedicated "Courses" tab similar to the structure found on sils.unc.edu/programs/undergraduate/bsis for improved accessibility.

Mean Score

Make information more straightforward

Participants valued the website's comprehensive information but suggested improving its organization, breaking content into distinct sections, and streamlining the presentation for easier comprehension of essential details.