Usability Testing
Heuristic Evaluation
Sarah Stumme
Yubo Zhao
Ruijingya Tang
I drafted scenarios and survey questions, conducted interviews and/or took notes, created interaction map, completed heuristic evaluation and analyzed data to generate a report with actionable insights.
This was a client-based usability testing project for a graduate course at the University of Washington HCDE program. My team was paired with Column to conduct usability testing for their product and provide detailed report on findings areas of improvement.
Column.us is a platform designed to modernize and streamline the process of placing public notices and legal ads in newspapers. It acts as a bridge between newspapers and individuals or organizations looking to publish legally required public notices.
By digitizing the process, Column.us simplifies submission, management, and distribution, making it easier for users to meet legal requirements. It aims to bring transparency and accessibility to public notices, reducing the traditionally cumbersome process while helping local newspapers maintain a critical revenue stream.
Study goal
The purpose of this study was to test and evaluate the new account registration flow of Column’s web application, including filling out profile info, creating organizations, joining organizations, and adding teammates into organizations.Specifically, this study aimed to:
Before starting the test plan, we created an interaction map of the Column’s registration flow to better understand what users go through. Seeing visual representation of the flow allowed us to identify critical touch points and prioritize organization view over the individual one as it had more nuances that could have cause the three issues that initially were identified by Column: high drop off rate, “a lot of wrong email addresses,” and “Duplicate organizations that add downstream manual work.”
Click to zoom
Column customer consisted of both individuals and organizational users. After talking to the Column team, we decided to focus on the organization users, because the number of organizational users was higher and that was more aligned with their business strategy moving forward. We recruited 8 total participants, but to cover a wide range of users, we decided to focus on their 3 different sectors: Law, Government and small businesses.
Our tasks were designed to get participants to test both creating and joining a current organization as we had the assumptions that the user expectations of between those would be different. Therefore, four of our eight interviewees executed tasks to “join an organization,” while the other four did so to “create an organization.” For each of those participants, we gave them the scenario that best fit their profession.
At the end of each interview, we asked participants five concluding questions to assess their general impressions of the registration flow and help us identify the most severe problems that they encountered while registering.
1. How did you feel overall about the registration process?
2. What part of the process was unexpected or what did you expect to see that wasn’t there?
3. Which step of the process did you find the most challenging?
4. How do you feel about registering before creating your notice?
5. If you could change one thing about the process to make it easier for you, what would that be?
Analysis
To thoroughly analyze the qualitative data gathered during the usability testing of the Column platform, our team employed a technique known as rainbow spreadsheet analysis through which the coded qualitative insights are then color-coded to reveal patterns and trends.
Having two buttons side by side, users were confused about whether they should have directly selected “Place a Notice” or “Login” first. Some users expected the system to automatically lead them to registration after submitting the notice.
There is was no indication either on the dashboard or as an alert box about whether their request to join an organization was successful or pending. Additionally, no information about the next step was provided to the users.
The title of the sections did not match what the system required the users to do. Misleading guides made the user second guess their choices and prolong their flow. It was unclear to some users whether they had to input their personal or organization information.
When the users reached the teammate role selections, all users were confused on what each role meant. Additionally, they were not sure whether the roles were for Column or their own organization.
Users could not find how to delete the first entry. In order to delete it, they could only overwrite the items and delete the last one. This was problematic when there was multiple teammates in the queue to to be added to the organization.
