UX Designer and Researcher
Refining the Experience
Moderated User Testing for Pratt Institute's Career Center
Abstract
Pratt Institute's Center for Career and Professional Development (CCPD) is dedicated to providing services to students to help them build skills and prepare materials for job searching. This project used moderated user testing to identify usability issues on the Pratt CCPD website, and offered potential improvements to mitigate those issues. Findings were presented in a slide presentation and report format.
The Why
Getting to know our client, and setting expectations
The team for this project was myself, Nidhi Gowda, John Veon, and Betty Yang - all graduate students in the Information Experience Design graduate program at Pratt. We first met with our client, CCPD Director Marias Lobianco, over a Zoom call where she described her motivations and goals for the project. Her main questions were:
​
1. Are students able to access key functions of the CCPD website? (Such as creating resume and portfolio review appointments, or signing up for workshops/career fairs).
​
2. How do students access these features currently? Are they making use of the website, or are they going through other platforms such as Handshake or email?
​
Based on the insights gained from this meeting, we were able to understand the value of the CCPD and what they offer to Pratt students, as well as the key areas of the site that needed to be tested to ensure these services are actually accessible.
The Process
Collecting the data
One of the most important pieces of information we extracted from our meeting with our client was more details about potential users. After the meeting we were able to create a user profile to better understand who we were aiming to serve. Meet Jane:
​
​
​
​
(Figure 1)
After narrowing down our target group, we were able to begin creating a screener questionnaire that targeted students that fit our profile. We posted a link to this screener onto a flyer and hung copies around Pratt's Brooklyn Campus, and had our client assist us by sending an email blast out to undergraduate students through the platform Handshake. This screener was able to reach 46 undergraduate students at Pratt Institute. We selected 8 participants from this bunch, attempting to make our testing sample as diverse as possible to properly represent all the students who can make use of the CCPD services.
​
We then started to really outline what our study was going to look like. We knew it would be moderated, and decided to conduct our sessions remotely. Tasks needed to be created that aligned with the key functions our client wanted to test, which brought us to the following:
​
-
You're looking to apply to jobs soon, locate information on resume building/review services and check meeting availability
-
You're graduating soon and have been thinking about starting to apply for jobs within your field. See if the CCPD offers any events that could help you with this
-
One of your friends could make use of the services at the CCPD, and they mentioned being interested in seeing if any workshops are happening. See how the CCPD could help with this.
-
You need to update your portfolio materials for a job interview you have coming up, find information about portfolio services offered on the CCPD website
-
You are trying to find your alumnus for advice regarding your career, and see if the CCPD has any resources to foster these connections.
​
We then created a testing script, so all four moderators could run their testing sessions in the most uniform capacity. Finally, we ran a pilot study to ensure our testing could be done in a timely manner, which was a success - each testing session lasted around 30 minutes.
​
In our testing, a big takeaway we learned was that all our participants really valued the CCPD services, but had a lot of frustration with actually accessing them.
transforming data into insights
All evaluators placed their findings in a shared Google Sheet. We then all wrote down major findings from each participant into a shared Figma board, and began to analyze. Below you will find our affinity mapping diagram:
​
​
​
​
(Figure 2)
Going over our findings together as a team brought fruitful results. Many of our participants encountered the exact same issues, or very similar issues when completing their assigned tasks.
The Results
Qualitative Results
The following are our main qualitative takeaways from our testing sessions:
​
-
4/8 of our participants did not see value in the CCPD website, and do not plan on revisiting it.
-
3/8 of our participants said they would not use the CCPD website at all, and use Handshake instead.
-
2/8 participants stated during testing that they would never go through this process (the task) and would rather email someone for help.
-
Participants greatly valued the CCPD services, but the difficulty in accessing them is an issue.
-
Participants really desire help with their resume and portfolio, and felt a lot of frustration over how difficult it was to create an appointment.
​
Quatitative Results
The following table outlines our quantitative results from testing. (Reference for table: Averaged responses following each task. Lower numbers indicate better results - 1 is best, 5 is worst. Participants were given a warning at 3 minutes to wrap up).​
​
(Table 1)
Big Takeaway
By far, our most important insight was the following:
​
Participants really desire help with their resume and portfolio, and felt a lot of frustration over how difficult it was to create an appointment.
​
Pratt students feel like the most important thing they leave Pratt with is their portfolio, and the resume is of course key to securing a job after graduation. One of our recommendations really focused on making this process of scheduling appointments easier.
​
Our first recommendation involved condensing content of the CCPD website, adding proper labels, and making hyperlinks clearer to users. The before and after is posted below:
​
BEFORE
(Figure 3)
AFTER
(Figure 4)
Conclusions
presenting to our client
In total, the team was able to come up with 5 recommendations that we strongly believed would improve the usability of the CCPD web interface. These recommendations were presented in the form of a slide presentation to our client and other CCPD representatives, as well as in the form of a more detailed report. During the presentation we received great feedback from our client.
Firstly, our findings reinforced some feelings they already had about the website based on their interactions with students. For example, the CCPD receives a ton of emails from students asking for help with scheduling appointments, because the process is not intuitive. They stated that getting data to backup this feeling was valuable
​
Secondly, Pratt Institutes entire web presence follows a rigid template. That means the kind of recommendations that we made needed to fit within the guidelines of this existing template. Our client said we really did a great job with this, and that our recommendations were not only valuable, but also very feasible to implement.
​
You can access the teams final presentation and report below:
lessons from this experience
Overall, I found this project to be very rewarding - I worked with a great team who was really willing to put in the effort to make sure we were satisfied with our end results. Since I've transitioned from a studying psychology to studying design, something I can work on is making designs or ideas feel less personal to me. The switch into a more creative field makes it easy to feel like the projects I create are somehow apart of me, which just isn't true. In this project I learned when to push for my ideas to be implemented, and when to accept that they were not the best, and that the team had a better direction in mind. I think over the last few weeks I've become a much better designer and teammate because of this.