x
Testing The New API Learning Tool
The Cooper Hewitt API provides its users with access to data on 215,000 items that can be analyzed and pulled to create numerous interesting projects. A usability and eye-tracking study needed to be conducted for the API to be more discoverable and easy to use.
2023 (3 months)
Duration
MY Role
UX Strategist
THE TEAMMATES
Priyanka Nair
Alya Zouaoui
TOOLS USED
Figma
FigJam
Project Overview
The Cooper Hewitt API provides its users with access to the data of 215,000 items in their collection, along with information about the people, companies, and other organizations that were associated with them.
This project tested how novice and expert API users interact with the Cooper Hewitt API website by conducting a usability study and eye-tracking. We provided several recommendations focused on improving how people locate the API website (discoverability), find API examples (readability), and learn to run a query (learnability).
Our pitch meeting to the Cooper Hewitt developers and managers.(Left to Right: Kato, Phil, Laiba (Me))
THE CHALLENGE
Stakeholders worry about how the platform can teach users how to create projects from various skill levels because of the lack of discoverability.
METHODOLOGY
TASK CREATION
What kinds of tasks can we give the participants to fulfill the objective?
Since the client's ask was to check how the users found the API documentation and how they could understand it, we focused on creating tasks that can individually help understand each user' process.
Which is why in order to track how they found the documentation, the first task was to get on the API page that they currently have linked. For testing how the documentatiion is read, the second task of finding an example and inspiration for projects lets the participants be able to explore and learn more about the platform. Lastly, the third task is created to see how the participants actually go about creating their queries and changing them so that we can see what they have learned from the documentation and if they know can start creating a project of their own.
Task 1 (Discoverability)
Find the API Documentation from the Cooper Hewitt home page.
Task 3 (Learnability)
Consult the API documentation and run a query. Now modify it!
Find an example of how the Cooper Hewitt API has been used in other projects and explore (i.e.: creatively, business application, etc.).
Task 2 (Readability)
PARTICIPANT RECRUITMENT
What kinds of users did we recruit and test for?
One of the main motives of the documentation from Cooper is that the platform is used and understood by not only advanced users who have used and created work from APIs before but also by new or novice users who don't use APIs often.
Therefore, we recruited both types of audiences so that we could see if there were any patterns and insights within the two groups and also how each participant individually learns the platform. We wanted to cover each base which is why there is a variety of experience and knowledge within each group.
Novice Users (N=3)
Aged 18 - 35
1-5 API knowledge
Less than 6 months of API experience
Advanced Users (N=3)
Aged 35 - 64
7-9 API knowledge
4+ years of API experience
EYE TRACKING STUDY
So, how did we conduct the test?
THE RESULTS
3 - Neutral
Tested By
Post Test Questionaire
Efficiency
58.5/100
(Average is 68)
Tested By
System Usability Scale
Satisfaction
14% Increase
Tested By
Pre & Post Test Questionaire
Effectiveness
Saccades, Fixation, Duration
For us to understand each user's journey. We used the Tobii platform that let us conduct the eye-tracking tests seamlessly with the participant.
We analyzed how each participant went through the platform by focusing on...
Saccades
The path of the center of vision from one part of the visual field to another.
Fixation
Locations on the page where participants stay for at least 200 ms.
Duration
The time it takes each participant to complete the task.
Retrospective Thinking Aloud
During the eye-tracking test, we encouraged the participants to conduct each task to the best of their ability without guidance.
Once each participant is done with the tasks, we review the recording with the eye-tracking movements and ask follow-up questions to better understand the participant' thought process.
After presenting to the Cooper Hewitt designers, developers and managers, they expressed how grateful they are for the extensive feedback and the thoughtful designs.
They asked follow-up questions on the two types of users we tested and the recruitment process for them which we were able to clarify on the levels of experience they had.
Interests
The recommendations were also well received and laid out for their own pitches. We also recommended more research and uset testing as well as better resources for A/B Testing and Data Analysis which could further help their discoverability and learnability efforts.
Next Steps
Final Presentation with Clients
Pre-Test
When recruiting our participants, we asked questions about their experience, knowledge, and technical skills.
Post-Test
After the eye-tracking test, we provided each participant with questions on how they can self-reflect their knowledge.
System Usability Scale (SUS)
The SUS 10-Question questionnaire offers a way to evaluate the usability of a website by a scale from 1-5 (Strongly Disagree to Strongly Agree)
1.
Focus On New Users
3.
Understand what users are making with the API
2.
Learn If Visitors Can Comprehend The API
THE ASK
What do the client' want to focus on?
THE OBJECTIVE
Identify the user behaviors of different types of users of the API documentation in order to improve site accessibility and learnability.
CHALLENGE 1
Users Found it Hard to Navigate to the API Page
5/6 Users
Had Trouble Finding The API
"
If they really want people to get to that page, I don’t know why they would make it so small at the bottom?” “If I wanted someone to look at it, I’d want to have it [in the global navigation].
- Participant 2 (Novice User)