top of page

x

PrattBlackLogo.png

Testing The New API Learning Tool

The Cooper Hewitt API provides its users with access to data on 215,000 items that can be analyzed and pulled to create numerous interesting projects. A usability and eye-tracking study needed to be conducted for the API to be more discoverable and easy to use. 

​2023 (3 months)

Duration

MY Role

UX Strategist

THE TEAMMATES

Priyanka Nair

Alya Zouaoui

TOOLS USED

Figma

FigJam

3943.png

Project Overview

The Cooper Hewitt API provides its users with access to the data of 215,000 items in their collection, along with information about the people, companies, and other organizations that were associated with them.

 

This project tested how novice and expert API users interact with the Cooper Hewitt API website by conducting a usability study and eye-tracking. We provided several recommendations focused on improving how people locate the API website (discoverability), find API examples (readability), and learn to run a query (learnability).

Our pitch meeting to the Cooper Hewitt developers and managers.(Left to Right: Kato, Phil, Laiba (Me))

THE CHALLENGE

Stakeholders worry about how the platform can teach users how to create projects from various skill levels because of the lack of discoverability.

METHODOLOGY

TASK CREATION

What kinds of tasks can we give the participants to fulfill the objective?

Since the client's ask was to check how the users found the API documentation and how they could understand it, we focused on creating tasks that can individually help understand each user' process.

 

Which is why in order to track how they found the documentation, the first task was to get on the API page that they currently have linked. For testing how the documentatiion is read, the second task of finding an example and inspiration for projects lets the participants be able to explore and learn more about the platform. Lastly, the third task is created to see how the participants actually go about creating their queries and changing them so that we can see what they have learned from the documentation and if they know can start creating a project of their own.

Task 1 (Discoverability)

Find the API Documentation from the Cooper Hewitt home page.

Task 3 (Learnability)

Consult the API documentation and run a query. Now modify it!

Find an example of how the Cooper Hewitt API has been used in other projects and explore (i.e.: creatively, business application, etc.).

Task 2 (Readability)

PARTICIPANT RECRUITMENT

What kinds of users did we recruit and test for?

One of the main motives of the documentation from Cooper is that the platform is used and understood by not only advanced users who have used and created work from APIs before but also by new or novice users who don't use APIs often.

Therefore, we recruited both types of audiences so that we could see if there were any patterns and insights within the two groups and also how each participant individually learns the platform. We wanted to cover each base which is why there is a variety of experience and knowledge within each group.

Novice Users (N=3)

Aged 18 - 35

1-5 API knowledge

Less than 6 months of API experience

Advanced Users (N=3)

Aged 35 - 64

7-9 API knowledge

4+ years of API experience

EYE TRACKING STUDY

So, how did we conduct the test?

THE RESULTS

3 - Neutral

Tested By
Post Test Questionaire

Efficiency

58.5/100

(Average is 68)

Tested By
System Usability Scale

Satisfaction

14% Increase

Tested By
Pre & Post Test Questionaire

Effectiveness

Saccades, Fixation, Duration

For us to understand each user's journey. We used the Tobii platform that let us conduct the eye-tracking tests seamlessly with the participant.

We analyzed how each participant went through the platform by focusing on...

Saccades

The path of the center of vision from one part of the visual field to another.

Fixation

Locations on the page where participants stay for at least 200 ms.

Duration

The time it takes each participant to complete the task.

Retrospective Thinking Aloud

During the eye-tracking test, we encouraged the participants to ​conduct each task to the best of their ability without guidance. 

Once each participant is done with the tasks, we review the recording with the eye-tracking movements and ask follow-up questions to better understand the participant' thought process.

After presenting to the Cooper Hewitt designers, developers and managers, they expressed how grateful they are for the extensive feedback and the thoughtful designs. 

They asked follow-up questions on the two types of users we tested and the recruitment process for them which we were able to clarify on the levels of experience they had. 

Interests

The recommendations were also well received and laid out for their own pitches. We also recommended more research and uset testing as well as better resources for A/B Testing and Data Analysis which could further help their discoverability and learnability efforts.

Next Steps

Final Presentation with Clients

Pre-Test

When recruiting our participants, we asked questions about their experience, knowledge, and technical skills.

Post-Test

After the eye-tracking test, we provided each participant with questions on how they can self-reflect their knowledge. 

System Usability Scale (SUS)

The SUS 10-Question questionnaire offers a way to evaluate the usability of a website by a scale from 1-5 (Strongly Disagree to Strongly Agree)

1.

Focus On New Users

3.

Understand what users are making with the API

2.

Learn If Visitors Can Comprehend The API

THE ASK

What do the client' want to focus on?

THE OBJECTIVE

Identify the user behaviors of different types of users of the API documentation in order to improve site accessibility and learnability.

CHALLENGE 1

Users Found it Hard to Navigate to the API Page

5/6 Users

Had Trouble Finding The API

"

If they really want people to get to that page, I don’t know why they would make it so small at the bottom?” “If I wanted someone to look at it, I’d want to have it [in the global navigation].

- Participant 2 (Novice User)