of mobile Online Collections for casual browsers of the museum
MET-igating low discoverability
CLIENT : The MET Museum
RESEARCH METHODS : Eye-tracking tests, Retrospective think-aloud, Pre/Post questionnaires, Google analytics, Hotjar maps
DURATION : 14 weeks
PROJECT TEAM : Shivani Kolte and 3 other usability experts
Overview
The The Met ‘Collections’ page is a part oof the museum’s website which details the various exhibitions, galleries, and art collections that the museum offers. The section highlights the different categories oof art collections the museum offers along with access to Research materials related to these.
Within this endeavour, a team of the four of us analysed how the mobile website performed in terms of usability and worked to discover the patterns in content interactions and user behaviours that this page generated.
Through this study solutions were suggested to improve the mobile website in three areas :
-
Improving the ‘Search’ experience and optimising the results for better discoverability of the Collections page offerings.
-
Usability of the ‘Filters’ and enhancing the object listings on the website.
-
Under the object details, articulating and strategising the content for a faster and mobile friendly experience.
My Role
-
Managing and communicating with the team to plan out the project according to schedules and priority of tasks.
-
Research Ops - Alongside 3 other classmates, wee as the research ops team defined a recruitment strategy and executed it successfully to provide participants to each of the four studies.
-
Moderation of 2/8 Eye-tracking studies and in-depth analysis to recommend actionable insights
-
Note-taking on 2/8 the Usability tests moderated by my team members.
Figuring out plan of action📝
Kickoff Meeting
Aligning goals and expectations for the study.
(Fig 1) - Eye tracking test set up. The monitor shows the calibration of the eyesight of the participant. The vision being tracked is directed towards the mobile screen in front of them.
Lying on the intersection of qualitative and quantitative research methods, Eye tracking is an evaluative research method that helps researchers understand visual attention.
An eye tracker is a tool that allows UX researchers to observe the position of the eye to understand where an individual is looking. Most modern eye-trackers rely on a method called corneal reflection to detect and track the location of the eye as it moves. Corneal reflection uses a light source to illuminate the eye, which then causes a reflection that is detected by a high-resolution camera. The image captured by the camera is then used to identify the reflection of the light source on the cornea and in the pupil.
Eye-tracking tests
Client :
-
We want to learn more about what people do on the collections page.
-
what they are looking for, their journeys on this area of the website (casual browsers, unfocused learners), learn what their users are drawn to
-
what content type they’re more interested in (videos, photos, podcasts, etc.)
Team :
What we got from this :
-
WHO - We understood that our focus was to be on young casual browsers.
-
WHY (Goals) - Once we narrowed the WHO, accordingly we decided to focus on mobile users since its a go-to in teh said target group and we wanted to explore how the traffic could be grown on Mobile for such a platform.
Methodology
Deciding on the method
Understanding that the client's need from this usability study was to find out what catches and holds the attention for new and casual browsers who use the website for preliminary research and quick tasks on the mobile interface. The most appropriate method to get insights on this came out to be Eye-tracking studies.
While we were confident with this method, we also wanted to consolidate our findings by triangulating them with user behaviours studies. For this we examined data from Google Analytics as well as Hotjar heatmaps to see patterns in general visitors of the website
The Tobii Pro software was used for eye-tracking to track our participants' vision and inspire dialogue for questions in the moderated eye-tracking sessions during retrospective think-alouds.
Google analytics was the tool that helped us quantify existing trends in user behaviours on the Collections page
Hotjar helped us note existing patterns in scrolling and gaze heatmaps.
We combined all our data on dovetail and synchronously derived and analysed insights from all sources of data in our study.
Where to focus, getting clarity on what the scope of examination should be 👌
Gaining perspective and identifying pain points
We decided to look briefly at the existing data first, to understand where our opportunity for testing lies and understood that the mobile Collections page was generating increasing user traffic. We established that it would be best to follow this graph and suggest improvements for a more mobile-friendly experience.
(Fig 2) -Google analytics data showing the increase in Mobile traffic on teh Online Collections page from May 2023 - Apr 2023
Research Ops🥷
(Fig 3) -Participant screening spreadsheet
A comprehensive screening questionnaire was sent across relevant social groups as well Pratt list-serves and spread across via personal network as well. The questionnaire clarified briefly the format of the test as well as helped us filter participants according to needs of the study.
Recruitment of the right participants - Research Ops
The criteria prioritised participants with high interest and low expertise and experience with the subjects of art and art history subjects (which was clarified upon during the pre-test questionnaire to assess their depth in knowledge and genuine interest.)
Interest in art/art history
Expertise/Experience in art/art history
Finally, 8 participants were chosen according to the above factors.
Execution is near..🚥
Defining Research goals
A comprehensive screening questionnaire was sent across relevant social groups as well Pratt list-serves and spread across via personal network as well. The questionnaire clarified briefly the format of the test as well as helped us filter participants according to needs of the study.
1
Discovering usability issues and interaction patterns of mobile users
2
Learning what content to strategically highlight on mobile Collection and how to do that
Collecting and sorting through data
(Fig 4) -Eye-tracking sessions in progress
Conduting the Eye-tracking tests
We conducted in- person mobile eye tracking tests. Our testing process involved a pre-test questionnaire consisting of five questions, followed by five tasks for our participants with time for a retrospective think aloud to ensure accurate and thorough feedback.
After the tasks were completed, we asked participants five post-task questions and administered a system usability scale to gauge overall user experience.
These tasks and post-task questions were devised around the most-likely goals of the users on the page, giving them a chance to explore the website but also see how well they interact with the content within, how usable the features are while they navigate, how clear the purpose of the pages and content was and how satisfied and comfortable they felt while and after completing their tasks.
Tasks :
01
Goal of task : Usage of art history categories or Search for a specific artist
You are interested in Vincent Van Gogh and want to learn more about them through The Met Museum’s website. Find out more about his works.
02
Goal of task : Browse the Collections page and see if they can find a new interest
Let’s say you’re visiting The Met Museum this month. Find two museum collections objects that you would be interested in seeing in person.
03
A
Goal of task : Testing discoverability of advanced filters
You are learning about Asian art at school. You are specifically interested in knowing more about Japanese Art from the 1800s. Check if there are any related art objects at The Met Museum that are on view.
03
B
Goal of task : Testing discoverability of advanced filters
Find other objects that are in the same gallery. Can you check if these items are in the same gallery?
04
Goal of task : Use other features of the Collections page
You are trying to write a well-researched essay on the history of fashion for school and want to know more about it. Find research material to help guide your writing.
One of the challenges we faced were, limited wording in the task so that it isn't leading and another one was to limit the number of tasks as RTA (Retrospective think aloud) used in this method proves difficult sometimes s participant's aren't able to recollect well.
We overcame this by having clear goals of the tasks and therefore limiting the scope of the task itself to test specific features.
Organising purposeful insights and pointing out areas of improvements to suggest recommendations for
After extensive testing, we used Dovetail to take notes and analyse our data synchronously. Dovetail was a highly helpful and productive tool to bring the team on the same page on the format of notes and insights so that they were understandable by all and we could consolidate findings harmoniously.
(Fig 5) - Our dovetail dashboard where we uploaded our test videos, took notes and recorded our pre and post-test answers, tagged insights, derived findings and generated recommendations from.
SUS Score Results
Post - qualitative analysis, quantitative data was sorted through. The first step to that was corroborating the SUS scores which are the System Usability Scale scores and understanding where we stand. These scores are based upon users rating questions such as 'How easy was it', 'how frequently would you use it', and 'how confident are you while using it'.
These help put numbers to the qualitative insights provided by our participants.
We had a slightly below-average score of 66.9. The website ranked in the 46th percentile of all websites, which means that its overall usability is higher than only 46% of all other websites.
This analysis made it clear that the ‘MOBILE’ Collection page needed improvement.
Research findings
Summarising our findings and recommending small but significant improvements
After deriving insights from qualitative analysis and the Post- test answers along with the SUS scores, we found a total of 60 major and minor areas of improvement which we identified and classified into three categories: 'Usability', 'Content', and 'Functionality'.
These insights were then combined and triangulated with the Google analytics data and Hotjar recordings to check how relevant they are and confirm their severity observed outside the study as well. On the basis of this, three main themes emerged which were basic but integral factors to be improved upon. These themes were 'Search', 'Listings' and 'Object details'.
What worked well
Participants highly appreciated the visuals of the website and found that it was helpful to have the information on the Collections of the museum online.
They also mentioned the content to be a good mix of text and images and were acknowledging of content segmentation such as 'overview', 'display timeframes', and 'related articles', to mention a few.
According to the SUS scores :
4/8
Participants wanted to use the website frequently and thought it was easy to use
5/8
participants felt that the website has a high learnability potential and were confident using the website.
When indicating data positively, it means the rating was 4 or 5 on a scale of 1 to 5.
Specific Areas of improvement and recommendations
01
Search Experience
-
Unhelpful jumplinks preventing users from navigating to useful categories
2. Unnatural zoom on the search bar
3. Missing error prevention in the search experience
4/8 of our users 'The artwork' section to navigate collectinos but the issuue was that it was hiddeen below the top area containing jumplinks which were never used.
-
(Vid 1 ) When users chose to search keywords, the search bar interaction of zooming in hindered a seamless interaction as it seemed unnecessary and disorienting.
-
(Fig 7 ) 3/8 of our users spelled 'Van Gogh' wrong and only one result showed up which could prove difficult for international users which is a target group that the museum focuses on as well.
1
(Fig 6) Mobile collections landing page
2
3
(Fig 7) Gazeplot of a search result showing Absent error prevention leading to only 1 result when the keyword was spelled wrong.
(Vid 1) Unnatural zoom in the search experience
Recommendation 1
-
Minimal top section on Collections page to improve discoverability of categories.
-
Isolated search bar for a focused and easy search experience.
(fig 8) Recommeded landing page for Online colllections
“There was too much going on there, so I skipped the Collection Areas before. But now it’s clearer.”
- One of the participants who used the site-wide search instead of the ‘Collections’ specific one, and skipped the jump-links as well
-
(fig 8) Removed all the jumplinks to have only important content present on landing also providing faster access to the categories section which is vital in a user's search journey on the page.
“The interaction feels much smoother and easier. I feel confident in searching.”
- One of the participants who spelled the Van Gogh’s name wrong
-
(Vid 2) The isolated search page helps the user have a focused experience along with a suggested auto-completion feature to help non-expert users.
(Vid 2) Recommeded searrch expreience
02
Listings
-
While pictures caught the users' eyes, the endless scroll did not support the exploratory search as they got frustrated
-
Users did not scroll past 6 or 7 images
-
The filters were not noticeable to users. One of our users also said :
“My brain just skipped the ‘Filter by’ part”
The accordion format of the 'Filters' was not familiar to them and they got confused by the presence of the prominent 'All fields' button beside the search bar
-
Long scrolls not aiding user journey
.png)
2. Low discoverability of 'Filters'
(Fig 11) Google Analytics data supporting the low engagement rates.
(Fig 10) Hotjar clickmap showing the 'Filter' dropdown receiving hardly any clicks
(Fig 9) Tobii Pro heatmap showing activity dwindling past a few listings supported by a gaze plot on the left.
Recommendation 2
-
Upfront access to filters
-
Standardised sizes for all preview images
-
Dedicated bottom pop-up for advanced filters.
-
We revisited a participant with the updated prototype, and they found the new filters significantly improved. The 'All Fields' button initially confused them, but the revised experience better matched their expectations and mental models.
(Fig 12) Recommended Improved 'Filter' function (more discoverable and clear in affordance)
(Vid 3) Recommended Improved 'Filter' function (more discoverable and clear in affordance)
“I wasn’t able to understand the ‘all fields’ box beside the search. This is definitely better, I get that it’s filters now.”
- One of the users who used the filters
03
Object Detail
-
Long Descriptions and unfolded accordions
-
Some descriptions run too long due to the nature of the subject which leads to users only doing quick scans of the text.
-
Unfolded accordions lead to long scrolls again and confusion about the status on the page
"I had to scroll through so much text in order to find what I was looking for. That kind of put me off a little bit cuz I would've liked to learn about this. But more by choice rather than this being pushed on me when it wasn't what I was looking for.”
Recommendation 3
-
Shorter, more consumable descriptions and folded accordions to give users the freedom of choice/ control over what information they want to access.
(fig 13) Exemplary page that our users appreciated a lot. The length of the description was very appropriate, and how the accordions are all grouped together and closed by default / helped our users easily navigate the content.
Other Findings
-
Confusing labels
2. Misleading content on the 'Research' page
3. Users are unsure of the objective of the 'Map' page
-
The menu was a source of confusion as the users struggled to distinguish between 'Art', 'Collection', 'Exhibition' and 'Gallery .
-
For the Libraries and research page, which was within the scope of our fourth task guiding them to explore research materials, One of our users hoped to find a search bar where they could enter a keyword or phrase and receive a list of relevant articles or other materials. - to help them navigate this page
Upon further exploration, the users were lost when they did not find digital sources such as paper and articles and instead found inks ABOUT the Research Centres and Archives.
-
Users were not sure what to expect when they suddenly landed on the map page. When they tried clicking around they expected a 'Google maps' type of interaction where they would be able to see which artwork was hosted in which gallery but were only able to scroll across the large image.
(fig 14) The team presenting our findings to the clients.
Conclusion
We presented to a team of four professionals from the MET museum who came from various teams within the organisation holding the titles : Head of Product, Senior Product Manager, Senior Data Analyst, and Product designer. This was a huge learning experience, to get different perspectives on what kind of changes are possible and how they are beign thought of by different teams. It also shed light on what issues might hold priority over others.
"It's great to see how much of the website this group was able to examine as a whole, and we saw a little bit of so many different plate areas of the website, which was really neat and exciting to see how you brought it all together."
Work for the future
-
Conduct in-depth usability studies of planned improvements
-
Collecting more data from Hotjar for better reliability.
-
Creating a custom dashboard on Google Analytics to track user behaviour with advanced filters and object detail page
-
Reimagine the museum map experience.
Learnings and Takeaways
-
Focus can keep shifting but combining research is always a difficult task, so referencing the goal set and what changed since can help get clarity in the process.
-
What might seem like a big issue while conducting the tests, might not be as severe or need as much prioritised attention as others, so revising and relying on insights quantitatively is necessary as much as it is qualitatively.
-
Designation of tasks helps focus better. This was a takeaway from being a part of the Research Ops team. Though, recruitment is tough, dedicating time and focussed energy to it did help.