Mastercard's Fanbot
Role
Software used
Project type
Duration
Overview
The Fanbot was a chatbot commissioned by Mastercard and Tennis Australia to bring people closer to the Australian Open 2018 (AO). By interacting with the chatbot, users were able to get live information about game scheduling, players' profiles, fun facts and more. Digital Arts Network (DAN) Sydney partnered with Wizeline (my team) to design and develop this chatbot.
My role
In my role as a UX Designer, I collaborated with DAN to identify the Fanbot's target audience and define its features. I conducted interviews and extensive testing to understand users' expectations and design the bot's conversation flows.
Users’ needs
For this project, we had to consider two clients, Mastercard and the Australian Open committee:
- As one of the main sponsors of the AO, Mastercard was looking for innovative ways to increase their brand visibility throughout the tournament.
- Mastercard wanted to show special offers to their clients and promote their Served App.
- Tennis Australia wanted to try new communication channels to reach out to their audience.
- Both wanted an experience that was good for fans attending the AO in Melbourne, as well as for those watching from home.
Design process
Because of our tight deadline, we didn't have enough time to interview users. I requested DAN's team to complete a questionnaire to outline their assumptions about potential users, leading to the creation of three proto-personas. Jessica, the casual fan, became our primary persona. Based on our personas, I created job stories and revised them with our client to prioritize tasks.
After that I mapped out the different concepts users could prompt about (e.g. players, scores, matches, etc.) and created diagrams to see how they were interconnected and how could users potentially jump between conversation flows.
Finally, we conducted multiple testing sessions to understand users' expectations about the bot, how they interacted with it and potential conversation dead-ends.
Project outcomes
The Fanbot was available on Facebook messenger and ran throughout the tournament's duration from January 15th to January 28th. It had over 8700 unique users and registered around 350,000 interactions. 57 percent of the fans that used it, returned daily and 64 percent signed up to receive updates on different AO topics.
The fanbot consisted of 14 different conversation flows which would trigger either by users' prompts or special events during the AO.
Conversation flows
For each flow, I drafted the "ideal" conversation a user would have with the bot. Then I tried to foresee the various directions each conversation could take to prepare the bot to suddenly jump into a different topic or recover from an error. Some conversation flows included the on-boarding flow to explain what the bot could do and how to prompt it; the subscription flow where users could subscribe to receive news about specific matches or fun facts; and the giveaways flow, where users could win prizes by sending a picture or answering a question.
In total we had 14 different flows, divided in three groups:
- Conversations triggered by the user: whenever a user asked a question to the chatbot or selected a quick reply.
- Conversations triggered by DAN: this included polls, Mastercard moments, fun facts (fun facts) or giveaways.
- Subscription messages: updates sent to users who subscribed to topics or players.
The prototype and tests
I used Framer to create a prototype of the chatbot that had quick replies, carrousels and users could input basic responses. I ran tests with company employees unrelated to the project while, simultaneously, DAN ran tests with users in Australia. The tests helped us to learn about users' expectations on how the chatbot should respond to queries and was a good tool to communicate with the engineering team. Some features made sense on paper but, once the team saw them in the prototype, they realized our platform wouldn’t be able to support them. Eventually, we were able to test with an unpublished, production-ready version of the bot. Here are a few discoveries from our testing sessions:
- Our on-boarding process was overwhelming. The amount of continuous text and the speed at which it was generated stressed users. To solve it, we added questions and quick replies in between the dialogues of the chatbot. This way we avoided having a chatbot monologue and gave users control of when the bot continued talking.
- Users could not start a conversation over. Sometimes users wanted to start a conversation from scratch so they would ask the bot to stop, but the only way to do this was by closing the conversation and opening again. The bot was adapted so it could recognize commands to stop, pause and restart a conversation and we also added a "help" flow.
Challenges
We developed this project before LLM's (e.g. gpt, llama) were as widely available as they are today. The natural language processing of the bot became more robust as the project advanced but it was still limited. We had to make sure that our conversation flows steered users towards predictable interactions; this reduced errors and, therefore, increased the perception of the Fanbot being a "smart" agent. On the other hand, the continuous presence of quick replies (i.e. pre-defined answer to the bot questions) distracted users from trying to input their own prompts. To improve this, the first time a conversation flow was triggered we gave some directions on how to prompt it again in the future.
Conclusions
Overall, the Fanbot was successful on increasing engagement and getting users closer to the Australian Open, having more than 8,000 unique users and 350,000 interactions. The bot worked so well that, one year later, its code and design was reused and rebranded for the Rugby World Cup 2018.