NerdWallet AI
Context
In 2023, I led a hackathon team to create an MVP chat module incorporating ChatGPT technology, winning the top award and inspiring leadership to establish a dedicated AI team, where I led design.
NerdWallet’s mission is to empower users with the information they need to make financial decisions confidently. The rise of LLMs presented a competitive challenge to us. But could we harness this technology and offer our own unique service?
Objectives
- User Needs: Quickly find accurate answers to complex financial questions.
- Business Needs: Differentiate NerdWallet from LLM competitors with a unique, trustworthy AI chat experience. Bonus: utilize the chat to drive registrations and traffic to our monetizing pages.
- Hypothesis: Providing a chat that answers user queries, as well as cites and directs them to fact-checked NerdWallet content, will increase user engagement and loyalty.
User Research
I began by conducting extensive user research studies, including surveys, preference tests, and video-recorded prototype tests with early design iterations. I made sure to procure my product manager's feedback as I designed these studies to keep her involved in the process and help ensure that what our tests were investigating would be relevant to her product vision.
Key Insights
- ⏰ Efficiency: Users struggle with information overload from lengthy articles. They value time and efficiency, making an AI chat that provides quick, precise answers highly desirable.
- ✔ Trustworthiness: Users are skeptical of LLMs. We need to build trust - for instance, by linking responses to NerdWallet’s expert-written content.
- 🔍 Discoverability: Amidst NerdWallet's vast content, the chat needs to stand out without adding too much noise or clutter to the user interface.
- 💡 Clarity: Users often confused our AI chat with a traditional customer service chatbots. Clearly signaling the advanced capabilities of the chat - through design elements or copy - would be crucial.
Project Planning
Research Synthesis
After processing the research, I shared out insights with my cross-functional partners in detailed research decks. Keeping partners informed on research results helps ensure alignment and buy-in.
Defining requirements
I supported my product manager to help define project requirements and milestones. Regular syncs with key stakeholders, including the head of SEO and our CEO, kept expectations realistic and aligned. We designed a phased approach for developing the chat: we would begin testing with an article companion chat-bot, then move to the home-page, then eventually integrate personalizated features.
DESIGN
user journey
We saw that the chat could be an entry-point for users, help them discover products through NerdWallet and compare products in new and customized ways, and build customer loyalty by driving registrations for premium chat features.
Clearly defining and visualizing the user journey helps scope the project correctly, design holistically, and anticipate edge cases.
Ideation and feedback
I began lo-fi to wrap my head around the problem space and explore design variants quickly, going wide before going deep. I kept cross-functional partners posted on my progress to ensure alignment, hone in on the best approach, and modulate designs based on engineering scope. I also solicited feedback from design peers through our cross-vertical design critiques.
I strive to be low-ego and un-protective of my work so that partners feel at liberty to bring their best thoughts to the table.
Competitor analysis
Taking stock of the competitive landscape helps us identify niche opportunities and find design inspiration. After studying existing LLM interfaces like ChatGPT and Bing, as well as direct competitors like Parthean and Moneymade, we discovered that NerdWallet could position itself uniquely by providing AI responses enhanced with fact-checked content written by our authoritative NerdWallet experts.
Visual refinement
I polish visuals by considering design principles like visual hierarchy, simplicity, and unity, as well as ensuring that the designs will transition seamlessly across all breakpoints.
Prototypes
Prototyping is an absolute must to get a feel for flow, optimize interactions, and test workflows. Sharing prototypes usually evokes more informed and invested feedback.
Research
Before handoff, I pressure tested full prototypes of our designs using UserTesting.com so that we could illuminate usability issues before investing in development time.
Handoff
To ensure seamless translation of design to dev, I thoroughly annotate designs, comprehensively enumerate design requirements, and conduct walk-throughs for the engineers. Putting the work in up-front helps reduce errors that surface in post-development design reviews.
Post-Development
Analysis and iteration
After rolling out NerdWallet AI, we used AB testing to evaluate the success of our designs. We supplemented the quantitative data with further video user testing to obtain a qualitative sense of the user experience. (I shared out these findings in detailed decks so that the team could have shared consciousness.)
Successes
User satisfaction
Users reported a high level of satisfaction with NerdWallet AI, remarking on its intuitive usability, speed, and the convenience with which it led them to relevant and trustworthy content.
User engagement
Through multiple rounds of research, analysis, and iteration, we increased the engagement with our article chatbot from 1.42% to 5.01%.
aiding the nerdwallet user journey
We discovered that the presence of the chat-bot could boost user engagment with the rest of the site, increasing average user page views and increasing likelihood of monetizing. For our article chat-bot, 28% of users would click a related link provided the chat, increasing their likelihood of monetizing. For the home page version, we increased mClicks by 17.1%.
Product expansion
Initially, we rolled the chat out on articles as a supplement to the user's reading experience. After seeing success, this expanded to a prominent placement on the home page. Most recently, we moved to include personalization capability by integrating the chat with user's bank account information.
