‘Trusted Feeds’ AI Content Fact Checking Feature PoC

Challenges

Business Challenge: How might we use AI to improve the user experience on the app and help users better engage with the app and with each other?

User Challenge: How might we help Facebook users quickly identify if posts have credible content, so they can avoid misinformation/disinformation and engage with content they can trust?

Course Instructor and Capstone Sponsor

Michiel Pruijssers

Business Goal

Increase user engagement
Increase the connections people have with each other

Solution

Build a Large Language Model (LLM) with Retrieval-Augmented Generation (RAG) to retrieve relevant and up-to-date information from external sources, determine if the post content is credible or not, and generate a credibility classification.

Then use GenAI to generate a report and summary for the user that indicates how it determined the credibility classification, and how confident it was in its classification.

My Roles

Sole contributor (UX, UI, and AI Lead)

Context

This PoC is the Capstone project for the AI Product Design course I completed through ELVTR that took place March 5th - April 30th, 2025. I attended all live classes and office hours, and completed 8/8 assignments, 3/3 small bonus assignments, and 1/1 large bonus assignment. I finished the course with __/100 points, a Certificate of Completion, and a Recommendation Letter from the instructor, which was only given to the top five students. (This info to be finalized once I have received my final grades)

Discovery and Concepting

I started with the assumption that misinformation/disinformation in social media was on the rise and was a major pain point, leaving people frustrated, struggling to recognize inaccurate content, and uncertain about whom to trust. Currently, Facebook’s open posting policies allow unverified content to manipulate public opinion, fuel social conflict, and influence decision-making. I felt this was a problem that could potentially be solved using AI, and I hoped to design a solution that would help people feel more confident in what they were reading and create a more pleasant social media experience. I started by quickly doing some discovery research to determine if this was a problem other Facebook users were currently facing.

Quick Discovery Research

Used Google to learn about Facebook’s goals and what they aimed to provide to their customers, so I could determine what the business goals might be
Used Google and ChatGPT to learn about the most common user complaints with the Facebook platform
Polled 4 people to validate if misinformation/disinformation on Facebook was a top concern for them
Discovered that misinformation/disinformation was, in fact, a top issue for people
Decided to use it as my problem space for the project, with Facebook as the social media platform I wanted to focus on

Concepting

I came up with 4 concepts, with the focus on taking the current state of the Facebook app and providing users with a quick and easy way to tell if a post was credible or not
For the concept selected, my goal was to create an LLM with RAG to retrieve relevant and up-to-date information from external sources to:
- Determine if the post content is credible or not
- Generate a credibility classification of Credible, Not Credible, or Undetermined
- Assign an icon to the post so users can quickly see how the post was classified
- Allow users to click on the icon to get more information on how the AI model determined the classification and how confident it was in its classification
Created a user flow
Designed a quick draft of a clickable prototype in Figma to visualize the concept and test the user flow. You can see some example screens 1-5 outlined in green to the right:
- 1. A one-time popup that users would see the first time they opened the app post new feature launch, briefly introducing the new AI feature.
- 2. The home screen with the new badge icon on a post.
- 3. The pop-up that appears when you click the “Credible” badge telling the user that the post was determined credible and how confident AI was in its classification.
- 4. The pop-up that appears when you click on the question mark icon next to the AI Confidence Level explaining what it is and how it’s determined.
- 5. The pop-up that appears when you click the “Not Credible” badge
Mapped out some potential user signals, AI technology, and metrics to consider
Documented potential ethical concerns to consider centered around Bias & Discrimination, Privacy, Inclusivity, and Trust & Transparency, and drafted some potential design adjustments to keep in mind
Used v0 to create some low-fi mockups to compare with my prototype
Refined my prototype

Narrowing Scope

I didn’t have the time to explore all the possibilities of what I could include so I narrowed the scope considerably and left out the following to look into at a later date:

Personalization:
- Embedding a tool into the platform, enabling the user to copy/paste a post source into it and have AI determine, in real time, if the source was credible or not
- The ability for users to click on the posts they like/trust so it populates their embedding space with similar and associated posts, so the user’s feed is more tailored to the types of content they want to see
Using AI to automatically curate the user’s feed based only on credible content
Set priorities and then change the weighting based on user behavior
UI changes that weren’t critical and/or related to ethics, accessibility, or impeding users’ goals and actions
Including supervised learning so the model can learn patterns and characteristics from human-labeled data to classify new content as credible or not
Including unsupervised learning by using Facebook’s data to learn patterns and features that can help identify misinformation and can adapt to evolving tactics over time
Including reinforcement learning with:
- ELO scoring
  - +1 for Beginner ELO score up to +5 for Grandmaster ELO score
- Including positive rewards based on the level of appropriate-fitting and accuracy of credibility, positive rewards based on a high confidence level, and positive rewards based on accuracy of “critical” settings.
  - +1 for classifying it as Credible or Not Credible
  - +5 for High confidence proved true
  - +5 for “critical” setting proved true
Creating a network of AI Agents to do specific tasks that would help verify content and deepen AI’s learning
Creating a business plan for the creation of a third-party oversight team charged with reviewing the AI model for bias while it’s being trained and tested, immediately after launch, and on an ongoing basis, and recommending changes based on their findings and user feedback data

Creating a Google Colab Notebook for AI Model Testing

Now that I knew this area was a pain point and had a rough idea of how I might solve it using AI, I needed to test it out to see if the AI model could function the way I intended. I did this by building a Notebook that allowed me to write Python code and run it to quickly test the functionality of the model without needing to set up a local environment or get an engineer to test it.

The Tech Build and How it Works:

LLM with Vector Database (RAG) - The model conducts a semantic search of relevant and up-to-date information from external sources.

Classification - Based on the semantic search, the model will determine if the content in the post is Credible, Not Credible, or Undetermined.

Badge Display - Once a post has been classified, it will display a badge icon on the post indicating the classification. This will automatically occur for every public post by a business or corporation.

Confidence Level - A confidence level of "Low, Medium, High" will be assigned based on how confident/certian the AI model is that it accurately classified the post.

GenAI Summary - The model will provide a high-level summary to users in the form of a pop-up that appears if they click on the badge icon, explaining why a post was rated "not credible" and what criteria it used to come to that conclusion.

Prerequisites:

An OpenAI key
AI prompt familiarity to generate Python code
A sample of public "Facebook posts" not containing PII

Notebook Roadmap:

Install - Python libraries
Link - OpenAI API key
Upload - Demo social media posts
Embed - Vector Database (RAG)
Classify - Demo posts (Credible, Not Credible, Unsure)
Badge - Create content badges and assign the appropriate badge to each post
Confidence - Generate a Confidence Rating for each post (Low, Medium, High)
Summarize - Creates a summary explaining why each post was labeled and rated the way it was
Demo - Provide a rough demo app to show how this would work

Roadblock!

So, I thought this was going to be pretty straightforward and that I would just use AI to help me generate the Python code to test out my ideas, but boy was I wrong. I took a global trek on the struggle bus with this. There was FAR more trial and error than I had imagined, and I had never accounted for the need to solve all the errors that came up, but I stuck it out and was finally able to successfully run my code!

This was my first time building a notebook, and though I struggled a bit, by the end I was able to:

Establish better prompts for getting the code I needed from ChatGPT
Better understand the process of building a functioning notebook
Better understand the errors I got and learning how to communicate with AI to help solve them

Note: Though I got my code to function and found out my concept was feasible, it was very inaccurate and would need a LOT of work to get it functioning at the level of accuracy needed to be considered a success.

Some of the Initial Screens I designed

Google Colab Notebook Intro and Roadmap

User Signals, AI Technology, and Potential Metrics

Potential Ethical Concerns

List of th 50+ steps it took to embed RAG

Sections and Cells

Demo Section Expanded with First Cell Showing

Research

Once I knew my concept was possible, I moved on to research. Because I didn’t have time to do much discovery research before I started concepting, I decided to combine user interviews and user testing at the same time to validate/disprove my concept and get feedback and suggestions on the usability and design.

Research Objectives and Assumptions

Research Objectives:

Identify user actions, behaviors, and experience
Understand the impact that misinformation has on users
Understand how users view AI
Learn if the concept helps alleviate their experience with misinformation
Learn about the functionality of the concept
Capture feedback about the concept

Assumptions:

People wouldn’t be inclined to trust a new AI feature
It wouldn’t change people’s general scrolling behavior

Approach

Talk with 3-7 social media users to better understand their experience with misinformation/disinformation on social media, test the functionality of the concept, obtain feedback, and then use ChatGPT to get additional feedback and suggestions on my concept.

Interview Preperation

Created a research plan
Created an interview guide
Created a test plan
Asked ChatGPT o4-mini to review my interview guide and test plan
Scheduled interviews

User Interviews and Testing

Conducted five 45-minute user interviews/tests
Asked pre-test questions focused on usage, misinformation, and AI
Users evaluated the lo-fidelity screens and pop-ups
Asked in-test questions
Asked post-test questions

Analysis and Synthesis

Copied interview/test notes into Fig Jam
Conducted Affinity Mapping
Ran the mockups through ChatGPT o4-mini-high for additional analysis & feedback
Pulled the key insights, strengths, weaknesses, and considerations from my interviews and from the AI output

Key User Pain Points

The amount of misinformation/dissinformation deterred them from spending time on Facebook
Many participants were left feeling mad, frustrated, and/or disappointed when they encountered misinformation/dissinformation on Facebook, contributing to a poor overall experience
Some participants felt like their feed wasn’t customized to them, and they wished what they wanted to read was easier to find/access

Key Insights

All participants liked the new feature and thought it would be helpful
All participants said they would trust the AI credibility classification shown
All participants felt the new feature would positively impact their conversations with people in the real world
Many participants felt they would use the app more and be more likely to share posts if the new feature was in place
Most participants said that they would look at the badges as they were scrolling and only read the posts that were marked as credible, instead of scanning everything like they do now

Considerations

Key Strengths

All participants liked the badge system as a quick way to determine if the post was credible or not when scrolling through posts

All participants liked that there was additional information available if people wanted to know more

Pivot

Based on my research and testing, there was one pivot I decided to make.

Removed the thumbs-up/down - I had initially included the icons as a method of reinforcement learning for users to give feedback to the model, but once I started thinking more about how to prevent bias from being introduced into the model, I realized this feature would definitely contribute to bias. I decided to remove that feature and rely on the “Give Feedback” button, which would still provide feedback but would be reviewed by humans and not sent directly to the model.

Key Vulnerabilities

It was unclear to some participants what types of posts the new feature would be applied to
The Credibility classification and Confidence level, and how they were determined, were unclear to some participants
The thumbs up/down was unclear to most participants
The text was too small in some places
The touch targets were too small in some places

It would be a very complex AI model to build for it to function with the level of accuracy needed for such subjective areas as credibility and trust

Need to ensure that bias is not introduced into the model

Affinity Map

Insights, Potential Changes, and Pivots Prioritized by - Must fix, Should fix, Could fix

Design Iterations

Following my research and testing, I decided to experiment with the idea of including more modalities and applied the critical ‘Must fix items I discovered during research to iterate and improve my concept.

Incorporating Multimodal Outputs

Originally, I had three levels of text-based outputs:

A visual indication of the credibility classification in the form of a badge icon on the top right of the public post
A pop-up summary if someone clicked on a badge icon with a brief, high-level explanation as to what the classification was based on and how confidant it was in its classification
And an additional longer and more detailed explanation if people wanted to click on the Read More button

I decided to add two additional outputs to explore different modalities for the model and provide the user with two possible actions:

Listen – Allowing the user to have AI read the popup summary and additional information out loud to them
Talk – Allowing the user to ask AI additional questions about its output and/or the topic of the post via voice and text

There are three main benefits/use cases for adding these addiotional outputs:

Allowing users to listen to longer content rather than having to scroll on their phone
Works as an accessibility feature allowing a user with impaired vision to have the pop-up summaries and additional information read out loud to them.
Note: It is likely that users who are blind would already have a screen reader enabled, so it would be important to ensure that the audio output wouldn’t interfere with screen readers that are already being used to go through the app.
Allowing the user to dig even deeper to learn more about the topic and get answers to questions via the Ask AI function

Identify Edge Cases

I did not have time to fully explore potential edge cases, but below are some questions I’d like to explore when I have more time:

What if an AI model were the one to write a post?
What if a user is offended by the AI-generated reason for why a post was classified the way it was?
What if a user purposely tries to bias or corrupt the model?

Key Design Changes

Based on the feedback I received from user interviews and from asking AI to analyze my prototype, I made a number of changes to the prototype:

Include an opt out section in the Settings
Increased the text size
Increased the size of the touch targets
Simplified the language so it wasn’t so technical
Clarified what posts would have this feature
Clarified what the Credibility Classification and Confidence Level were
Included multimodal outputs
Prioritized the top 3 most impactful criteria that AI used to classify the post the way it did, but made the full list of criteria/reasons available through ‘Read More’
Removed offering a similar article for posts marked Not Credible
Removed the links to the sources AI used to determine credibility
Include a ‘Summary’ on the Credible and Undetermined pop-ups

Updated Prototype

After making the key design changes, I created an updated prototype.

‘Ask AI’ Feature & ‘Show More’ link

Welcome Pop-up Before

Info Pop-up Before

>>>>

Welcome Pop-up After

Info Pop-up After

‘Listen’ Feature & ‘Scroll’ Feature

Not Credible Pop-up Before

Give Feedback Pop-up Before