Imagine you and I search for a movie on the same app and we both end up getting a completely different recommendation list – even when both of us like same type of movies. And what if it was just because of your age, gender, race — or even personality type. Sounds strange, right?

But that’s exactly what can happen when AI-powered systems recommend things to us. Introducing FairEval – a framework that is trying to solve this problem.

FairEval is a method to test whether AI recommendation systems — especially those built on Large Language Models (LLMs) — are treating different users fairly.

Most of today’s AI tools try to guess what you might like, based on what you’ve liked in the past. But they may also (without meaning to) treat you differently depending on who you are.

FairEval checks if this is happening.

👉 How does FairEval work?
It runs two sets of prompts through the AI:
One that is completely neutral — no personal details.
One that includes sensitive info — like race, religion, or personality.

Then it compares the two sets of recommendations to see if they are different — and if so, how much and why.

To do this, it uses several smart techniques:
✅ Sensitive-to-Neutral Similarity Range (SNSR) & Sensitive-to-Neutral Similarity Variance (SNSV) check if certain groups are getting better or worse recommendations
✅ Personality-Aware Fairness Score (PAFS), a new one, checks how consistent the system is across different personalities
✅Search Engine Results Page Score (SERP) & Preference Ranking Agreement Grade (PRAG) check how much impact your identity has on the top results

🎯 Why does this matter for ecommerce and marketplace platforms?
Most ecommerce and marketplace sites today use AI to show you:
1️⃣ What products to buy
2️⃣ Which listings to see first
3️⃣ What price and promotions you qualify for

Now imagine if these recommendations vary just because of your gender, religion, or personality — not your actual preferences.
That’s a problem.

Let’s say two users want a backpack. But one is shown more expensive or lower-quality options just because of how the AI interpreted their profile. That creates a poor experience — and it’s unfair.

Frameworks like FairEval help catch these problems early. They make sure the AI is not unintentionally biased — and that everyone gets a fair shot, whether you’re a shopper or a seller.

✅ If you’re building AI in ecommerce, ask yourself:
💠 Is the AI fair across age, gender, region, and personality?
💠 Are we testing for fairness before going live?

FairEval gives us the tools to ask these questions — and fix what’s broken before it affects real people.