Most large language models (LLMs) are built to answer fluently, not to hold back. Their instinct is to always generate something, even if they are not sure. That is how hallucinations creep in.

Take healthcare as an example. Imagine you ask about a new treatment. Instead of giving you a confident but potentially wrong explanation, the AI could say “I don’t know the latest clinical guidelines, please check trusted medical sources.” That single line can make all the difference between blind trust in a wrong answer and an honest response that pushes you to reliable information.

What is interesting is how we measure AI performance is also evolving.
Early models were judged mainly on accuracy against benchmarks like reading comprehension or translation. The focus was on “Did the model get the right answer?”

Today, the challenge is not just accuracy but also knowing when not to answer. Evaluations now include whether the model can refuse gracefully, express uncertainty, and still remain useful.

Researchers are exploring this shift through three techniques:

1️⃣ Refusal benchmarks: Special datasets where the only correct answer is “I don’t know.” Example: asking “Who won the FIFA World Cup in 2035?” A good model should refuse rather than hallucinate.

2️⃣ Calibration tests: These check if confidence matches reality. If a model says “I’m 90% sure this is the answer,” it should really be correct 9 out of 10 times. Poorly calibrated models sound confident even when wrong.

3️⃣ Selective prediction curves: These measure accuracy when the model is allowed to abstain from low-confidence answers. Imagine a medical Q&A system that only answers when it is 80% confident. It will answer fewer questions, but the accuracy of the ones it does answer goes up sharply.

The day when AI models can consistently and usefully say “I don’t know” is the day they shift from being smooth talkers to truly trusted reasoning partners.