Gemini now checks its work (if you ask it to)
The Hallucination Problem
Hallucinations — false, seemingly plausible results — are a major drawback of generative AI (GAI). Because of the way the underlying large-language models (LLMs) work, GAI systems have no notion of truth. Their “knowledge” is all based on statistical probabilities. Because of this, they sometimes make up “facts” with complete confidence. (Check out this document from Google to learn more about AI hallucinations.)
A Hidden Verification Feature
Advancements like ChatGPT search and systems like Perplexity.ai help address this problem by citing their sources. But hallucinations remain a problem. Recently, Google added an interesting, mostly hidden feature to Gemini that directs it to double-check its responses. Since this feature is buried in a submenu, I’m not sure when this was rolled out. I noticed it for the first time yesterday.
Gemini adds some buttons at the end of its output:
The icons represent liking, not liking, regenerating, and sharing the response. Clicking on the three dots brings up this:
I wanted to copy this output to put it in an Apple Note, so I clicked on the three dots. In addition to copy, clicking brings up two interesting options. Listen reads the output to you, which is kind of interesting and useful, but that’s not what caught my eye. “Double-check response” immediately captured my attention.
How Double-Checking Works
Clicking on double-check response triggers Gemini to do a web search to find sources to support or refute parts of its output. Gemini goes through different sections of the output and selects those elements that need verification (based on its assessment). It selects elements that are presented as facts rather than opinions or explanations of reasoning.
When I did this on the output above, Gemini flagged two aspects of the output as being verified through online sources. I asked Gemini to double-check a more fact-based part of the same conversation and it found supporting sources for more than half the output. Here’s what that double-check showed:
Verified elements are highlighted in green, refuted or unverified elements in brown and elements that weren’t checked are not highlighted. If you click on “Understand the results” (which is shown at the end of the output) this window pops up.
Seeing the Sources
What’s really nice is that clicking on the expand button for any checked statement brings up the reference that verified or refuted the statement. Clicking on the reference opens the page in a separate browser tab.
How Gemini selects particular references is a mystery, but it’s nice to have at least some verification of facts. Of course, if the underlying reference is wrong, so is the output. So, for anything important it’s a good idea to do independent verification.
The Bigger Picture
Will this replace Perplexity, which is my current go-to tool for serious searches? No. But, Gemini seems to be steadily improving, which leads me to use it more. It’s nice to be able to do a quick verification without pulling up another tool. Double-check is available on the free version of Gemini, which is a nice bonus.
The bigger picture here, I think, is that Google keeps improving Gemini. As I’ve mentioned before, keep an eye on Google. They got off to a slow start with GAI, but they’re catching up quickly. Given Google’s ubiquity on college campuses, they may end up the dominant player in higher education AI.
Well, that’s all for this time. If you have any questions or comments, you can leave them below or email me - craig@AIGoesToCollege.com. I’d love to hear from you. Be sure to check out the AI Goes to College podcast, which I co-host with Dr. Robert E. Crossler. It’s available at https://www.aigoestocollege.com/follow. Thanks for reading!