I have developed a healthy suspicion of any ChatGPT answer that imagines no negative possibilities or doesn’t have any follow-up questions for me. Simulated AI arrogance can leave any user with unexpected obstacles.
But one useful way to avoid those pitfalls is simply to follow up your question with another that forces ChatGPT to consider its blind spots and blithe self-assurance, however artificial it might be. Simply inquiring “What are you unsure about in your answer?” will open up some surprising insights into where the AI has missed the mark.
It’s pretty simple. You have a normal conversation, get an answer, and then ask the AI where it thinks that answer is weakest.
So I started trying it. ChatGPT would often point directly at an assumption it had quietly made earlier in the conversation. Once that assumption was visible, I could give it the missing information and get an answer that was genuinely better.
Planning a trip
I started with a travel itinerary, as it’s something ChatGPT is very good at producing and occasionally terrible at understanding. I asked it to plan a three-day family trip. I gave it some basic information and the resulting itinerary looked good. There were morning activities, lunch suggestions, afternoon stops, and enough apparent breathing room that I could imagine happily following it.
There was just one problem. ChatGPT had created the trip without considering who the trip was for and without thinking of any possible errors or slowdowns that might put us off the schedule. Instead of telling ChatGPT the itinerary was too busy, I typed my new follow-up: “What are you unsure about in your answer?”
“I’m unsure about the pacing of the itinerary, particularly the second day. I’ve assumed your family can comfortably handle two substantial activities with a break between them, but I don’t know your child’s nap schedule, usual energy level or how much downtime you prefer while traveling. I’m also estimating travel times without knowing exactly where you’ll be staying. If any of those assumptions are wrong, the schedule could feel much more hectic in practice than it appears on the page.”
ChatGPT had spotted the problem with its own plan without me pointing it out. That gave me a much better next prompt. I told it that I wanted only one significant scheduled activity each day, and that afternoons needed to remain flexible. ChatGPT rebuilt the itinerary around those constraints, and the new version immediately felt more like something a real family might survive.
Repair or replace?
For another experiment, I gave ChatGPT a hypothetical laptop dilemma. Imagine I had an older computer that was beginning to struggle and wanted to know whether I should repair it or replace it. The AI created a framework involving the laptop’s age, likely repair costs, performance, and the price of a replacement. After considering everything, it suggested that replacement was probably the more sensible long-term option.
ChatGPT is very good at producing prose that has the shape of a conclusion. You can easily forget that some of the evidence was never actually provided. So I asked my follow-up again: “What are you unsure about in your answer?”
“I’m unsure about recommending replacement without knowing the actual repair cost or what is failing. I’m using the laptop’s age as evidence that further problems may appear, but age alone doesn’t tell us whether replacing it is financially sensible,” the chatbot wrote. “If the problem is something relatively cheap, such as a battery or storage upgrade, repairing it could extend its useful life considerably. I’d be much more confident after knowing the model, specifications, symptoms and estimated repair price.”
This time the weakness was obvious once ChatGPT said it aloud. I had essentially asked whether an unidentified problem was worth fixing without saying what the problem was or how much fixing it would cost. Yet ChatGPT had still managed to give me an answer.
Humans are quite comfortable saying they cannot make a recommendation until they know more. ChatGPT has been designed to be helpful, and helpfulness often means filling gaps so that the conversation can continue. Sometimes it means the chatbot has an imaginary set of ideas that it fills in the gaps with regardless of their reality.
Taking it further
The trick can be extended in other ways too. “What are you unsure about right now?”is a good starting point, but the answer can become the basis for another question.
If ChatGPT lists three uncertainties, I can ask which one is most likely to change its recommendation. If it says that important context is missing, I can ask exactly what information it needs from me. Asking “what could I tell you that would make you more sure?” helps set the AI up for better answers.
This is particularly handy for decisions. If I am comparing products or choosing between several options, I can ask ChatGPT for its initial take and then immediately interrogate the weakest part. The process feels far more natural than spending five minutes constructing a giant prompt before the conversation has even started.
There is an obvious limit to all of this. ChatGPT’s description of its confidence is itself generated by ChatGPT. The chatbot can be wrong about what it should be worried about. It can miss a crucial problem entirely. For anything involving facts that matter, asking ChatGPT whether it feels uncertain is no replacement for checking reliable sources.
But for everyday conversations, the trick addresses one of my biggest frustrations with AI surprisingly well, simply by forcing into view some of the invisible scaffolding holding up the conclusions in its answers and determining if they are at all solid enough to rely upon.
