Say you ask ChatGPT for the best plumber in Tampa and get three names. That afternoon, your office manager asks the same question and gets three names too. Only one matches your list, and this time your company is on hers.
Both are real answers, and neither is a ranking. OpenAI's help pages explain why they differ, and our October 2026 study shows by how much. The basics of how assistants choose are in our guide to AI search for home service companies.
Is ChatGPT broken when it gives different answers?
No: by OpenAI's own account, the same question may get different answers. ChatGPT writes a response one word at a time, and there are several plausible ways to finish a sentence. That, OpenAI says, leaves "an inherent element of randomness" in its replies.[1]
Questions about local companies add a second moving part, the web search. OpenAI says ChatGPT may search the web on its own when a question would benefit from current information.[2]
When it uses its search partners, it usually rewrites your question into one or more searches. It may send more specific ones after the first results.[2]
In our October 2026 study, ChatGPT ran 2.13 web searches per answer, across 432 answers. So two answers to one question can rest on different searches, and search results change from day to day. OpenAI also warns that search results and citations can be incomplete, out of date or wrong.[2]
One answer that leaves you out is no reason to rewrite your website. Count a dozen answers first, as the test further down shows.
Does ChatGPT give the same answers to everyone?
No. OpenAI says ChatGPT may use an approximate location from your IP address to find local results. A VPN can shift that location, and sharing your device location is optional and off by default.[2]
If memory is on, ChatGPT may use saved memories when it rewrites a search. Saved location details can steer the results too.[2] So the list you see at the shop may not be the list a homeowner sees across town.
Two habits help when you test. Put the city in every question, which is OpenAI's own advice when local results show the wrong area.[2] And ask with memory off, or signed out, so what ChatGPT has saved about you stays out of the test.
How much does ChatGPT's list change when you ask again?
A lot, going by the 432 answers ChatGPT gave us in October 2026. We asked it nine buyer questions about six home service trades in four US metros, each question twice. In a typical pair of answers to the same question, about a third of all the names were on both lists.
Scored as the names in both lists divided by the names in either, the median overlap was 0.33 over 216 pairs. For the middle half of the pairs, it ran from 0.20 to 0.50.
Claude, compared the same way, kept more of its list: a median of 0.54, also over 216 pairs. The two sit side by side in our ChatGPT and Claude comparison.
We asked outside the ChatGPT app and outside the Claude app, with the city in every question. The apps can answer differently: they may know where you are and remember earlier chats.
ChatGPT's lists were short, too. Across its 432 answers it named 2.75 companies on average (a median of 3), against 6.96 in as many of Claude's (a median of 6). Neither assistant left an answer without a company.
Across the 18 answers in each city and trade, ChatGPT named a median of 20 different companies. Treat each answer as a slice of a longer bench, not as your rank.
Which names come back on a second ask?
The ones at the top hold best. In the same 432 answers, just under half of the names in ChatGPT's first three places came back on a second ask. Of the names below them, just over a third came back.
In as many of Claude's answers, about four in five of the top names came back, and about three in five below them. Our study of Claude's repeat answers found the same pattern over 864 answers, with a median overlap of 0.55 across 648 pairs.
Even the leader isn't on every list. In a typical city and trade, ChatGPT's most-named company showed up in a little over half of the 18 answers there. Count how often you're named, and how high, over many asks.
Does one ChatGPT answer decide who gets the call?
Usually not. Most people who use AI to find a local business keep checking after the answer:
Only 18% of consumers who use AI for local search felt ready to contact the business it recommended; the rest kept researching on Google and other channels.
Source: BrightLocal consumer search behavior study · (opens in a new tab)
And the people asking ChatGPT haven't left Google:
95% of ChatGPT users also use Google—unchanged from Sept 2025 to May 2026.
Source: Similarweb AI search research · (opens in a new tab)
In 2025, Google said clicks to websites held roughly steady:
Google says total organic clicks to websites stayed "relatively stable" year over year.
So a homeowner who got three names from ChatGPT often looks them up on Google and opens their websites. That second look lands on your Google Business Profile and your site. Search your trade and town on Google tonight, then your company name, and look at what a homeowner would see.
How do you check where you really stand?
Ask the same few buyer questions several times, on different days, and count the names. Use a new chat each time and copy the wording exactly. Here's what that looks like for a plumbing company.
A tally for a plumbing company in Tampa
- Three questions, word for word: "Who are the best plumbers in Tampa?", "I need a water heater replaced in Tampa this week. Which companies should I call?" and "Which plumbers in Tampa are licensed and well reviewed?"
- Four asks of each, on four different days, each in a new chat: 12 answers.
- The tally: your company in 5 of 12 answers, named first in 2 of them. One competitor in 10 of 12, another in 7 of 12.
- What it says: you're in the rotation but not the steady core. You never came up for the licensed-and-reviewed question, so check how your license and reviews show on your website and Google profile first.
Illustrative examplemade-up numbers · use your own
Write down the date, the exact question and the names each time. For a fuller version with ten questions and a sheet to fill in, see how to run an AI search audit.
- Don't change your website or Google profile because of one answer.
- Don't judge from your own account alone, for the reasons above.
- Don't treat a screenshot of one answer as proof, whoever sends it.
What can you work on to come up more often?
Work on the same evidence that gets you found on Google: your website, your Google Business Profile, your reviews and your listings. In our 432 ChatGPT answers, nearly half of its citations went to the websites of the companies it named. So say plainly on your homepage what you do, where you work, and your current Google rating and number of reviews.
The other steps, from your Google profile to local mentions, are in how to get recommended by ChatGPT. If it keeps naming the same competitors and not you, see why ChatGPT names competitors instead of you. If it names you with the wrong hours or phone number, trace the wrong detail to its source.
Our AI search visibility audit runs a fixed set of buyer questions for you, dates the results and reports how often you were named. On our AI search optimization plans, every month we run the same buyer questions across the AI assistants we test (currently ChatGPT, Claude and Perplexity). The Expert Action Plan ($149) is the full report plus a 30-minute consultation on what to fix first.
What two answers can't settle
Two answers can't tell you which plumber does the best work, or why one company keeps coming back. Our study measured which companies ChatGPT named, not which list is right.
Our numbers are a snapshot of October 2026, from four metros and six trades. Other cities, trades and wordings can give other lists. ChatGPT and Claude answered at different times that month, so part of the gap between them may come from the timing.
The part you can fix is the evidence: your website, your Google profile, your reviews and your listings. Fix what your tally points to, then run the same questions again next month and compare.