When a homeowner asks Claude for a roofer and then asks again, do the same companies come back? We asked each buyer question four times in each city and trade, compared the lists, and counted how often each company was named.
Original study · v1.0
Ask Claude twice, get a partly different list of contractors
We asked Claude the same buyer questions again and compared the lists of contractors: how much changed, which names held, and what “near me” without a city brought back.
By Found ClearlyPublished October 29, 20265 min read
Part 1 of 4 in our study How Claude picks contractors, with the method and the other parts.
Found Clearly sells AI-visibility audits and plans.
Can’t find your company?
Find out why with a free check of your website and Google profile, done by hand.
Abstract
In October 2026 we asked Claude 9 buyer questions about 6 home service trades in 4 US metros, each question four times in each city and trade: 864 answers, 36 for each city and trade. Asked the same question again, Claude named a partly different list: the two lists of companies overlapped by a median of 0.55 (Jaccard: the names in both lists divided by the names in either). The first 3 names held up better: 79% came back on a repeat, against 61% of the names below them. Per city and trade Claude named a median of 43.5 different companies, and a median 29.7% of them came up in one answer only. In 48 more answers to “near me” questions without a city, Claude said every time that it did not know where the buyer was.
Key points
- Ask twice, get a partly different list: the two lists of companies overlapped by a median of 0.55 (648 pairs).
- The first 3 names held up better: 79% came back on a repeat, against 61% of the names below them.
- A short core and a long tail: per city and trade, the most-named company was in a median 81.9% of the answers, and a median 29.7% of all the companies Claude named came up in one answer only.
- “Near me” without a city: in 48 of 48 answers, Claude said it did not know where the buyer was. Put the city in the question.
Our data · Oct 2026 · Claude · Method →
Repeat answers
Ask twice, get a partly different list
Most of this page’s numbers come from 864 answers: each question asked four times in each city and trade, 36 answers for each.
We compared each pair of answers to the same question by the companies they named. The score is the Jaccard index: the names in both lists divided by the names in either list. Two lists of 6 names that share 4 hold 8 names between them, so they score 4 ÷ 8 = 0.50. In that example a third of each list changed, not half.
0.55
Asked again: median overlap of the two lists (middle half of pairs 0.40–0.71; 648 pairs)
0.50
Asked again on another day: median overlap (0.38–0.67; 648 pairs)
79%
of the mentions in an answer's first 3 places came back on a repeat (3,862 mentions)
61%
of the mentions below them came back (5,180 mentions)
The first names hold and the tail moves. Claude’s searches were steadier than its lists: in the same two comparisons, the pages its searches returned overlapped by 0.64 and 0.58.
Our data · Oct 2026 · Claude · Method →
Who gets named
A short core and a long tail
For each of the 24 city and trade pairs we counted in how many of the pair’s 36 answers each company was named. A company named twice in one answer counts once.
43.5
different companies per city and trade (median; 28–56)
81.9%
of the answers named the most-named company (median)
22.2%
of the answers named the 10th most-named company, and 2.8% the 30th (medians)
29.7%
of the companies came up in one answer only (median)
Per city and trade: 36 answers in each of 24 pairs
| Measure | Median | Range | All pairs together |
|---|---|---|---|
| Different companies named | 43.5 | 28–56 | 1,023 in all |
| Companies named in one answer only | 29.7% | 17.6%–53.8% | 30.7% (314 of 1,023) |
| Answers that name the most-named company | 81.9% | 72.2%–91.7% | 82.4% (712 of 864) |
| Answers that name the 10th most-named company | 22.2% | 13.9%–27.8% | 22.0% (190 of 864) |
| Answers that name the 30th most-named company | 2.8% | 0.0%–8.3% | 4.3% (37 of 864) |
| Share of all mentions that go to the 10 most-named | 66.4% | 56.8%–81.1% | 66.1% (3,978 of 6,017) |
| Companies named per answer | 7.01 | 5.67–7.97 | 6.96 |
No company was named in all 36 answers of its pair; the most any company reached was 33. In just 1 of the 24 pairs Claude named fewer than 30 companies, so its 30th counts as zero.
We added these counts after the answers were collected. They describe the data and test no hypothesis.
The companies behind these counts, city by city: who Claude names most often in San Antonio, Kansas City, Indianapolis and Richmond.
Our data · Oct 2026 · Claude · Method →
Location
“Near me” without a city
We also asked the three discovery questions (the first three in the method’s list) with “near me” in place of the city, for roofing and bathroom remodeling in Kansas City and Richmond, each asked four times: 48 answers. Claude’s web searches had the buyer’s location.
48 of 48
answers said Claude did not know where the buyer was
With the city in the question, every one of the 2,376 answers in the main study named at least one company. For buyers, the advice is short: put the city in the question. Our “near me” answers treated the buyer’s location as unknown. The Claude app can answer differently, because it may know where you are.
Our data · Oct 2026 · Claude · Method →
What this cannot tell you
- We asked Claude outside the Claude app. The app can answer differently: it may know where you are, remember earlier chats and use other settings.
- Claude and the way it searches the web change over time, so read this as a snapshot of October 2026.
- The study covers 4 metros and 6 home service trades. Other cities, trades and wordings can give other lists.
- The core-and-tail counts were added after the answers were collected. They describe the data and test no hypothesis.
- We measured which companies Claude names, not whether those companies do good work or whether the lists are right.
How to cite this
Found Clearly. "Ask Claude twice, get a partly different list of contractors", from "How Claude picks contractors", version 1.0. Published 2026-10-29. Data: Claude (Anthropic), October 2026.
The text and the figures on this page are under CC BY 4.0: quote, chart and republish them with credit to Found Clearly and a link to this page. That includes competitors.
Primary sources
FAQ
Related questions
Does Claude name the same contractors if you ask the same question twice?
Only partly. When we asked Claude the same buyer question again, its two lists of companies overlapped by a median Jaccard of 0.55 (648 pairs; our data, October 2026). The first 3 names held up better than the rest: 79% came back on a repeat, against 61% of the names below them.
What happens when you ask Claude for a contractor “near me”?
In our study, Claude answered as if it did not know where the buyer was. In 48 of 48 answers to “near me” questions it said so, even though its web searches had the buyer's location. With the city in the question, every one of the 2,376 answers in the main study named at least one company. Put the city in the question. The Claude app can answer differently, because it may know where you are.
How many contractors does Claude name for one trade in one city?
In our data, a median of 43.5 different companies per city and trade across 36 answers, and a median 29.7% of them came up in one answer only (our data, October 2026). The company Claude named most often was in a median 81.9% of the answers.
Keep reading
Evidence you can check. Public prices, month to month.