Why Does ChatGPT Recommend Different Businesses Each Time?

A plain-language note on a University of Toronto paper. The numbers below are theirs.

You can ask the same “near me” question twice and get two different lists. That is common. Researchers at the University of Toronto measured how much the list changes.

The paper is Location, Location, Generation: Where You Ask Matters for Generative Engine Optimization, by Mahe Chen and Nick Koudas. Every number on this page comes from that paper.

Their setup is not the phone chat you open at home. They used ChatGPT through a programming interface, model gpt-5.4-mini, with web search on. They told that interface a city. They also used Gemini, model gemini-3.1-flash-lite, with Google Search, from different places on the network. A normal chat may pick up your location in another way.

What the research found

The paper counts the businesses, brands, and products a near-me answer names. A near-me answer from ChatGPT names 4.3 of them on average.

Two identical requests share 23% of these names. Two different declared locations share 4%.

Identical means the same question and the same city, sent twice. A declared location is a city the software was told. The question itself did not name a place. The 4% line is that same question with a different city. Most names change even when nothing else changes. When the city changes, the lists barely overlap.

The authors wrote 300 questions. None of them named a city. They collected 31,200 answers, and they asked each question twice so they could see the noise of asking again.

Why your city changes the answer

ChatGPT follows the city an application declares, down to that city. For near-me questions, it writes the place into 100% of the searches it runs. The search system behind it can still favor local pages when those search words name no place.

The paper says this does not fade with distance. To ChatGPT, every other city is a different place. A check set to another city shows that city’s list.

ChatGPT also ignores the network in this study. When only the country of the internet connection changed, its sources stayed inside the noise of asking the same question twice.

ChatGPT vs Gemini: where they get your location

The paper says the two tools take location from different channels. ChatGPT uses the declared city and ignores the network. Gemini ignores a declared location and ignores map coordinates. It follows the country of the network.

Gemini’s searches name the country in 56% of near-me answers, and they never name a city. Between New York and Los Angeles, the paper’s index for Gemini’s sources is 0.00. Those two cities did not move the sources. Between countries, the sources do change, including for some questions that are not about a place.

Asked “credit unions in my area,” ChatGPT named different local institutions for New York, Boise, Miami, and London. Asked “which store has the cheapest groceries,” Gemini named different chains by country, such as No Frills in Canada, Tesco in the UK, and Woolworths in Australia. In the US rows, East and West both start with Aldi and Walmart, then split: Costco in the East, WinCo Foods in the West.

What this means for your business

If your customers are in one city, a test set to another city answers for that other city. For ChatGPT, the paper says no city stands in for another, even a nearby one. Each city you serve needs its own test.

For Gemini, the country is the level that moved the answers. One network location per country was enough in this study. A business in several countries still needs a look in each country. That includes some questions that never mention a place.

When you compare two dates, keep the location fixed. The paper says comparisons over time are valid only when the location stays the same. A new city can look like a swing when the tool is answering for a different place.

How to test fairly (same question, several runs)

The 23% figure is the warning. Same question, same city, two runs, and the lists still share only a slice of the names. One answer is a snapshot.

• Use the words a customer would type. If they say “near me,” keep those words.
• Hold the city still when you compare runs. A new city is a new test.
• Ask more than once. The researchers used two runs so the noise was visible.
• Save every name, not only a yes or no for your own business. You can appear once and be missing the next time.
• If you serve more than one city, repeat the test in each city.
• Write down how the location was set. This study declared a city for ChatGPT and used the network country for Gemini. Keep your own setup the same each time you compare.

Check if AI tips you

TippedByAI checks whether ChatGPT, Grok, Perplexity, and Gemini say your business name. Claude is coming soon. The free check is one question, one time. It is a snapshot. It does not copy the University of Toronto method, and one result cannot show a trend.

The tracker asks the same questions again each week, for €19 a month. You pay in Stripe before the first scan, and you cancel in Stripe. A weekly check can show whether the names moved. One check cannot.