The New Research That Should Make You Question Your AI Rankings

According to a report from Search Engine Journal, new research shows that AI visibility rankings change so much from one query to the next that most of the movement is just statistical noise. This matters because many businesses now rely on those numbers to decide their SEO strategy. If the data is unreliable, decisions based on it could be a waste of time and money.

The study, led by the team at IQRush, tested how often different AI tools like SearchGPT and Gemini give different answers to the same question. The results show that a single measurement is almost meaningless. You need to run the same query many times before the rankings start to stabilize. And even then, the number of queries required varies wildly by platform and topic.

Why AI SEO Is Trickier Than Traditional Search Rankings

Traditional Google rankings are relatively stable. You check your position, it stays roughly the same for days or weeks. With AI search tools, every answer is a fresh roll of the dice. The models are designed to add randomness, so the same question can pull different sources each time. This means a three-point jump in your citation share could be real progress—or just a lucky roll.

The real problem is that most AI visibility dashboards show a single number and call it a ranking. But as the research proves, that number is just one snapshot of a moving target. Without repeated measurements, you cannot tell if you are ahead of a competitor or if the difference is just noise. For Australian businesses trying to decide where to invest their marketing dollars, this uncertainty is dangerous.

What This Means for Australian SMBs

Small and mid-sized businesses in Australia often have limited budgets. They cannot afford to chase fake trends. If your SEO dashboard says your AI visibility went up after a content update, you might be tempted to double down on that strategy. But the research shows that a single before-and-after reading cannot separate real improvement from random fluctuation. You could be making decisions based on a coin flip.

This is especially tricky for local businesses that rely on appearing in AI answers for "plumber near me" or "best café in Melbourne." The AI might cite you one day and ignore you the next, not because your content is bad, but because the model is inherently unstable. Understanding this helps you avoid panic and make smarter, data-driven moves.

What You Can Do Now

  • Ask your SEO provider how many times they run each query before reporting a ranking. If they only run it once, demand a repeat test over several days.
  • Focus on building strong, authoritative content that stands out clearly from competitors. When your site is far ahead, the noise matters less.
  • Track your own AI mentions manually using free tools for a few weeks. Look for a consistent pattern, not a single spike.
  • Before investing in paid AI visibility tracking, check if the tool shows a margin of error or a range of results, not just a clean number.
  • If you see a change after a content update, wait and measure again at least three more times. A pattern that lasts is real; a one-off bounce is noise.

At MS&VG, we help Australian small and mid-sized businesses navigate this shifting landscape with our AI-powered SEO service. Instead of chasing unstable numbers, we focus on building content that earns reliable citations over time. If you want to make sense of your AI visibility without the noise, we can help you see the signal.