Google now tells platforms that pages users create for sharing, collaboration or one-off utility should be kept out of its index by default. Its October 26, 2026 update to the spam-prevention guide says to apply noindex or block crawling in robots.txt. Those two are not equivalent: Google's own documentation says a robots.txt block hides the noindex and can still leave the URL in search results. For pages that must stay out of the index, noindex is the one that works.

The change targets a specific kind of abuse — spammers using a platform's share-link, generator or app-builder features to mint thousands of indexable pages on a trusted domain. But the fix touches any product that lets users publish something with a URL.

Key takeaways

  • On October 26, 2026, Google added an "Only index content meant for Search" section to its guide on preventing user-generated spam.
  • Pages made "for direct sharing, collaboration, or utility rather than public search discovery" should get noindex (robots meta tag or X-Robots-Tag header) or a robots.txt block by default.
  • Google's documentation also says a page blocked by robots.txt "can still be indexed if linked to from other sites" — and the crawler "will never see the noindex rule."
  • Retrofitting is slow: Google says "it may take months for Googlebot to revisit a page," so a noindex added later takes time to land.
  • Google now asks platforms to alert on spikes in new public pages and on "a small number of users generating a large share of indexable URLs."

What exactly did Google change?

The update landed in Google's guide to preventing user-generated spam, logged in the Search Central changelog as guidance on "blocking indexing of link-shared content" and "monitoring abuse in Search Console." Google's stated reason: to help platforms keep user-created pages that aren't meant for Search out of the index, and detect spam earlier.

Part of the guide What it now says
New section: "Only index content meant for Search" If a service lets users create shareable pages, generated outputs or interactive tools, consider whether each is meant for public search discovery
Default handling Pages made for direct sharing, collaboration or utility: apply noindex via robots meta tag or X-Robots-Tag, or block crawling in robots.txt, by default
Monitoring Check the Page Indexing and Search performance reports regularly for unexpected spikes in indexed pages, impressions or off-topic queries
Alerts Set alerts for spikes in newly published public pages, and for a few users generating a large share of indexable URLs

The change sits in a wider cleanup. Five days earlier, on October 21, Google refreshed its "control what you share with Google Search" page to list every option in one place — including the robots meta tag, data-nosnippet, X-Robots-Tag, and the Search Console control that sets whether a site appears in AI Overviews and AI Mode.

Which pages count as "link-shared content"?

Google does not publish a list, but the language — "shareable pages, generated outputs, or interactive tools" — maps cleanly onto common product features:

Feature Example Default indexing
Share links Shared chat transcripts, shared docs, shared boards noindex
Generated outputs AI-generated reports, images, resumes, quotes noindex
User-built tools and apps Calculators, forms, mini-sites built in a no-code tool noindex
Preview and staging URLs Draft links sent for review noindex (or password-protect)
Curated public profiles or listings the user chose to publish A portfolio page, a public template gallery Indexable, ideally after the account earns trust

The distinction is intent. A page someone creates to send to a colleague was never meant to compete in search. If it gets indexed anyway, the platform's domain lends it credibility it has not earned — which is exactly what spammers exploit.

Why does "noindex or robots.txt" matter so much?

Because Google's own documentation says they do different jobs, and only one of them keeps a URL out of results.

noindex robots.txt Disallow
What it controls Indexing Crawling
Can the page still appear in results? No, once Google recrawls it Yes — "can still be indexed if linked to from other sites," shown without a description
Does Google see your instructions on the page? Yes No — "the crawler will never see the noindex rule"
Best for Keeping shared pages out of Search Managing crawl load on huge URL spaces
Risk if misused Slow to take effect on pages already indexed False sense of privacy; URLs leak into results via outside links

Google's robots.txt documentation is blunt: it "is not a mechanism for keeping a web page out of Google." Shared links are, by definition, pasted into emails, chats, forums and social posts — exactly the outside links that let a disallowed URL get indexed anyway. Search Console even has a status for it: "Indexed, though blocked by robots.txt."

The practical rule:

  1. Use noindex for shareable pages. An X-Robots-Tag: noindex header is easiest to apply at the route level and also covers PDFs and other non-HTML files.
  2. Do not also block those URLs in robots.txt. Combining the two defeats the noindex.
  3. Use robots.txt only for crawl control — for example, infinite parameter spaces where crawling itself is the problem.
  4. Password-protect anything private. Google lists authentication alongside removal as the reliable way to keep content out entirely.

What happens when a platform gets this wrong?

There is a well-known recent example. In mid-2025, ChatGPT's share dialog included an opt-in checkbox to "make this chat discoverable," with a smaller, greyed explanation that it allowed the chat to appear in web searches. Thousands of shared conversations — some containing personal details — turned up in Google results. OpenAI withdrew the feature around July 31, 2025, calling it a "short-lived experiment" that created "too many opportunities for folks to accidentally share things they didn't intend to."

That case was a privacy failure rather than a spam attack, but the mechanics are the same ones Google's update addresses: a share feature on a high-authority domain, an indexing default that was too permissive, and a cleanup that depended on search engines recrawling and dropping pages one by one.

Why should the default be noindex from day one?

Because removal is slow. Google's documentation says it "may take months for Googlebot to revisit a page," depending on how important the page seems, and a noindex only takes effect after that recrawl. Search Console's own guidance says fix validation "typically takes up to about two weeks," and new pages take "a week or so" to start getting crawled and indexed.

Put those together and the asymmetry is clear. A spammer can generate and get thousands of URLs indexed within days. A platform that reacts afterwards needs weeks to months to get them back out — while the abuse is still visible on its domain. Shipping the default up front costs nothing; retrofitting it is a project.

What should a platform monitor in Search Console?

Google's new guidance names the reports and the signals. Turned into a checklist:

Signal Where to look What it suggests
Sudden jump in indexed pages Page Indexing report Share links or generated pages leaking into the index
Impressions for queries unrelated to your product Search performance report Spam riding on your domain (pharma, gambling, piracy terms)
"Indexed, though blocked by robots.txt" growing Page Indexing report You are relying on robots.txt where you need noindex
Spike in newly published public pages Your own analytics Automated account abuse
A few accounts creating a large share of indexable URLs Your own database A single bad actor or bot farm

Two cautions on reading these reports. Indexing reports are not real-time, so check the date range on each chart before drawing conclusions. And a flat indexing chart on a stable site is normal — the Search performance report tends to show real change first, through impressions and queries.

Frequently asked questions

What did Google change in its spam guide on October 26, 2026?

It added guidance to keep user-created pages meant for sharing, collaboration or utility out of the index by default, using noindex or robots.txt, and to monitor Search Console for unexpected spikes in indexed pages.

Is blocking a page in robots.txt the same as noindex?

No. Robots.txt stops crawling, not indexing. A blocked URL can still be indexed if other sites link to it, and Google cannot see a noindex on a page it is not allowed to crawl.

Should I use noindex and robots.txt together?

Not on the same URL. If robots.txt blocks the page, Google never reads the noindex, so the page can still appear in results. Use noindex alone for pages you want kept out of Search.

How long does noindex take to remove a page from Google?

It works after Google recrawls the page, which Google says can take months for less important URLs. You can request a recrawl with the URL Inspection tool to speed up individual pages.

Does this update apply to small sites?

Mostly it targets platforms where users can create pages — SaaS tools, AI generators, no-code builders, community sites. A site where only your team publishes content is largely unaffected.

Can I stop my site appearing in AI Overviews?

Google's updated "control what you share" documentation lists a Search Console control for a site's inclusion in generative AI features on Google Search, including AI Overviews and AI Mode.

Check your noindex and robots rules now

Before a spike shows up in Search Console, confirm what your pages are actually telling crawlers. The free meta tags checker shows the robots directives a URL serves, and the AI crawler checker shows what your robots.txt allows and blocks for each bot.

If your platform generates pages at scale, read why Google loses faith in sites built on programmatic SEO — the same site-level trust logic is what makes indexed spam on your domain so costly.

Sources