Updated October 20, 2026

Home › Extensions › URLSweep › Guides › Remove a staging or dev site from Google

How to Remove a Staging or Dev Site From Google

By the Pick Ads team · Tested on Chrome · Published October 20, 2026 · Updated October 20, 2026

Answer: Verify the staging host in Search Console, then use the Removals tool's "Remove all URLs with this prefix" option, for example https://staging.example.com/, to hide it within about a day. Then fix it for good with HTTP authentication, an X-Robots-Tag: noindex header, or 404/410 responses. Don't rely on robots.txt.

Who this is for

Agencies and developers whose staging, dev or preview host shows up in Google results, often with duplicate content from production.

Why staging gets indexed

Google finds URLs through links and sitemaps. Google says it can index any URL it finds unless the page carries a noindex directive, and someone else's link to your page counts too (Page indexing report). Common leaks include links from production, a staging sitemap, a shared preview link that gets posted publicly, or a robots.txt Disallow that someone assumed would hide the site.

Immediate removal (prefix)

Add the staging host as a property in Search Console and verify it. The Removals tool only works for URLs in a property you own (Removals tool).
Open Removals → Temporary Removals → New Request.
Choose Temporarily remove URL → Remove all URLs with this prefix.
Enter https://staging.example.com/.
Submit. Google says processing usually takes up to a day.

Production is safe if you enter the right prefix. Google says blocking https://staging.domain doesn't block www.domain or the root domain. But blocking https://www.domain does also block https://domain. Double-check the host before you submit.

The removal lasts about six months and doesn't stop crawling. See how long a removal lasts.

Permanent fix

Method	Example	Notes
HTTP authentication	Basic auth on the whole staging host	Google's Page indexing report lists 401 pages as not indexed. Google can't see the content, and neither can anyone else without credentials.
`X-Robots-Tag: noindex` header	Set at the server or CDN for the whole host	Works for every file type, including PDFs (noindex)
404 or 410	Retire an old staging host	Google removes URLs that return 4xx from its index

Choosing between them: Removals vs noindex vs 410.

The robots.txt trap

Google says robots.txt isn't a way to keep a page out of Google. A disallowed URL can still be indexed if it's linked from elsewhere, just without its content (robots.txt introduction). Worse, if robots.txt blocks crawling, Google never sees your noindex. Remove the Disallow until the pages have dropped out, and let Google crawl them to see the noindex or the 401.

Many leaked URLs → bulk

A prefix request usually covers a whole host. If the leaked URLs are spread across hosts or paths you don't want to remove entirely, queue a reviewed list with URLSweep in your Removals tab. See bulk remove URLs from Search Console.

Prevent it next time

Put HTTP authentication on every non-production host.
Send X-Robots-Tag: noindex from the server on non-production environments.
Don't publish a sitemap for staging, and never link to staging from production.
Check the deploy before going live: production must not inherit the staging noindex header.

Verify

Check the request status in Removals.
Inspect a staging URL in URL Inspection and confirm Indexing allowed? shows noindex, or that the fetch returns 401. Remember the indexed result reflects the last crawl (URL Inspection tool).
Open a staging URL in an incognito window and confirm it asks for credentials.

Last verified: 2026-10-20

FAQ

Can I remove a site I don't own? Not with the Removals tool. It only works for URLs in a Search Console property you own. If you don't control the staging site, ask its owner to lock it down. If its content is already gone, Google's Refresh Outdated Content tool can update the old result.

How fast? Google says Removals requests usually take up to a day and aren't guaranteed. The permanent fix depends on Google recrawling the staging URLs, which can take longer for pages it rarely visits. Keep the temporary removal in place while that happens.

Is robots.txt enough? No. Google says robots.txt manages crawling and doesn't keep pages out of Google. A disallowed URL can still be indexed if it's linked from elsewhere, and blocking crawling hides your noindex from Google. Use HTTP authentication or a noindex header instead.

Will production be affected? Not if you enter the staging host exactly. Google says a prefix removal for a staging subdomain doesn't block the www or root domain. Be careful the other way round: removing the www prefix also removes the non-www version. Also make sure the staging noindex header never ships to production.

How to remove a URL from Google: all removal cases.
How long a Google removal lasts: why the permanent fix matters.
Bulk remove URLs from Search Console: working through a list of leaked URLs.

Clear leaked URLs in bulk with URLSweep - Free

Sources

Removals and SafeSearch reports tool – Search Console Help: https://support.google.com/webmasters/answer/9689846?hl=en
Page indexing report – Search Console Help: https://support.google.com/webmasters/answer/7440203?hl=en
Block Search indexing with noindex: https://developers.google.com/search/docs/crawling-indexing/block-indexing
robots.txt introduction and guide: https://developers.google.com/search/docs/crawling-indexing/robots/intro
How HTTP status codes affect Google's crawlers: https://developers.google.com/search/docs/crawling-indexing/http-network-errors
URL Inspection tool – Search Console Help: https://support.google.com/webmasters/answer/9012289?hl=en