Robots.txt vs noindex | SEO Forum - Moz

I recently started working on a site that has thousands of member pages that are currently robots.txt'd out.

Should I use noindex or robots to remove pages from the Google ...

Yes, remove the robots.txt disallow for that section and add a noindex tag. Noindex is the only sure-fire way to de-index URLs, but the crawlers ...

Custom Result

This is a custom result inserted after the second result.

Should I use meta noindex and robots.txt disallow? | SEO Forum - Moz

Best of both worlds: crawl budget isn't overtaxed by thousands of noindex pages, and internal links used to index robots.txt-disallowed pages ...

Robots.txt vs. meta noindex, follow | SEO Forum - Moz

The problem with robots.txt is that it doesn't always keep pages from being indexed especially if there are other external sources linking to ...

No index tag robots.txt | SEO Forum - Moz

Hi Jens You can't add a noindex in the Robots.txt file. Firstly you need to add a noindex tag to all of the pages in the /node/ directory.

Easy Question: regarding no index meta tag vs robot.txt - Moz

This seems like a dumb question, but I'm not sure what the answer is. I have an ecommerce client who has a couple of subdirectories ...

Why is noindex more effective than robots.txt? | SEO Forum - Moz

The disallow in robots.txt will prevent the bots from crawling that page, but will not prevent the page from appearing on SERPs. If a page with ...

Robots.txt & url removal vs. noindex, follow? | SEO Forum - Moz

The preferred option would be the noindex, follow tag. The robots.txt file is a choice of last resort. The best robots.txt file for a site ...

Robots.txt vs noindex | SEO Forum | Moz

I think what I am going to say is going to sound like it is going against the grain, but it really isn't. I have noticed in some places if you want an ...

Good to use disallow or noindex for these? | SEO Forum - Moz

Disallowing these URLs in your robots.txt file might seem like a good way to save the crawl budget, but it can have unintended side effects.