When Matt Cutts Talks…
… people listen. And recently he’s been talking! I came across this Stone Temple interview with him (no, not those stone temples) where he talks at length about NoFollow and NoIndex and why you might want to use them.
As with most anything he writes, it is well worth a read - this interview was particularly frank about things - a lot of times I find that there’s a bit of ambiguity when he speaks because of all the G’s secrets. These two tags are your website’s ticket to telling Google what the deal, yo. A NoIndex meta tag on any page means that you do not want it showing up in Google’s search results - stick something like this in your page’s header and poof no mo’:
<meta name=”robots” content=”noindex”>
Quick and brutal. It is similar to using a Disallow in your robots.txt file except that Google will still actually crawl the page - thus it still gets PageRank and it still can pass that link juice on to other pages that it links to, it just won’t show up on search results pages. This is distinct from robots.txt which will actually prevent Google from crawling the page at all (thus no page rank and any pages linked to from that page will not be passed any link juice).
Matt and the interviewer give some examples of good uses of NoIndex:
For example you might want to have a master Sitemap page and for whatever reason NoIndex that, but then have links to all your sub Sitemaps.
Another example is if you have pages on a site with content that from a user point of view you recognize that it’s valuable to have the page, but you feel that is too duplicative of content on another page on the site
Another good example is, maybe you have a login page, and everybody ends up linking to that login page. That provides very little content value, so you could NoIndex that page, but then the outgoing links would still have PageRank.
The other thing they talk about is NoFollow - this is a metatag as well as a link attribute. So you can apply it to a page:
<meta name=”robots” content=”nofollow”>
<a href=”/some/url” rel=”nofollow”>link text</a>
The tag itself tells google not to follow any links off that page, similar to robots.txt. So if google crawls a page that has the nofollow meta tag on it it will not crawl to other pages that are linked off it - but unlike robots.txt it may still crawl those other pages if they are linked to from other pages that don’t have the nofollow tag. If you apply it to a particular link Google won’t follow that specific one - thus NoFollow is very granular.
The idea here is to focus the link passing on your site - so the example Stone Temple talks about is:
What we’ve been doing is working with clients and telling them to take pages like their about us page, and their contact us page, and link to them from the home page normally, without a NoFollow attribute, and then link to them using NoFollow from every other page. It’s just a way of lowering the amount of link juice they get.
I was also surprised to read that even in the event of a robots.txt preventing them from crawling the page, Google may still actually show a search result for a link. That is - they retained the fact that a url was linked to and kept a bit of information on it (even pulling a summary from the ODP). The only surefire way to keep things out of the results pages is to allow Google to crawl it but give it a NoIndex tag. That’s mildly nutty…
And so, when someone came to Google and they typed in eBay, and we haven’t crawled eBay, and we couldn’t return eBay, we looked kind of suboptimal. So, the compromise that we decided to come up with was, we wouldn’t crawl you from robots.txt, but we could return that URL reference that we saw.
The interview is long, in depth and fantastic - they talk about much more than what I just wrote about - those just happen to be the things I found most interesting. Having concrete examples helped me enormously to understand these two fairly simple concepts - simple in the SEO world at least. Even more concrete are the two postings on performancing.com, one with examples of using NoIndex and another for NoFollow both provide details on how and why which were enlightening.
