This is not anything I recommend you to do on your site or blog. But if you don’t want your site or information to be searched for on Google, Yahoo! etc this is how you do.

It’s actually easy, you just need to add some meta-tags in your HTML code to stop the search engine “spiders”. In my post “Accessibility is basic SEO knowledge” I explained very shortly what “webcrawlers” and “spiders” is and how they work.

This meta-tag tells the “spiders” that they shouldn’t index your site.
< meta name=”ROBOTS” content=”NOINDEX” />

If you add “NOFOLLOW” the “spider” won’t follow and index the links on your site.
< meta name=”ROBOTS” content=”NOINDEX, NOFOLLOW” />

To prevent that the “spider” archives your site you simply add this meta-tag:
< meta name=”ROBOTS” content=”NOARCHIVE” />

Place these meta-tags below the title tag in your HTML document!

Please note that the search engines should support these meta-tags, but like always there are no guarantees for that, at least Google say they support them.

You can also add a ordinary txt file on your server where you can tell the “spiders” what sites they can and can not index and a few other things. Read more about this over at robotstxt.org/wc/norobots.html