Is there a way to stop a page from being ever found by a search engine? I mean can you have a special header or something so that you can only find the page by typing in the URL / following a link? This would be useful as although I have a proper login thing for my admin area, if I do a particular search with google, the login page is found, and I don't want that page advertised.
thanks!
Just don't put any link to your login page since search engines follow links on your html pages recursively.
If you must deny access with a HTTP_REFERER set, try this:
RewriteEngine on
# if request_uri is your login.html
RewriteCond %{REQUEST_URI} ^/url_path/to/login\.html$
# and if http_referer is not empty
# opening your login.html from bookmark or by typing the url has a blank http_referer
RewriteCond %{HTTP_REFERER} !^$
# [403] it
RewriteRule ^(.+) - [F]
Forgive me for not understanding the code you provided me with, but it is useful to know that if the page has no hyperlink to it, it is not found, so I will replace the hyperlink with a picture of the hyperlink, in a form, that goes to the login page instead.
Thankyou for your help!
Don't know if you've done this already, but you might want to make up a robots.txt file to instruct the indexing robots what files you want left alone. It's not always acknowledged by robots but most of the time it is:
http://www.searchengineworld.com/robots/robots_tutorial.htm