Mod Rewrite Rules for Uniform Spidering

I mention in my earlier post that you should choose to use www or not , but regardless, stick with it. Another issue that arises is that you can’t always control how someone else will link to your site. While many will include www. others will not. To insure that all you links are cataloged in the search engines uniformly, you should implement a couple of rules in your .htaccess file.

  1. A rule to insure www is (or is not) always used.
    To Redirect mysite.com to www.mysite.com
    Options +FollowSymLinks
    RewriteEngine On
    RewriteCond %{HTTP_HOST} ^mysite\.com$ [NC]
    RewriteRule ^(.*)$ https://www.mysite.com/$1 [R=301,L]    To Redirect www.mysite.com to mysite.com
    Options +FollowSymLinks
    RewriteEngine On
    RewriteCond %{HTTP_HOST} ^www.domain\.com$ [NC]
    RewriteRule ^(.*)$ https://domain.com/$1 [R=301,L]
  2. A rule to always redirect your home page (index.html, index.php, default.htm, etc) to just root (https://www.yourwebsite.com/)
    REDIRECT 301 "index.htm" https://www.somewebsite.com

There’s is a good guide on mod rewrite here  at the source.

While it won’t apply to most of the people reading this blog, it is also important not to have multiple views of the same information. So if you have a site with a page of toasters and you give the browser the option to sort by price, color, etc, be sure that the spiders only see one sort of the information.

Related Posts: