December 9, 2008 ByDennis Dornon
Recently John Mueller, a Webmaster Trends Analyst for Google, reported on his personal blog that hackers have been stealing PageRank from websites.
In this particular case, a "massive amount" of sites were hacked and likely redirected through suomi.co.in.
The Hackers were able to 301 redirect pages for search engine spiders only so the webmaster and site visitors never saw a change on the site. By redirecting the Spiders this way the Google believed the site had moved and began giving PageRank and inbound link credit to Suomi.co.in site. Thereby stealing the work of the original website for their own benefit.
John Mueller then goes on to show you two ways that a site can detect this if they feel they are a victim.
The first way assumes that you have actually noticed that URLs from your website are no longer being listed in the Search Engines.
1. Use an online tool such as Web-Sniffer. It's pretty easy to use and is somewhat close to an actual crawler.
2. Use FireFox with the User Agent Switcher plugin. If you use this plugin, you'll have to add the user agent yourself. I usually use the current Googlebot user agent string:Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Note: if you use Firefox for this, make sure that your Firefox installation is up to date and locked down properly in case you run into a site serving malware like this. Sometimes it even makes sense to use a virtual machine for this.
3. (I wish there were a half-"li" ) There's also "wget", which is easy for those of you who prefer use console tools. I usually use the above user agent string with wget.
If you access the site using one of these tools, you'll often be able to spot these redirects (or other issues that a site might be having with regards to being accessed by search engine crawlers).
Below is an image of what you would see if this is happening with your website. Please note that Suomi.co.in could be any website that is not yours.
The second way is actually very easy and something you should implement on all your sites anyways.
When you submit a Sitemap file, Google will show warnings for URLs that redirect. By design, you should be listing the final URL in your Sitemap file, so if the URL is redirecting for our crawlers (as in this case), we'll show a warning in your account.