Google and I are on an on again off again kind of relationship with one of my websites. Right now, we’re on the “off again” stage. I really like Google and know that it’s a super powerful weapon; the only weapon of choice for some marketers.

For the last few weeks, I noticed that my website had completely fallen off the face of Google’s rankings for a certain keyword (I’ll reveal my work for this particular site someday…I promise) only to start making its way back…first at around 120 then slowly back to the top hundred, to the top fifty, to the top 10, and finally resting at number 7 for about two weeks. I was very happy with this result as I was making a few bucks every day. Every visitor to that website ended up netting me about $2.12, so the ranking at #7 was working well for me and I was trying to get the website to rank higher.

At that point in time, roughly 3 weeks ago, I loved Google for that website! This website is also my highest earner, so when I noticed that, without any changes on my part, the site was ranking at roughly at #120 I was very shocked, surprised, and confused.

For the last three weeks I have been working on redesigning the website and trying to figure out exactly what made it fall from grace. I know its not the competition…no offense to their webmasters but their sites are bad and have very little original content and my website is completely original with the exception of a little bit of text taken from the affiliate program’s website. All in all, my site is more original than a lot of Hollywood movies and certain TV shows.

Last week I found out about Google’s supplemental index and how it can impact your SERP. Google’s supplemental index, from what I gather, is where bad pages go to pasture. The supplemental index is where Google places pages that are either duplicate in content to other pages of the website or pages that it considers low in quality. The supplemental index is also a problem for anyone running a Wordpress blog, like me. Wordpress creates more than 1 copy of each page upon publishing: 1 on the main index page and one on the archive as a permalink. So every time that any Wordpress blogger publishes a new page, Wordpress creates (at least) 2 pages and Google sends those pages to the supplemental index, eventually.

What can you do to fix the supplemental index problem? I’m not exactly sure how to go about removing pages from Google’s supplemental index, but I know that it is possible to control the Google Bot for future crawls by having a well done robots.txt file. This is what I have done to my robots.txt file in order to control my supplemental index problem:

sitemap: http://www.URLGOESHERE.com/sitemap.xml

User-agent: *
Disallow: /backup/
Disallow: /cgi-bin/
Disallow: /design/
Disallow: /images/
Disallow: /blog/wp-admin/
Disallow: /blog/wp-includes/
Disallow: */page/*
Disallow: */wp-images/
Disallow: /archives/
Disallow: */trackback/
Disallow: */feed/
Disallow: /*?*

User-agent: Googlebot-Image
Allow: /wp-content/uploads/

User-agent: Mediapartners-Google
Allow: /

User-agent: duggmirror

Disallow: /

There is a lot to talk about here, so the details will have to be for another post. The long and short of it is that I want Google to not go to certain parts of my website and hope that from here on out, my supplemental index entries will be kept to a minimum.