RSS feedThis site is pretty dependant upon Google, with it bringing in around 40% of the traffic. Another 40% is from other people linking to it, and about 15% is from blogging search/monitoring engines. Google seems to crawl this site every few days, which means quality and stability, not always in that order, are more important than ever before.
I usually make most changes to the site here on the live version, which means it breaks every now and then, as you’d expect from doing live development. Big refactors I do offline, but it is still fairly habitual to make most on the live site.
Last week, while making some changes, I broke the CGI, which pretty much drives everything. After looking at the log, I found out that it happened while Google was crawling the site, which means I’ve lost much of my Google ranking, including my favourite she bangs richard.
So what do you know, when Google finally gets around to crawling the site again, this time I have a bug in my dynamic robot rules, and it only indexes the home page and the blog index. So much for the actual blog items, noindex,follow all the way. So I now have the best of both worlds, several hundred Google page views in my log, and nothing to show for it.
Recently, one of our customers at work thought they had a similar problem, in that Google wasn’t crawling their site correctly, which could have meant a massive decrease in traffic, and possibly an end to their funding if true (it wasn’t). To live and die by Google. This isn’t such an uncommon situation to be in.
Ultimately, I don’t really care if I’m missing 40% of my old traffic. While I love new visitors, that’s not really why I have a personal web site. But it is scary to think that Google has so much control over so much of the industry’s, or more significantly the Internet VC’s cash.
Throw in the current concerns about the privacy of Google’s GMail, in that they preserve and index every single email you’ll ever send through them, and their power suddenly comes into perspective. If Microsoft was in this position, the industry wouldn’t stand for it. Google’s only saving grace, is that it hasn’t yet done anything wrong. We’re all infallible, especially big companies, and especially those who must answer to investors and shareholders. This may not be Google, but for how long?
Don’t get me wrong, I love Google, but I’m starting to be wary of the idea of them having so much control over my virtual presence.
Now if you’ll excuse me, I’ll get back to fixing my damn code so I can get my high rankings back.