I’ve done my best to make my site ‘indexable’ by Search Engines but I’ve never really tried to optimise it. Martin Peck got me thinking about it more when I shared my analysis of my IIS logs last year. Back then my referrer list looked something like this:
As Martin rightly pointed out, Google is by far the most popular search engine and I really should see ~80% of my traffic from Google, not Live Search. My first job was to added Description and Keywords to every page. Initially it was all manually entered and I wasn’t really sure how helpful it would be.
This month all my guessing disappeared when Microsoft released a Beta of the IIS Search Engine Optimization Toolkit. It sounded promising so I gave it a whirl. It’s pretty straightforward to use, you click on the icon in IIS Manager and start an analysis of your site (or anyone else’s for that matter). The results of my first check were interesting!
A couple of things immediately jumped to mind; Wow I have lots of URLs & Links and boy I have some issues to fix! 54,305 to be exact! Clicking on the report showed lots of helpful detail:
Clicking on each of these errors gives tons of info; what the issue is, what it means, what pages are impacted, what the content of that page is, what the headers are, what pages it links to, what pages link to it, etc etc. You get the idea, tons of data to help you resolve the problems.
A couple of hours later (over the last two weeks) I have been able to make significant improvements. Here’s the latest analysis for my site:
Three things to note:
- My URL count has dropped by 30%. Why? All my URLs are lower case, so they are no longer considered unique. Why is this important? Well now a Search Engine will see less pages. Which means the PageRank I would have got for five different pages will now be higher for the one single page these URLs pointed to.
- My Link count has dropped 40%, again using lower case URLs is the ‘fix’ here.
- My violations are down 90%!
Looking at the data shows that there’s really not much more I can do to get the number lower. That said I’m going to try and get it to zero (ignoring the Robots info which is perfectly valid):
So has it helped? Since starting this work I’ve noticed a considerable change (in just two weeks) to the traffic hitting my site. First off, my Referrer list is now in the correct order (Google, followed by Live). And secondly the number of referrals in one month is almost equal to all the data I analysed last year!
The thing I found the most interesting, is what did people search for. Well in turns out it’s really all about my blog and my photos aren’t really generating traffic to the site! Obviously the saying “a picture speaks a thousand words” isn’t true on the web, not until image recognition improves.
The other, less obvious, benefit of this effort has been the sharing feature on Facebook. When they parse a site for content they look for a number of different headers. If found they use these instead of the ‘guess’ they normally make. The image on the left is a guess, the image on the right uses the headers I’ve been adding. Clearly there’s no comparison! The image is one I’ve chosen and the description is real text, not links that don’t say anything useful.
 |  |
| (Before) | (After) |