When you wish to keep a record of the position of your website within search results for targeted search terms there are a couple of steps you can take to help with your manual checks.

First, if you haven’t already, download and install the latest version of the FireFox browser. Open a new browser window and within 3 separate tabs go to Google, Yahoo! and Live Search (or MSN). Now in each search engine do searches for your first search term (for example: Search Engine Optimisation). Then in each of the search engines you will notice the have either a ‘Preferences’ or ‘Options’ link, click on this and you will be taken to a page where you can make changes which alter the way the search results are displayed. In Google and Yahoo change the number of results per page to 100. In Live Search change the total results per page to 50 (as this is the limit) but also uncheck the ‘Group results from the same site’ as this will show truer positions for individual webpages. Save the settings and you will then be taken back to the results for your search term.

You will already notice that Yahoo numbers its results so that it is easy to quickly find your position within the listings for your search term. To do the same for Google and Live Search we first have to install the FireFox add-on GreaseMonkey. This add-on allows you to customize the way a webpage displays using small bits of JavaScript. We now need to add extra add-ons that work with GreasMonkey and these are Number Google Results and Number MSN Results from ilovejackdaniels.com. Click on the download links and choose to install. One more step you will need to take once all are installed is in the FireFox toolbar, go to Tools -> Add-ons and then click ‘Options’ in the GreaseMonkey add-on. Choose ‘Number MSN Results’ in the left tabs and in the ‘Included Pages’ section add http://search.live.* to the list.

So now refresh your search listings for your search term in each tab and you will have numbered results 1-100 in Google and Yahoo, and 1-50 in Live Search/MSN. To quickly find your position for a search term use the FireFox find tool (Ctrl+F then Ctrl+G to repeat find) and look for your domain URL in the results (eg. positiongoldltd.com). You can now set up an excel spreadsheet with say 10 search terms and quickly monitor and record your position in each of the search engines listings.

This post will detail the actions you should take when trying to include your new website within the search engine results of Google.co.uk, Yahoo.co.uk and Live.co.uk.

1) Submit your New Website through “Add URL” Interface

The three main search engines listed above each have their own “Add URL” page which will allow you to suggest your website URL so that next time their bots crawl the web searching for new web pages to index then your website will be on the list:

• Add URL to Google
• Add URL to Yahoo!
• Add URL to MSN (Live Search)

2) Create a Sitemap.xml

Search engines now universally recognise XML sitemaps as a standard way to supplement the data found through crawlers finding your website. To allow crawlers that support sitemaps to pick up all URLs in the sitemap you should firstly add a sitemap link within your robots.txt, for example:

Sitemap: http://www.positiongoldltd.com/sitemap.xml

You can then also submit your XML sitemap through Google’s Webmaster Tools and Yahoo!’s Site Explorer.

3) Obtain Inbound Links to your New Website

The quickest and most beneficial way to have your new website indexed with the search engines is to gain as many inbound links to your website as possible. This will help your site be found by search engine bots especially if they are finding your website through a link from a popular website that is regularly crawled and re-indexed.

Some Final Suggestions

The main thing you should remember about search engines is that they need to find your new web pages through links, so by creating and structured navigation and including sitemaps on your website you are giving the search engine bots every chance to dig deeper into your website. Search engine bots do have problems with reading some forms of navigation such as JavaScript drop downs, Flash and HTML form menus, so it is always best to keep it simple and provide alternatives if the previous items must be used. Gaining inbound links to your website will help the search engine bots index your web pages quicker and more regularly which will help keep your results in the search engine listings fresh and relevant.