February 2006 Newsletter

Site Maps

**** The 4specs Perspective

Are all your pages included in Google?

Unfortunately not all website pages are included in the major search engines. This is primarily due to home pages that use images and javascript for links. In addition, some content management systems create search engine problems. Both problems can be resolved by adding a sitemap - as described in this article.

If all of your important web pages are not included, it could cost you 3,000 users per month from just Google. This was the subject of a newsletter about two years ago - Get all the users you can for FREE.

This month's newsletter covers the use of site maps, including a new program from Google. The cost to implement step 1 should be under 2 hours and is well worthwhile.

1. Site Maps - put a text link on the bottom of your home page to a page with up to 100 pages you want to ensure are in the search engine's database. Call the link "Sitemap" in text - not an image. It is that simple. One 4specs' advertiser had an increase in 3,000 users per month (yes - every month!) from Google simply by adding their sitemap. This first step will work for Yahoo, Search.MSN and other general search engines as well as Google.

2. Google Site Map - Google has a new Sitemap program that Google will use to index your entire website - even sites over 50,000 pages. So far Yahoo and Search.MSN are not using the Google sitemap.

The easiest way to start developing a Google sitemap is to use the [site no longer works] sitemap generator. Be sure to scrutinize your sitemap carefully before submitting. Be certain that every page is included and pages you don't want included aren't included. Place the resulting XML page on your server and notify Google the site map is there. Update the sitemap page whenever changes are made to your site.

Most construction product websites are fairly static and not often updated. We frequently change 20-50 pages per day on 4specs by adding new ads and new listings and deleting or changing listings that no longer work. The 4specs sitemap is downloaded every day by Google and ensures the changed pages are updated.

After adding the AuditMyPC sitemap, we changed to the Google sitemap generator, running it on a nightly basis. This is more complicated as it runs in Python (a computer language found on most UNIX servers) and took me 3-4 hours to get the configuration file setup and working as expected. The AuditMyPC map took less time to initially get started, but requires a manual operation to run and upload the sitemap after each website update.

3. Google Sitemap Benefits - Several days after submitting the Google sitemap (either method), you can visit the Google page and identify the following:

4. House Cleaning - The next step after the Google sitemap is up and running is to clean up problems in your site reported by Google. This can be time consuming. I believe that your search results will be better if the search engine records few problems from your site - i.e. your website cleanliness increases the credibility of your site from the search engine's perspective.

Here are the key tasks you can expect to do in the house cleaning:

Let me know if there are any questions or suggestions. Let me know too when you are successful with this program.

--------------------------------------

Colin Gilboy
Publisher - 4specs
Contact us

9/2008 - Also consider using https://www.xml-sitemaps.com/ to develop sitemaps.

2/2010 - You can also add the site map to your robots.txt file. The first line in our robots.txt file shows our gzipped site map: http://www.4specs.com/robots.txt