March 2005 Newsletter

How to Understand Your Web Site Activity

**** 4specs Recommended Reading

"Usability and search friendliness go hand in hand. If products and services are easy to find, and the Web site easy to use (resulting in higher conversions), the site should easily maintain high-quality search engine traffic."
[link no longer valid] article of Log File Analysis and Search Engine Optimization
[Cannot locate this article anymore]

**** The 4specs Perspective

Tracking where web site traffic comes from is vital to your marketing efforts - and it can be simple, fast and even free.

How to Analyze Your Web Site Stats

As 4specs enters our 10th year in the Internet business (starting as in February 1996) I know that tracking and understanding your web site activities is extremely important. The tracking can be done by your consultants, your IT department or yourself. However it is done, the marketing staff needs to monitor web site activity.

This is important for the following reasons:

  1. Lets you know the activity level of your web site and compare it to past periods
  2. Shows you changes in your web site activity and your search engine results
  3. Shows you the results of your advertising programs
  4. Shows you the results of web site design changes

Each server records every file served. The server records the following for every page delivered and every image displayed:

Server Log Access
The first step is to access your server logs. I recommend you download the files each month to your desktop machine. Most web providers will let you download or otherwise access your logs. Some free and low cost providers will not. If you cannot get access to the server logs, you should move to a new host. Ask me for leads on providers with $10-30 accounts that can work for all but a few companies. 4specs, with 90,000 user sessions a month, could easily be run on a shared server with a $30 account.

Statistical Analysis Programs
You can choose among many statistical analysis programs. WebTrends is perhaps the best known, and expensive, program. I use Analog, a free program, to track the stats for 4specs. Analog can be a bit tricky to set up, so I am recommending you try a $100 program - Mach5.
[Products have changed and we are now recommending Google Analytics - a free program]

Analog, WebTrends and Mach5 all provide similar results, just cost different amounts. Mach5 has a 25 day trial period. Download the server logs for the last month to your desk machine and run Mach5. It is that simple. I was impressed at how easy it was to run. This will get you started on understanding your web site activity.

The Analog reports are highly configurable (which is why it takes some programming to get it to work properly). For example, the referring site could be set for any site that referred more than 5, or 10, or what ever depth you want.

Recording the Data
The next step is to build a spreadsheet to summarize the statistical analysis results. I suggest you track the following information for each month:

Monitor Broken Links and Robot Activity
In addition, you should monitor any broken links and the robot activity to be sure that your site is completely indexed by the search engines. Most important is and msnbot. Yahoo's robot (called Slurp) may be more difficult to track as it uses a Mozilla user agent. You will want to verify that the search engine robots are downloading your site on a routine basis. For example, 4specs has approximately 800 pages. Googlebot downloaded 2,800 pages during the month (not included in this public report), meaning the entire 4specs site was downloaded every 10 days or so. This keeps our information fresh in the search engine index.

Let me know if you have any questions. If you send me a copy of your server logs I will be happy to process them and give you a brief report. Let me know and we can discuss how to get them to me.


Colin Gilboy
Publisher - 4specs
Contact us