SEO Monitoring and Tracking

Monitoring Your Site Traffic

This is a must-do activity. If you are not viewing and analyzing your site traffic and visitor statistics over time, you are essentially flying blind. This would be akin to a retail store not tracking how many customers come into their store, what they buy, and on which days.

You should first take advantage of the free statistics or “stats” program that most Web hosting companies offer in their packages. These work by reading the contents of the server log file for your site. Such programs are also called log file analyzers. More often than not, such programs don’t provide the information you need or they present it in poorly-organized, hard-to decipher reports. As such, I highly recommend you use a third-party program or service to obtain the information you need to track your site. You can often customize the of reports you want to view and download them into Excel.

You have two choices – use a different log analyzer program, which runs either on the server or on your desktop computer, or sign up for a monthly service that monitors real-time traffic for you. There are pros and cons to each as follows:

Using a Log-file Analyzer


Log-file analyzers can be installed on the server or on your desktop computer. Unless your log files are really large, I recommend the latter. However, getting your Web host to install and configure a different log-file analyzer than the default one they offer can be a frustrating experience. Regardless, make sure the referrer option is enabled for your site (it is usually disabled by default), otherwise you’ll be reading IP addresses instead of domain names to figure out where your traffic is coming.

One issue with log-file analyzers is the information isn’t shown in real time – the data is a day old. This usually isn’t a problem. One of the more popular analyzers is Urchin, which Google now owns. Two free log-file analyzers that are worthwhile include Funnel Web Analyzer (http://www.quest.com/funnel_web/analyzer/) and AWStats (http://awstats.sourceforge.net/).


Using a Tracking Service


Real-time tracking, also called browser-based tracking, is sold as a monthly service. Instead of reading a log file, you include JavaScript tracking code on each page of your site. Each time a visitor comes to your site, the JavaScript code sends information to the service provider where it is stored. Information can be accessed in near real-time and usually the quality of information is better (more accurate visitor and page view counts) than with a log-file analyzer. However you are paying a monthly recurring expense and you are charged by how much traffic you receive on your site – this can be very expensive for high-traffic sites.

More popular service providers include WebTrends Live and HitBox, which start at around $30/mo for low-traffic sites. There are also a couple of even lower-cost vendors that I recommend – Webstat.com (http://www.webstat.com) and IndexTools (http://www.indextools.com). Both are excellent choices for the value.

What to Monitor?


At a minimum you should check your traffic stats weekly as they are a goldmine of information. Of particular importance is tracking the following for your web site:

Keywords: This lists the actual keywords people typed into search engines to find your site. Also listed is the percentage of the total traffic each keyword brought in. There will probably be an entry called “other”, “no keyword” or something similar. This represents people that either directly typed your site into their browser or that have bookmarked your site in their Favorites list.

Spend time determining which search terms visitors used to find your site. You may uncover some new keyword combinations that you didn’t think of using. If this is the case, tweak your site or create a new page around this phrase accordingly.

Search Engines: This lists the search engines that visitors used to find your site. Also listed is the percentage of the total traffic each search engine brought to your site. Usually Google is at the top of the list.

Referrals: This lists the websites that brought traffic to your site and what percentage of the total traffic each “referral” site brought in. Over time, you should start seeing referral traffic from websites you've exchanged links with. There will probably be an entry called “direct”, “no referral” or something similar. This represents people that either directly typed your site into their browser or that have bookmarked your site.

Tip: Create a favicon.ico file. A favicon is a small 16 x 16 pixel icon that is displayed when you bookmark a page and add it your Favorites. Place this file in the root directory of your server and you can track how many referrals are coming from people that have bookmarked your site. Favicons are created with special software that creates the correct file format. They are also great for branding purposes. For more information, see http://www.favicon.com/.

Page Views: This represents the most viewed (or popular) pages on your site. This is useful for determining where visitors are spending their time on your site.

Click Path or Visitor Path: This shows the actual path that a visitor took while browsing through your site. This is great for determining what visitors are looking for.

Exit Pages: These represent the last pages that a visitor views before leaving your site. Hopefully it is your sales page and not your home page!

Length of Session: This shows how long visitors spend on each page and on your site in general. Are people leaving your site too fast? Try to find out why.


Monitoring Your Ranking

Although the focus of this book is on how to get top rankings on Google, what you are really after is lots of traffic to your site that you can turn into sales. Google is one way (albeit an important way) of getting traffic. Don’t get too hung up on your rankings - if you are in the top 10, you will do fine. Some people obsess over getting a #1 ranking to the exclusion of all else, when what is really important in the end are conversions and sales. Keep this in mind.

There are two ways to check your ranking on Google for a particular keyword phrase – check it manually by simply counting your position in a search results pages (this works OK if you are in the top 20 or so), or by using software.

The premiere software program for checking ranking is WebPosition. This is a powerful, full-featured tool designed for the professional. It contains several modules, but only one is really recommended for use – the Reporter module.

Some of WebPosition’s features have gotten people in trouble with search engines in the past. Before you use this tool, make sure your read the online manual and understand how it works. For more information, go to http://www.webposition.com.

WebPosition (and other programs like it) should be used during off-peak hours and only when really needed. Google, along with other search engines, have a dislike for the chronic use of such tools as they impact the performance of their servers. Google has been known to block access to their site from computers that carelessly and frequently run such tools.

Google changes their ranking algorithm all the time, and ranking changes you may see on your site are more likely due to algorithm changes and not because of small changes you may have recently made to your pages. With that said, you should wait a few months after initial optimization before changing anything.

Even though your server logs may indicate that GoogleBot visited your site recently, it takes time before Google indexes the information and synchronizes it across all its datacenters, and can months after that for a stable ranking of your pages to stop bouncing around.

If your site is kicked out of the index for a spam penalty it will usually come back after 60 days if the factor(s) that triggered the spam penalty have been removed. To be proactive, send an email to help@google.com after you have cleaned up your website, explaining in detail what you did to fix the issue and promising not to do it again. If you are still having problems after emailing them, give them a call at 650-330-0100.

However, before assuming that Google has penalized your site, make sure your Web host hasn't implemented a process to block search engine spiders from visiting their hosted sites in order to save bandwidth. This has been documented to happen with some of the lower-end hosting companies.

Important: If you have a new site, or an existing site that has been redesigned to the extent that page filenames have changed, your site will likely be placed in the “Google Sandbox”. In this case, it can take 12-18 months to get a decent ranking for your most important keywords, especially for competitive terms.


Monitoring Your PageRank


You can see an approximation of the actual PageRank that Google uses by downloading and installing the Google Toolbar at http://toolbar.google.com.

Some people have turned the monitoring of PageRank (PR) into an obsession. Don’t waste your time being one of them. PR is but a single factor that influences ranking. PR displayed in the Toolbar can also be inaccurate, but it is better than nothing.

The Toolbar PageRank (PR) scale goes from 0 to 10. Bear in mind that there are vast gulfs between ranges at the upper end, due to the logarithmic nature of actual PageRank. Also bear in mind that sometimes the PR value shown for a new page may not be real and is only a guess.


Checking Number of Pages Indexed


If you have a new site or if you have added new pages to your site, check to see if those pages have been added to the Google index. The easiest way to check is to go to http://www.google.com and in the Google Search box, type:
site:www.YourDomain www.YourDomain replacing YourDomain with your domain (such as www.xyz.com).


Checking Number of Incoming Links


Managing an active linking campaign involves seeing who links to you and to your competitors. The most accurate way to see the total number of incoming links to your site is by using Yahoo. Go to http://www.yahoo.com and type the following in the Search the Web box:
linkdomain:www.YourDomain -site:www.YourDomain
replacing YourDomain with your domain (such as www.xyz.com).

Using the link command in Yahoo gives the most accurate number of incoming links today, but it is not 100% accurate.

Don’t use the Google Toolbar to count incoming links for a page, this method is totally inaccurate. This feature is on the Toolbar by clicking Page Info, then by clicking Backward Links.

For an comparison of how many links each major search engine has on record to your site, use MarketLeap’s Link Popularity Check tool at
http://www.marketleap.com/publinkpop/default.htm. MarketLeap also has a great tool for checking the number of pages indexed on the major search engines.


Measuring Sales Conversion and ROI


At the end of the day, what matters are your sales and your bottom line. After all your hard work, are you converting your visitors to customers? Do you know what percentage of visitors turn into customers? Do you know what your return on investment (ROI) is when you have added up your web site development costs, web hosting costs, consultants, books and all other expenses related to driving traffic to your web site? Do you know what that cost per customer is? Few people do.

The subject of sales conversion and ROI (and how to measure and increase them) is complex and is really beyond the scope of this book. Nevertheless, this is an important topic that should be introduced for you to think about. For more information on calculating sales conversion and ROI, as well as improving copywriting, improving website usability, and in general creating a high-performing website, see my other book Desperate Websites at http://www.desperatewebsites.com.

Quite a number of business owners don’t make their money back on their web sites. Commonly this is because they got carried away with the look for their site (We just have to have Flash and all those gorgeous graphics!), bells and whistles on their site (We have to have that interactive, self-updating, daily survey!), or what the site should say (We just have to use those paradigm-speak, marketing buzzwords – that is what our company is all about!). As a result, you have a case of “Corporate Egos Gone Wild”. Well guess what? The customer does not care about any of this. The customer wants to find a solution to their problem, they want to find it fast, at a value, and they don’t want to be patronized.

You need a method to track visitors from beginning to end and “close the loop”. This means tracking a single visitor from which keyword they typed into Google to find your listing, to which page they landed on your site, to the “sales” page where they took an action. The “sales” page can be an actual product purchase page, form or request for information page, or any other page that represents the next desired action you want visitors to take on your site before.

The easiest way to track visitors in this way is to use a service like Conversion Ruler at http://www.conversionruler.com. For a monthly fee, they will set this up for you.
Alternatively, you can also place custom JavaScript code on each page of your site that obtains the referral URL of the page that a visitor came from before landing on your site, and then storing a cookie that tracks the visitor through the site. The code then emails this information to you when a “sale” takes place (product confirmation email, form submission, etc).