feedthebot.com How to follow the Google webmaster guidelines

 

 

Google webmaster guidelines > Design and Content guidelines > Guideline eight of nine in this category states...

"If you decide to use dynamic pages (i.e., the URL contains a "?" character), be aware that not every search engine spider crawls dynamic pages as well as static pages. It helps to keep the parameters short and the number of them few."

 

 

Definitions-

 

dynamic pages - Web pages automatically generated by ASP, PHP or ColdFusion or other technology. Databases and many "online stores" are dynamic. Most dynamic pages have a "?" in the URL.

URL - The URL of Google, as an example, is http://www.google.com URL stands for Uniform Resource Locator. A website URL is commonly referred to as its "address" on the web.

search engine spider - Also known as a "crawler" or a "bot", a search engine spider follows links to web pages and then reads and retains the information it finds. This information eventually becomes the "copy" of a website in a search engine index. This process is often referred to as "crawling" the web.

static pages - A static page only displays the information written on the page, it is a pre-formatted file that only displays the information in the file as opposed to a dynamic page which might display information pulled from a database. Most beginner sites are static. Your web pages are not static if they use a database to display information.

parameters - Sometimes a URL looks like this -

 www.example.com/something/lookat.php?OrderBy=avail&PropType=&PHPSESSID=

In the above example, "OrderBy=avail" is a parameter, "&PropType=" and "&PHPSESSID=" are also parameters. They are part of the URL that assigns an input or value depending on what a visitor is looking for.

 

Examples and Explanations

 

To understand this guideline requires that you know what language your website is written in. If  your website only has static pages this guideline does not apply to you at all and you need not worry about it.

Sometimes websites have both static and dynamic pages, this guideline would only apply to the dynamic pages of such a website.

 

If your website does use dynamic pages, then this guideline is telling you that your website may not be indexed as well as it could be if the pages were static.

As search engines become more sophisticated, they are finding more ways to crawl and index dynamic pages. Google has changed their guidelines to reflect this.

There used to be a Google webmaster guideline stating...

"Don't use "&id=" as a parameter in your URLs, as we don't include these pages in our index."

But that guideline is now gone and they have stated that they now index such pages.

They also have stated two important things concerning this issue (I am quoting the "Updating the webmaster guidelines" article posted by Vanessa Fox from the Official Google blog - there is a link to this article in "resources" below)

 

1) "dynamic URLs with a large number of parameters may be problematic for search engine crawlers in general, so rewriting dynamic URLs into user-friendly versions is always a good practice when that option is available to you"

2) "If you can, keeping the number of URL parameters to one or two may make it more likely that search engines will crawl your dynamic urls."

 

If you have dynamic pages and are worried about this guideline.

-  Dynamic websites do get crawled and indexed but they take much longer to be fully indexed than static pages.

-  It is a reality that dynamic pages are a big part of the internet future. Google knows this (Google has dynamic pages themselves).

-  There are ways to create more "Google friendly" pages even if you use dynamic pages.

-  Google sitemaps will help Google find your pages. Submit a Google Sitemap as part of Google Webmaster Tools

 

Making your dynamic pages search engine friendly

-  In general, you can create static pages that are copies of your dynamic pages but you must "tell" the search engine spiders that it is a copy through a robot.txt file that they are copies. Only one version of any page should be available to Googlebot or other spiders. The goal of this website is to define the Google webmaster guidelines, not to make recommendations. The best source of information about Google is Google itself. They give specific advice for what to do and how to do it in their help pages (see resources below).

-  There are a multitude of solutions and articles on the web that can be found by searching for "dynamic vs static webpages".

-  Google states in a FAQ - "Typically, URLs with 1-2 parameters are more easily crawlable than those with many parameters. Also, you can help us find your dynamic URLs by submitting them to Google Sitemaps".

 

Key concepts -

If your website is entirely made of static pages, this guideline does not apply to you.

If you use dynamic pages then there is a potential problem with crawling and indexing your web pages completely.

If you use dynamic pages it is better to have less parameters in your URLs

 

The best way to know your options is to read what Google has stated on this subject.

Dynamic pages are discussed often in Google help pages (which is the best source of information on anything Google) and I have links to a few Google resources below to get you started.

 

Resources

 

From Google:

Google help page that offers direction for dynamic pages:

Does Google index dynamic pages?

The change to the Google webmaster guidelines I quoted:

Update - dynamic pages

FAQ page I quoted above that offers suggestions and solutions for this and other scenarios where you are concerned that your pages are not being indexed.

Pages not in the index

 

From other sources:

This article highlights some of the benefits and challenges of both static and dynamic pages.

Dynamic pages vs Static pages