Make sure your web server supports if modified since

The name is a bit daunting but what it does is really quite simple. The If-Modified-Since HTTP header tells a search engine spider one of two things about a webpage...

  • This webpage has not changed, no need to download again.
  • This webpage has changed so download again because there is new information.

To find out if your webpages support If Modified, use the if modified since tool.

One way to describe If Modified Since is to think of that little flag on your mailbox outside your house. If you put the flag up, then the postman (or woman) knows you have mail in there and will come and get it. If the flag on your mailbox is down then the postperson knows they don't have to pick up any mail.
If Googlebot was the mailperson, and your webpage was the mailbox, Googlebot looks to see if that "flag" is up or down before it accesses your page.

Why is that important?

Since Google spiders billions of pages, there is no real need to use their resources or yours to look at a webpage that has not changed.
For those people who have very large websites the crawling process of search engine spiders can consume lots of bandwidth and result in extra cost.

How does it work?

Let's say you have a website about pets that has pages about dogs, cats, and turtles.
You have just updated the turtle page with new photos.
Every once in a while Googlebot (the search engine crawler of Google) will visit your pages. It will check each page in your website and if it none of the pages have changed, it will not load any of those pages.
But in our case, the turtle page has changed (and has a "flag" up"). So when Googlebot comes it will see the "flag" for the turtle page and it will access the turtle page.

It knows that the turtle page has been updated because of the If Modified Since header code. Since none of the other web pages have been updated, there is no reason to get a new copy of them since it already has a current copy in the index.

304 status code

Status codes are listed in your log files, and you will typically see then in your statistics report.

If a search engine crawler sees a web page status code of 304 it knows that web page has not been updated and does not need to be accessed again.

How do I know if my web server supports the If-Modified-Since header?

There are a few ways of doing this, probably the easiest is to use our online If Modified Since Tool. If you have access to your statistics and/ or log files you can just look for the status code of 304, if you see 304 as a status code in any of your logs then your web server supports this header. If you can't do that then you should call your web host and ask them. When deciding on a web host this is one of the questions you should ask. "Does your web server support the "If-Modified-Since HTTP header?" Virtually every modern hosting company supports it, but it is still worth checking.

Key concepts

  • Using a web server that supports the If Modified Since header is recommended, and will result in less bandwidth being used by search engine crawlers.
  • The overwhelming majority of web hosts use this, you likely need not worry about this guideline unless you have a massive website with thousands or tens of thousand of pages.

How to determine if you are following this webmaster guideline

To see if a web page supports the if modified since http header you can use this tool: If Modified Since Tool

by Patrick Sexton