Pages Not Indexed By Google Because Of Autofresh

Pages Not Indexed by Google Because of Autofresh

About a year ago, I decided to add an autorefresh to pages that contain feeds so that content would always stay fresh. I set it to autofresh every 3 minutes. I could have been done this via ajax, but my ajax kung fu wasn’t good enough then (or now). I went with the easiest method, which is a simple meta tag addition to those pages. I got the idea from Drudge Report.

It was fine for about a year until the week of the Panda Update 2.2, when all pages with the autorefresh would give the Googlebot errors and not get indexed.

I only discovered it by accident while Googling info about Panda. It’s not related, but because I was losing traffic, I was wondering if pages were being indexed. You can find out by searching Google for “,” click on “show search tools” at the bottom of the left sidebar, and then “Past hour.” That will show the number pages from your site that’s indexed in the past hour. For my site, it was just my blog and the main channels. None of the featured headlines and teasers from the main sections were indexed.

That affected traffic, of course, because less pages on Google means less search referrals.

I went into Webmaster Tools and started looking for crawl errors. There were lots of “not followed” errors. I learned that one of the causes for those errors are autorefresh meta tags.

As soon as I commented it out, new pages started appearing on Google’s index within minutes.

I got a slight traffic boost from this, but is still down overall because of Panda.


URLs not followed errors > Redirect error

Sections:  note from the editor   
Topics:  autorefresh   coding   google   google index   html   panda update 2.2   search engine   seo   site updates   wopular   
  • Google's Panda Update Took a Big Bite Out of My Traffic
    Google have been tweating their search algorithms this year to battle content farms and scrapers. They named this tweak Panda Update. In February of this year, they released the first version, which was aimed at lowering the rankings of content farms - i.e. More
  • Don't Put Google Analytics at the Top
    A new tracking code from Google Analytics was released recently. GA is a free web traffic reporting tool that let's you track how many users are viewing your website. The new code snippet uses an asynchronous process, meaning your site would load up without having to wait for the tracking code to finish executing. More
  • Link Removal Part II: I'm Still Not Removing Your Link, But ...Link Removal Part II: I'm Still Not Removing Your Link, But ...
    About a year ago, I wrote about getting daily requests/demands/threats to remove links to random sites on Wopular. I didn’t add those external links myself. They’re all done automatically through RSS feeds. If another site’s content is related to a headline on Wopular, then a link to it is generated. More
  • Google's New Search Algorithm Change Took Out 40% of My Google Traffic
    Last Thursday, Google launched a new tweak to their search algorithm. It was targeted at content farms, which many consider to be spammy sites whose content are created solely to attract search engine traffic for specific keywords. I do notice more and more of these sites on Google’s search results. More
  • Want Your Articles on Google Instantaneously? Digg it.
    I'm just starting to submit stories from my blog to Digg, hoping against hope that they'll get some Digg love and drive some traffic back here. So far, no luck. The only Digg love my blog entries get is from me - and ONLY me. More


Comment On This Story