Pages Not Indexed By Google Because Of Autofresh

Pages Not Indexed by Google Because of Autofresh

About a year ago, I decided to add an autorefresh to pages that contain feeds so that content would always stay fresh. I set it to autofresh every 3 minutes. I could have been done this via ajax, but my ajax kung fu wasn’t good enough then (or now). I went with the easiest method, which is a simple meta tag addition to those pages. I got the idea from Drudge Report.

It was fine for about a year until the week of the Panda Update 2.2, when all pages with the autorefresh would give the Googlebot errors and not get indexed.

I only discovered it by accident while Googling info about Panda. It’s not related, but because I was losing traffic, I was wondering if pages were being indexed. You can find out by searching Google for “,” click on “show search tools” at the bottom of the left sidebar, and then “Past hour.” That will show the number pages from your site that’s indexed in the past hour. For my site, it was just my blog and the main channels. None of the featured headlines and teasers from the main sections were indexed.

That affected traffic, of course, because less pages on Google means less search referrals.

I went into Webmaster Tools and started looking for crawl errors. There were lots of “not followed” errors. I learned that one of the causes for those errors are autorefresh meta tags.

As soon as I commented it out, new pages started appearing on Google’s index within minutes.

I got a slight traffic boost from this, but is still down overall because of Panda.


URLs not followed errors > Redirect error

Sections:  note from the editor   
Topics:  autorefresh   coding   google   google index   html   panda update 2.2   search engine   seo   site updates   wopular   
  • "Storm The Gates" Mobile Game
    My friends just launched a mobile game called "Storm the Gates"! It's available on both Android and iOS. I'm not much of a gamer, so I didn't know what to do when I started it. But my 9-year-old son turned it on and knew exactly what to do. More
  • "Ip Man 3" Has New Release Date in China and Posters to Prove It
    Over at, I’ve written two articles regarding “Ip Man 3,” the film that’s holding its own in Asian territories against “Star Wars: The Force Awakens.” If you’re not familiar with Asian cinema,”Ip Man” one of the most popular franchises in that area of the world. More
  • Live-Blogged The Golden Globes at MWB
    Yesterday, I live-blogged The Golden Globes -- for various reasons. First, it gets traffic. Second, I might as well do something useful while watching TV. Lastly, I was looking forward to seeing Ricky Gervais host the event again. Nothing against Tina Fey and Amy Poehler, but he’s my favorite. A google search revealed that the last time I did this for this award show was More
  • Kings vs. Mavs: Ominous Stats, News for Sacramento Before Game
    The preview on and the injury report from don’t look good for the Sacramento Kings going into tonight’s game against the playoff-bound Dallas Mavericks. More
  • What?! Kings Beat Thunder, 116-104
    (Image from the Sacramento Bee. Click here to view more images from the game.) What a win. It was a wild one, especially in the first half. We [Sacramento Kings] were down by 17 in the first quarter. In frustration, I was about to turn off the TV, but the teaml called a timeout. More
  • Google's Panda Update Took a Big Bite Out of My Traffic
    Google have been tweating their search algorithms this year to battle content farms and scrapers. They named this tweak Panda Update. In February of this year, they released the first version, which was aimed at lowering the rankings of content farms - i.e. More
  • Don't Put Google Analytics at the Top
    A new tracking code from Google Analytics was released recently. GA is a free web traffic reporting tool that let's you track how many users are viewing your website. The new code snippet uses an asynchronous process, meaning your site would load up without having to wait for the tracking code to finish executing. More
  • Link Removal Part II: I'm Still Not Removing Your Link, But ...
    About a year ago, I wrote about getting daily requests/demands/threats to remove links to random sites on Wopular. I didn’t add those external links myself. They’re all done automatically through RSS feeds. If another site’s content is related to a headline on Wopular, then a link to it is generated. More
  • Google's New Search Algorithm Change Took Out 40% of My Google Traffic
    Last Thursday, Google launched a new tweak to their search algorithm. It was targeted at content farms, which many consider to be spammy sites whose content are created solely to attract search engine traffic for specific keywords. I do notice more and more of these sites on Google’s search results. More
  • Want Your Articles on Google Instantaneously? Digg it.
    I'm just starting to submit stories from my blog to Digg, hoping against hope that they'll get some Digg love and drive some traffic back here. So far, no luck. The only Digg love my blog entries get is from me - and ONLY me. More


Comment On This Story