Pages Not Indexed By Google Because Of Autofresh

Pages Not Indexed by Google Because of Autofresh

About a year ago, I decided to add an autorefresh to pages that contain feeds so that content would always stay fresh. I set it to autofresh every 3 minutes. I could have been done this via ajax, but my ajax kung fu wasn’t good enough then (or now). I went with the easiest method, which is a simple meta tag addition to those pages. I got the idea from Drudge Report.

It was fine for about a year until the week of the Panda Update 2.2, when all pages with the autorefresh would give the Googlebot errors and not get indexed.

I only discovered it by accident while Googling info about Panda. It’s not related, but because I was losing traffic, I was wondering if pages were being indexed. You can find out by searching Google for “site:www.yoursite.com,” click on “show search tools” at the bottom of the left sidebar, and then “Past hour.” That will show the number pages from your site that’s indexed in the past hour. For my site, it was just my blog and the main channels. None of the featured headlines and teasers from the main sections were indexed.

That affected traffic, of course, because less pages on Google means less search referrals.

I went into Webmaster Tools and started looking for crawl errors. There were lots of “not followed” errors. I learned that one of the causes for those errors are autorefresh meta tags.

As soon as I commented it out, new pages started appearing on Google’s index within minutes.

I got a slight traffic boost from this, but is still down overall because of Panda.

REFERENCE:

URLs not followed errors > Redirect error
http://www.google.com/support/webmasters/bin/answer.py?answer=35156

Sections:  note from the editor   
Topics:  autorefresh   coding   google   google index   html   panda update 2.2   search engine   seo   site updates   wopular   
SENH'S LATEST BLOG ENTRIES
  • The Identical Look of Firefox and Chrome Caused Site ShutdownThe Identical Look of Firefox and Chrome Caused Site Shutdown
    In my previous article, I mention how the latest version of Firefox has a nearly identical look to Google’s Chrome web browser. When I’m working on new features for my sites, I use Firefox for testing and Chrome to see what the results look like to the users. I knew one of these days, because both browsers look the same now, I would confuse one for the other and put up untested programming code that would cause havoc on my sites. More
  • Firefox 29: Looks Just Like Chrome NowFirefox 29: Looks Just Like Chrome Now
    I just upgraded to Firefox 29, the latest version. Well, it’s more like it upgraded itself. Although I like the newer cleaner look, it’s harder and harder to tell the difference between it and Chrome, except the latter is still seemingly snappier. Firefox 29 has buttons at the top left for both bookmarking and access to the bookmark list. More
  • Link Removal Part II: I'm Still Not Removing Your Link, But ...Link Removal Part II: I'm Still Not Removing Your Link, But ...
    About a year ago, I wrote about getting daily requests/demands/threats to remove links to random sites on Wopular. I didn’t add those external links myself. They’re all done automatically through RSS feeds. If another site’s content is related to a headline on Wopular, then a link to it is generated. More
  • Moved SimplePie File Caching to DatabaseMoved SimplePie File Caching to Database
    A while back, My ISP told me my backups took half a day to complete, which is too long. If anything goes wrong during that period, the process would have to start all over again, which happened quite often. The customer rep told me that my server had too many files, and even though they didn’t take up much disk space when added up, the process of copying each one of them takes a lot of time. More
  • Pixlr’s Online Image Editor, When All You Want To Do Is Resize An Image
    I was looking for a Chrome app that would allow me to resize images and came upon Pixlr’s online image editor. Open up their website on any browser (I’ve only used it on Firefox and Chrome), and you get a fast simpler version of Photoshop online. More
SENH'S RELATED BLOG ENTRIES
  • Google's Panda Update Took a Big Bite Out of My Traffic
    Google have been tweating their search algorithms this year to battle content farms and scrapers. They named this tweak Panda Update. In February of this year, they released the first version, which was aimed at lowering the rankings of content farms - i.e. More
  • Don't Put Google Analytics at the Top
    A new tracking code from Google Analytics was released recently. GA is a free web traffic reporting tool that let's you track how many users are viewing your website. The new code snippet uses an asynchronous process, meaning your site would load up without having to wait for the tracking code to finish executing. More
  • Link Removal Part II: I'm Still Not Removing Your Link, But ...Link Removal Part II: I'm Still Not Removing Your Link, But ...
    About a year ago, I wrote about getting daily requests/demands/threats to remove links to random sites on Wopular. I didn’t add those external links myself. They’re all done automatically through RSS feeds. If another site’s content is related to a headline on Wopular, then a link to it is generated. More
  • Google's New Search Algorithm Change Took Out 40% of My Google Traffic
    Last Thursday, Google launched a new tweak to their search algorithm. It was targeted at content farms, which many consider to be spammy sites whose content are created solely to attract search engine traffic for specific keywords. I do notice more and more of these sites on Google’s search results. More
  • Want Your Articles on Google Instantaneously? Digg it.
    I'm just starting to submit stories from my blog to Digg, hoping against hope that they'll get some Digg love and drive some traffic back here. So far, no luck. The only Digg love my blog entries get is from me - and ONLY me. More

 

Comment On This Story