Cron Jobs And Feed Updates


Lately, we’ve been adding more local news feeds to the site. Eventually, I want to have most of the major cities across the U.S. covered. What I had noticed since starting this local news project was that the more feeds we've added to the site, the slower the feeds on the homepage updated.

It’s not as fresh as it should be, and I didn’t quite know how the updating of feeds work within Drupal. I knew two things. For the management of feeds, I’m using a module call Aggregation. I can set when each feed updates. I also have a module called Elysia Cron, which is a customizable management of cron jobs, to set up how often it should call up the Aggregation module to update the feeds.

On Elysia Cron, I set Aggregation to do it every 15 minutes, but it never does. It looks like it only updates every hour. Often times, I get a “w” next to the Aggregation module, which means the feeds didn’t get updated as scheduled.

On Wopular, I have several cron jobs that I manage with Elysia Cron. Aggregation is the most time intensive. Elysia allows for groupings of cron jobs, so I separated Aggregation into its own dedicated channel. This way, even if it fails, it won’t affect the other cron jobs.

I also changed the default settings for “Time Limit” and “Stuck Time” to 15 minutes. Before, they were 4 minutes and an hour respectively. I couldn’t quite figure what they do from the description because it was written by a foreigner. I just figured that since “Stuck Time” was equal to the time the feeds actually do update, I lower it to when I want the feeds to update.

So far, these changes seem to be working. Feeds on the homepage are updated more regularly, and I don’t see the dreaded “w” next to the cron jobs. Ah, the things one can accomplish by just tinkering.

Sections:  note from the editor   
Topics:  aggregation   cron job   drupal   elysia cron   rss   site updates   
  • "Storm The Gates" Mobile Game
    My friends just launched a mobile game called "Storm the Gates"! It's available on both Android and iOS. I'm not much of a gamer, so I didn't know what to do when I started it. But my 9-year-old son turned it on and knew exactly what to do. More
  • "Ip Man 3" Has New Release Date in China and Posters to Prove It
    Over at, I’ve written two articles regarding “Ip Man 3,” the film that’s holding its own in Asian territories against “Star Wars: The Force Awakens.” If you’re not familiar with Asian cinema,”Ip Man” one of the most popular franchises in that area of the world. More
  • Live-Blogged The Golden Globes at MWB
    Yesterday, I live-blogged The Golden Globes -- for various reasons. First, it gets traffic. Second, I might as well do something useful while watching TV. Lastly, I was looking forward to seeing Ricky Gervais host the event again. Nothing against Tina Fey and Amy Poehler, but he’s my favorite. A google search revealed that the last time I did this for this award show was More
  • Kings vs. Mavs: Ominous Stats, News for Sacramento Before Game
    The preview on and the injury report from don’t look good for the Sacramento Kings going into tonight’s game against the playoff-bound Dallas Mavericks. More
  • What?! Kings Beat Thunder, 116-104
    (Image from the Sacramento Bee. Click here to view more images from the game.) What a win. It was a wild one, especially in the first half. We [Sacramento Kings] were down by 17 in the first quarter. In frustration, I was about to turn off the TV, but the teaml called a timeout. More
  • Broken Feeds: Is RSS Going the Way of the Dinosaurs?
    I’m been fixing a bunch of broken feeds since the new year. It started with CNN, which launched a redesign at the beginning of the year. At the time, all of their RSS feeds were broken as a result of that (more about this later). More
  • Moved SimplePie File Caching to Database
    A while back, My ISP told me my backups took half a day to complete, which is too long. If anything goes wrong during that period, the process would have to start all over again, which happened quite often. The customer rep told me that my server had too many files, and even though they didn’t take up much disk space when added up, the process of copying each one of them takes a lot of time. More
  • Site Updates: Fixed Feeds for Washington Post, Yahoo Buzz, and Ft. Worth Star-Telegram
    Apparently, the Washington Post had a redesign recently and their feeds were modified. Yahoo Buzz also had some changes lately - seemed like they streamlined the site and eliminated some categories. It is one of those “sunset” sites that they’re either planning to shutdown or sell off. More
  • Finally, Got Rid of That Pesky Count Query
    While updating Wopular, I noticed that from time to time, the site would locked up for a couple minutes. Most of the time, I would just wait it out. I finally looked into the slow query log for mysql and found the troublesome query. More
  • Most Fed Articles
    Look to the right of the homepage and you’ll see a new module called “Most Fed Articles.” They’re AP articles ranked by the number of times they’re being used in the top newspapers in the US within the last six hours. More


Comment On This Story