Moved Simplepie File Caching To Database

SimplePie

A while back, My ISP told me my backups took half a day to complete, which is too long. If anything goes wrong during that period, the process would have to start all over again, which happened quite often.

The customer rep told me that my server had too many files, and even though they didn’t take up much disk space when added up, the process of copying each one of them takes a lot of time.

I almost immediately figured out that it’s the files from the SimplePie cache. There are so many that I couldn’t even list them. Well, I could, but it would take hours for the list of files to show up. The rep confirmed to me that that was the culprit.

One suggestion the rep had was to put all those files on another server since they’re just cache and don’t really need to be backed up.

I didn’t know how to do that, so I went with another solution. SimplePie also has an option to cache the feeds into a MySQL database. Apparently, this option is kinda new and isn’t used much because it doesn’t work unless you modify one of the files.

I followed the directions here (in the MySQL section), which required creating two new tables in MySQL to store the cached feeds. Next, I commented out a few lines in the MySQL.php file that came with the SimplePie installation, as instructed here. Afterwards, I enabled the MySQL pdo driver in the PHP.ini file, as directed here (mine has a ".so" extension instead of ".dll"). Regarding the latter, remember to use php_info() to find where the correct php.ini file is located.

After doing those three things, caching to the database worked. The only problem -- and it’s a major one -- is that my server load went up ten times. Yup, ten times. I think the problem is with the way the cache tables are implemented. There’s a table for the feed and another for the individual feed items. That’s a lot of reading/writing to the database, which slows down the site by quite a bit.

So instead of that, I decided to disable the cache on SimplePie and just write my own script for it. I created just one table and cache the entire feed in one row. This works pretty well. The load is only slightly higher than using the file cache, and the site is nice and speedy.

Now, the backup only takes about an hour instead of twelve. Big difference.

Sections:  note from the editor   
Topics:  database   mysql   rss   simplepie   site updates   web development   wopular   
SENH'S LATEST BLOG ENTRIES
  • "Storm The Gates" Mobile Game
    My friends just launched a mobile game called "Storm the Gates"! It's available on both Android and iOS. I'm not much of a gamer, so I didn't know what to do when I started it. But my 9-year-old son turned it on and knew exactly what to do. More
  • "Ip Man 3" Has New Release Date in China and Posters to Prove It
    Over at MoviesWithButter.com, I’ve written two articles regarding “Ip Man 3,” the film that’s holding its own in Asian territories against “Star Wars: The Force Awakens.” If you’re not familiar with Asian cinema,”Ip Man” one of the most popular franchises in that area of the world. More
  • Live-Blogged The Golden Globes at MWB
    Yesterday, I live-blogged The Golden Globes -- for various reasons. First, it gets traffic. Second, I might as well do something useful while watching TV. Lastly, I was looking forward to seeing Ricky Gervais host the event again. Nothing against Tina Fey and Amy Poehler, but he’s my favorite. A google search revealed that the last time I did this for this award show was More
  • Kings vs. Mavs: Ominous Stats, News for Sacramento Before Game
    The preview on NBA.com and the injury report from SactownRoyalty.com don’t look good for the Sacramento Kings going into tonight’s game against the playoff-bound Dallas Mavericks. More
  • What?! Kings Beat Thunder, 116-104
    (Image from the Sacramento Bee. Click here to view more images from the game.) What a win. It was a wild one, especially in the first half. We [Sacramento Kings] were down by 17 in the first quarter. In frustration, I was about to turn off the TV, but the teaml called a timeout. More
SENH'S RELATED BLOG ENTRIES
  • Downtime & How to Make a Repair Job on MySQL Go Faster
    The site crashed yesterday morning at around 9am, although it’s hardly noticable until a couple hours later. I didn’t notice it until I started picking stories to feature on the homepage. Whenever i tried to feature an article, it would stop the site from loading up. From that, I knew that the MySQL database must be checking for corrupt tables. More
  • Facebook Added Website Comment Notifications to User Page
    I just started noticing this recently. It’s a nice touch. Before, you would have to go into a specific comment moderation page to keep track of comments. Now, it’s all tied into your profile page, just like when someone likes or comments on your status updates. Months ago, I converted my commenting system to Facebook’s. More
  • Another Network Outage ... Argh ...
    I spent the whole day dealing with servers. The facility where Wopular and MoviesWithButter are hosted were having some network issues. I was able to get MWB up and running within an hour or so. Wopular was a whole different story. More
  • Site Updates: Fixed Feeds for Washington Post, Yahoo Buzz, and Ft. Worth Star-Telegram
    Apparently, the Washington Post had a redesign recently and their feeds were modified. Yahoo Buzz also had some changes lately - seemed like they streamlined the site and eliminated some categories. It is one of those “sunset” sites that they’re either planning to shutdown or sell off. More
  • Finally, Got Rid of That Pesky Count Query
    While updating Wopular, I noticed that from time to time, the site would locked up for a couple minutes. Most of the time, I would just wait it out. I finally looked into the slow query log for mysql and found the troublesome query. More

 

Comment On This Story