Comment on Moved SimplePie File Caching to Database

Moved SimplePie File Caching to Database

SimplePie

A while back, My ISP told me my backups took half a day to complete, which is too long. If anything goes wrong during that period, the process would have to start all over again, which happened quite often.

The customer rep told me that my server had too many files, and even though they didn’t take up much disk space when added up, the process of copying each one of them takes a lot of time.

I almost immediately figured out that it’s the files from the SimplePie cache. There are so many that I couldn’t even list them. Well, I could, but it would take hours for the list of files to show up. The rep confirmed to me that that was the culprit.

One suggestion the rep had was to put all those files on another server since they’re just cache and don’t really need to be backed up.

I didn’t know how to do that, so I went with another solution. SimplePie also has an option to cache the feeds into a MySQL database. Apparently, this option is kinda new and isn’t used much because it doesn’t work unless you modify one of the files.

I followed the directions here (in the MySQL section), which required creating two new tables in MySQL to store the cached feeds. Next, I commented out a few lines in the MySQL.php file that came with the SimplePie installation, as instructed here. Afterwards, I enabled the MySQL pdo driver in the PHP.ini file, as directed here (mine has a ".so" extension instead of ".dll"). Regarding the latter, remember to use php_info() to find where the correct php.ini file is located.

After doing those three things, caching to the database worked. The only problem -- and it’s a major one -- is that my server load went up ten times. Yup, ten times. I think the problem is with the way the cache tables are implemented. There’s a table for the feed and another for the individual feed items. That’s a lot of reading/writing to the database, which slows down the site by quite a bit.

So instead of that, I decided to disable the cache on SimplePie and just write my own script for it. I created just one table and cache the entire feed in one row. This works pretty well. The load is only slightly higher than using the file cache, and the site is nice and speedy.

Now, the backup only takes about an hour instead of twelve. Big difference.

 

Comment On This Story