Wopular, Movieswithbutter.com | featured news

Relieving High Server Load by Blocking Search Bots

Coding

Over the years, whenever the site was slow to load up, I always went through the same routine. I checked traffic logs to see if it was due to a sudden increase in traffic. If not, I would look at the mysql slow query log to see if there were slow queries bogging up the database. If that wasn’t the culprit, then I would take a look at the access logs to see if there were any irregular activities from search engine bots.

 

Another Network Outage ... Argh ...

I spent the whole day dealing with servers. The facility where Wopular and MoviesWithButter are hosted were having some network issues. I was able to get MWB up and running within an hour or so. Wopular was a whole different story. The Database is about 21GB. It's a huge db, so whenever there's a network issue, I can always count on at least one corrupt table. And it always happens to be the largest table, which takes several hours to diagnose and repair.

 

Subscribe to this RSS topic: Syndicate content