At the end of last week we had a catastrophic failure that resulted in our losing most of our posts. We had a security problem where someone had managed to compromise one of our blog accounts with administrative privileges. Some of the files were modified. We noticed it right away and decided to restore the site files and database from our nightly dump.
However … it turned out that when we did a major WordPress update back in February 2006, we created a new database but failed to update our backup script. So, for the past 19 months, it’s been creating a nightly backup of the old database. Restoring the old database not only resulted in loosing 19 months worth of posts, but also left the database out of sync with the current WordPress version.
One of our former students (thanks Filip!) wrote a script to recover the old posts from Google’s cache and reinsert them into the database. it was a tour de force demonstration of quick programming skill. There are still some problems that we’ll need to attend to — we’ve lost all of the new categories that we’ve added since 2/2006, the ‘related posts’ plugin is no longer working, I think the feed links aren’t all right, etc. But we recovered the posts.
We’ve tightened up our security but continue to see lots of malicious visitors knocking on the door and checking the locks.
It’s a jungle out there.