There’s been a lot of talk about the downfall of journalspace.com and their incredibly shortsited (sic) backup policy.
“It was the guy handling the IT (and, yes, the same guy who I caught stealing from the company, and who did a slash-and-burn on some servers on his way out) who made the choice to rely on RAID as the only backup mechanism for the SQL server. He had set up automated backups for the HTTP server which contains the PHP code, but, inscrutibly, had no backup system in place for the SQL data.”
Journal Space Blog
This isn’t a data backup strategy – it’s a strategy for failing over to another machine when things (temporarily) go awry. The very nature of your setup is that anything (bad) that happens will immediately be applied to both sets of your data. I would literally be afraid to work in an environment where an innocent accident (bug) could completely wipe out *all* of the data you have ever worked with! Add on top of that that this data is not “theirs” – they are (were) in the business of hosting other people’s data… just completely unprofessional.
There are undeniably serious IT lessons to be learned her. Not only do you need to have multiple copies of your data (which technically they did). The copies need to be stored in disconnected systems/media and geographically disparate locations.
But you also need to protect yourself from sabotage – and your first line of defense there is clearly HR.
“A disgruntled member of the Lagomorphics team sabotaged some key servers several months ago after he was caught stealing from the company; as awful as the thought is, we can’t rule out the possibility of additional sabotage.”
Sabotage is certainly preventable but can be much more difficult. If you are going to have one “guy handling IT”, it’s quite conceivable that he’ll have the authority to destroy all the backups and clear data from your servers. A person hired into that position must not only be up to the task technically but absolutely trustworthy. You must be willing to say: I would bet my company that <<person we’re hiring>> would never do something like that. If you’re not willing to say that, you need to pass on hiring him — or (more likely, since you may not know the person that well) provide a backup structure that you completely understand and cannot be circumvented by one person.