According to a recently released survey by Compuware, most data loss is attributable to either user negligence or malice. Only 1% of data loss is due to hackers. I loved the headline on this story: "3/4 idiots, 1/4 bitterness."

I have to confess to being part of the "idiot" crowd. Three weeks ago I left my laptop sitting on my seat when I got off the train in New York. I remembered it just in time to see my train, with laptop still aboard, disappearing down the track. Except for occasionally recovering individual files that I accidentally delete or overwrite, I haven't actually had a PC disaster since starting Carbonite 3 years ago. So, aside from the pain of having to buy a new laptop, it was fun to use my own product to get everything back. I was really proud of how well it worked.

What I don't see in the Compuware survey is data lost to hard drive failure. For some reason this doesn't show up in the survey, even though I will bet you that it tops all the other categories. We use a LOT of hard drives in our data center, and our statistics show that roughly 3% of all hard drives will fail each year. That's why we use RAID arrays which are 36 million times more reliable than a single drive. Google also publishes their disk failure rate, and it's roughly the same as ours. Hard drives are a data disaster waiting to happen, in our experience. That's why you need a LOT of redundancy in your data storage architecture, as we do. We store our customers' encrypted data on 16 drive arrays. We would have to lose 3 of the 16 drives simultaneously AND your PC would have to crash all at the same time before any data is lost. When you figure the odds of this happening, it's very very close to zero.

I hope you never leave your laptop on Amtrak, but if you do, you'll be glad you've got Carbonite.

— Dave
CEO, Carbonite

I thought you all might be interested to see where all your files live when you back up with Carbonite. This is one aisle of disk drives from our Boston data center. What you're looking at are arrays of 16 1TB data-center grade drives in a RAID-6 array. 3 of the 16 drives would have to fail simultaneously before we would lose any data. This RAID configuration is 36 million times more reliable than a single disk drive. Generally we don't even wait for a drive to fail — we have software that can tell when a drive is starting to get flakey and an alarm goes off on our operations console. A technician pulls the disk and puts in a new one. Within an hour, the new disk is automatically rebuilt and the full redundancy is restored. Every day we back up almost 60 million new files. We have backed up over 11 billion files since we turned our data center on in May 2006. The data center has over 9 petabytes of storage (a petabyte is a million gigabytes). All of this data flows in and out of our data center on two little fiber optic cables the size of a lamp cord. Truly amazing.

— Dave
CEO, Carbonite

Recently, online storage space startup divShare announced on their blog a recent security breach by "a malicious user." Lucky for them, only basic profile information available through the database was accessed during the intrusion. But the important question here is what else could have been taken by a more skilled trespasser?

Many people think that backup is a simple application – what's so hard about backing up a PC?   I remember one of my MIT students grousing about Google's success: "Anyone can write a search engine," he said.  Backing up the data is not the problem. The problem is dealing with huge volumes, millions of database transactions, hundreds of thousands of customers, and all the complexity that this implies – all while making sure that there is 100% security.  Carbonite backs up over 50 million new files every day without losing any of them.  Like any other web site, we constantly get attacked by hackers, but we have enough security measures in place that these attacks are always unsuccessful. As I mentioned in a previous post, Carbonite was one of only two backup services that the guys at Heise Security weren’t able to crack. 

If you’re doing your engineering properly, online backup can be made to be extremely secure.  For instance, Carbonite starts with encrypting the data BEFORE it leaves your PC so that by the time we get it, it's already useless to an intruder in the very unlikely event that someone acutally gains access to our system. We also make sure that the authentication is rock solid, so that there are no "man in the middle" vulnerabilities.  And, we actually pay people to constantly test our defenses. 

After we get your encrypted files, we want to make sure that we don't lose them, so we store all your data on RAID-6 redundant arrays that are 36 million times more reliable than a single drive.  The main Carbonite data center is located in a "bomb-proof" building, alongside those of major Boston financial institutions and telco companies.

Online backup is a hot area right now and you'll see more startups entering the space over the next couple of years.  Not all of them will know enough about security to be really bullet-proof.  It isn't easy or cheap, but I can tell you that for Carbonite it's a live-or-die proposition. 

— Dave
CEO, Carbonite

Several people have asked me to post a description of our infrastructure. As I mentioned in my previous post about HP’s infrastructure difficulties, "HP Upline and the challenge of large scale backup," keeping billions of files safe is no small task.

The first thing you should know about our architecture is that we never handle unencrypted data. Carbonite encrypts all files before they leave your PC. We use 448-bit Blowfish encryption. I’ve been told that Blowfish has never been cracked. It is the strongest commercial encryption on the market.

Carbonite employs the most sophisticated firewalls and intrusion detection systems available. We pay a professional hacker firm to attack the data center constantly, looking for security holes. I think our defenses are as good as most banks. Heise Security recently wrote about how they hacked into many of our competitors’ backup systems but were unable to hack into Carbonite Their so-called “Man-In-The-Middle” test attack is something we designed against from the beginning. Frankly, I was amazed that most of the other vendors were so easily hacked by these guys and backed up files either compromised or deleted.

At our secure data center, your data is stored on arrays of 1-terabyte enterprise-grade drives. Carbonite uses RAID-6 redundant arrays which spread copies of the data across multiple hard drives. Each array has 16 drives. Three of the 16 would have to fail simultaneously and the user’s PC would have to crash at the same time before any data would be lost. These RAID-6 arrays are 36,000,000 times more reliable than the hard drive in your computer. We have redundant power, redundant Internet connections, redundant Web servers and so forth. The data center is guarded 24 hours a day, seven days a week; and admission is controlled by fingerprint ID locks.

As you can imagine, we use a lot of bandwidth. We currently back up over 40 million new files every day and we have over 7 billion already backed up. Given the amount of bandwidth we use, it’s best to be located in a major telecoms center where multiple carriers converge. Therefore, we chose to build our data center in one of those so-called “bomb-proof” buildings with all the major Boston financial institutions and telcos.

— Dave
CEO, Carbonite