So, you've got data to delete...

For me, the timing of DogAte has been interesting.  I've been in the middle of completely overwriting the hard drive on my new laptop (it caught a lovely little difficult-to-uproot virus -- I'd like to send a big "thank you" to the creators of "vundo" -- and in the course of attempting a disinfection, I accidently nuked Windows).  In the process of re-installing everything from scratch, I decided to encrypt part of the hard drive.

So, say you want to completely erase sensitive data and/or protect private information from prying eyes.  How do you go about it?  As it turns out, it's not all that tough, and most of it can be done with software that is absolutely free.

What follows is a Windows-centric description of this process.  Similar (and sometimes, the same) programs are available for most operating systems, though.

The first thing I did, of course, was back up my data.  I couldn't get into Windows, and couldn't even "repair" my copy of the operating system, since one or more Windows Updates had left the system incompatible with the original install.  However, all was not lost:  I had a copy of a Windows Live CD onhand (a version of Windows that can be booted and run entirely from a CD-ROM).

Anyone can make such a CD in case of emergency.  The easiest way (in my opinion) is to download and install a program called "BartPE" (link).  Run the program, and it basically walks you through the process and can burn the CD directly, assuming you have a CD burner.  You will need your Windows CD or the Windows installation files (frequently found on your computer, anyway).

So, I booted into Windows, attached an external hard drive via a USB cable, and backed up everything critical.  What next?

I wanted to start from scratch.  I didn't want to take the chance of any private data being left someplace it shouldn't be on the hard drive.  So my next step was to nuke my hard drive (not literally).

The best-known and easiest way to do this, IMHO, is to use a program called DBAN (short for "Darik's Boot and Nuke".  As the name suggests, DBAN is a program that lets you boot a computer and completely overwrite all data on its hard drive.  As with BartPE, you'll need to download the program and burn a CD-ROM.  It comes in several forms; the easiest is the .iso format (a disk image), which can be burned to CD's using programs such as Nero or the freeware Windows plugin ISO Recorder.

The author of DBAN, by the way, has another excellent free program out there.  Eraser can overwrite files and clear free space on a hard drive, making data selectively unrecoverable.

Start your computer with the DBAN disk in it and follow the directions.  Just be ready for everything on the disk to be destroyed beyond any hope of recovery.  That's what you want, right?

My laptop has a 100 GB disk, and I overwrote everything on it once with random data.  The process took about 30 hours.  No, it isn't quick.  But there's a lot of data to destroy, and DBAN will destroy absolutely everything, with one exception: there is an area on many hard drives called the "Host Protected Area" that DBAN can't reach yet.  The HPA may or may not be in use, but if it is it has to be removed before doing the wipe, or DBAN won't get it all.

On to the next step.  This being a Dell laptop, there were a number of specific things I had to do.  But the long and the short of it is, I needed to partition the hard drive to make room both for the operating system and the encrypted data.  I made two partitions, one 35 GB in size and one about 55 GB in size (the remainder was for Dell-specific stuff).  This can be done using a Windows install disk, by the way.

Once done, I loaded Windows onto the 35 GB install partition, and proceeded to re-install everything I needed.  By now, I had two accounts: an administrator account and a limited user account.  It was the limited user account I wanted encrypted.

My program of choice for this task is called Truecrypt.  Truecrypt allows you to create and mount encrypted files that act like little hard drives.  It also allows you to do this with entire partitions.  Truecrypt is an open-source program that implements several world-class encryption algorithms including AES (certified by the NSA for government use in protecting data up to the top secret level).  It can run in the background, providing seamless access to your encrypted data once you enter the password.

In this case, I wanted the profile (e.g. everything normally found for a Windows account under "C:\Documents and Settings\xxx" for the limited user entirely encrypted.

Fortunately, a Truecrypt user has created an add-on to do just this.  It's called TCGina, and it works in conjunction with the normal Windows logon to mount an encrypted partition that contains the profile in question.  The installation is a little bit beyond the scope of this writeup, but the included manual/help file is more than adequate for describing the process, which is highly automated.

So here I am typing away on a laptop in which all my data is stored entirely within encrypted partitions, utterly unrecoverable without the password.  Not bad.  Wonder if the RNC did anything like that?

•  Tip Jar

  This jar will self-destruct in five seconds...

  by Jonathan on Thu Apr 12, 2007 at 07:31:47 PM PDT

  •  So I'm wondering

    If your tip jar didn't self destruct in time, how reliable are your instructions?

    <snark>

    by Auntie Neo Kawn on Thu Apr 12, 2007 at 07:38:35 PM PDT

    •  Good point

      And that gives me hope that even Karl's "deleted emails" might still be recovered.

      by Jonathan on Thu Apr 12, 2007 at 07:40:31 PM PDT

    •  Answer.

      Not very.

      Send me his machine and I will recover everything.  He has only perored the most elementary precautions.   They are inadequate if confronted by a professional with the right equipment.

      For example, overwriting data once is simply inadequate, in needs to be done around 10 times.

      The reason for this is that the relationship betwen the magnetic domains on the disc and the heads.

      by Demena on Thu Apr 12, 2007 at 07:44:41 PM PDT

      •  Really?

        You'd be the first person I've ever heard of who can actually make use of such recovery techniques.  I've certainly read papers on them, but that's a different story.

        So, what technique do you use to recover the data?  MFM?  Do you actually have the facilities to perform such a recovery, or are you speaking theoretically?

        Encase or Winhex won't do the trick in this case, you know.

        by Jonathan on Thu Apr 12, 2007 at 07:47:46 PM PDT

        •  Oh, and...

          if you are talking about actual hardware methods of recover such as magnetic-force microscopy, 10 times won't do it (at least, according to papers such as Guttman's).

          Of course, most of us don't have multi-million-dollar clean rooms, the microscope and data storage equipment do apply such methods.  If you actually do, I'm genuinely interested in hearing about it.

          by Jonathan on Thu Apr 12, 2007 at 07:50:37 PM PDT

          •  I don't

            but friends do.

            With regard to the 7/10/x times.  Each pass makes the data harder to recover.  Ten to fifteen will drop the probablility of recovery to infinitesimal (with current tech).

            by Demena on Thu Apr 12, 2007 at 08:16:50 PM PDT

            •  OK

              To be clear, IMHO most of this sort of thing is TV fantasy.  Are these techniques possible?  Yep, theoretically speaking.  But I have a background in microelectronics fabrication, and I work in government.  And I have a pretty good idea just what would be involved.  We're not talking cheap or easy.

              What's more, if data were actually recovered from a hard drive and used in any sort of law enforcement case, it would attract quite a bit of attention, as the actual applicability of these techniques is even today a hotly debated issue.  So while I would imagine some organizations with bottomless pockets can make use of these techniques, I would also imagine few of us (not involved in espionage, etc.) will ever have our hard drives subjected to this sort of scrutiny.  Especially when there are so many easier ways of obtaining data.

              I'm not calling you a liar.  But I am very skeptical.

              by Jonathan on Thu Apr 12, 2007 at 08:23:41 PM PDT

              •  Ok

                To answer both your posts.

                We're not talking cheap or easy.

                That claim I never made.  But 98% of costs are set up costs.

                What's more, if data were actually recovered from a hard drive and used in any sort of law enforcement case, it would attract quite a bit of attention, as the actual applicability of these techniques is even today a hotly debated issue.  So while I would imagine some organizations with bottomless pockets can make use of these techniques, I would also imagine few of us (not involved in espionage, etc.) will ever have our hard drives subjected to this sort of scrutiny.  Especially when there are so many easier ways of obtaining data.

                You are never, never going to hear about it. It is never going to appear in court.  What will appear in court is evidence found elsewhere by information gained from the drive.  Never throw techniques at a court that the court will not understand.

                Anyway, I am drawing a line here before I get myself into more trouble.

                by Demena on Fri Apr 13, 2007 at 12:30:22 AM PDT

                •  Oh, come on

                  The bottom line is, no one has demonstrated the ability to do what you claim.  Not law enforcement, and not data recovery specialists.

                  Many (if not most) of the services and technologies used by the government are marketed TO the government.  If it could be done, do you really think some data recovery company out there wouldn't be marketing it?

                  And how do you think a case brought against someone using this technology could NOT be made public?  The defense would cross-examine the origin of the data inside and out.  By law, they get to do that.  The prosecution doesn't just get to say "oh, we have this information.  We can't tell you where we got it."  That's called "chain of evidence", and it exists because where you got the evidence matters.

                  Our justice system, even now, simply doesn't work the way you claim.

                  I'm an engineer.  My stock in trade is data, not anonymous claims on the Internet of a nature that are often made, but never backed up.  I'll believe it when I see it.

                  by Jonathan on Fri Apr 13, 2007 at 03:47:06 AM PDT

  •  The presence of your noted software

    on a WH drive, by itself is probably illegal, given the explicitly stated archiving rules.

    by episty on Thu Apr 12, 2007 at 07:42:46 PM PDT

    •  The presence of that noted software

      also obviates the terms "mishandling" or "lost" or "my dog ate it."

      by episty on Thu Apr 12, 2007 at 07:44:42 PM PDT

    •  Maybe not illegal...

      But very suspicious to outsiders, and most likely against White House IT policy.  Chances are, they already use encryption on their laptops, etc.  But it's sanctioned officially.

      I don't work at the White House, but I am at a federal agency.  Generally speaking, you're not supposed to install anything without permission.

      by Jonathan on Thu Apr 12, 2007 at 07:45:14 PM PDT

•  Impressive diary.

  I've been using Jetico's BC Wipe for a decade and it seems to successfully wipe any unwanted files.  The original was DoD 7 stage random data overwrite, but it has since expanded to far more serious erasures.  Do you know bc wipe?

  Thanks for the informative diary.

  by bigchin on Thu Apr 12, 2007 at 07:42:28 PM PDT

  •  Seven

    passes will make it hard to recover but not impossible.  Ten to fifteen is needed for real security.

    I am not as impresed with the diary as you are.

    by Demena on Thu Apr 12, 2007 at 07:45:55 PM PDT

    •  Seven is an extended DoD wipe

      One is entirely sufficient for anything that a normal attacker would bring to bear, including such forensic tools as are normally used in law enforcement.

      So I guess the question becomes, what do you think you need to defend against?

      Frankly, IMHO if you have data worth enough that someone will bring techniques to bear that can still recover the data, then you've got bigger problems that I'm intending to solve with the techniques I've employed.

      So I'm sorry if you're not "impressed" with my diary, but then, my diary isn't really intended for you.

      by Jonathan on Thu Apr 12, 2007 at 07:54:38 PM PDT

      •  Well...

        One is entirely sufficient for anything that a normal attacker would bring to bear, including such forensic tools as are normally used in law enforcement.

        Normaly where?  I think you might want to re-examine this.

        So I guess the question becomes, what do you think you need to defend against?

        Which is precisely why I am unimpressed.  You didn't ask the question in your article and it is the very first question that should be asked.  Instead of asking it you gave a methodology which is only appropriate to to a subset of the answers to that question.  

        Thereby risking assuring people that this is adequate for their purposes when it might not be.  Ethicly dubious.

        Frankly, IMHO if you have data worth enough that someone will bring techniques to bear that can still recover the data, then you've got bigger problems that I'm intending to solve with the techniques I've employed.

        Which you failed to mention in the diary...

        So I'm sorry if you're not "impressed" with my diary, but then, my diary isn't really intended for you.

        Then who was it intended for?  People able to be misinformed?

        Audience and scope are the first things you should be thinking of.

        by Demena on Thu Apr 12, 2007 at 08:23:28 PM PDT

        •  OK...

          Normaly where?  I think you might want to re-examine this.

          I have examined this, which is exactly the trouble.  The actual applicability of these techniques is a hotly debated issue.  It would not be so if there were even one case of a person being convicted based on evidence recovered in this way.

          Computer forensics standardly employs software such as Encase (the gold standard).  So when I say "normal", I am using the word with a common-sense definition.

          Which is precisely why I am unimpressed.  You didn't ask the question in your article and it is the very first question that should be asked.

          If you can cite a single case of evidence introduced against someone that was recovered from a correctly-overwritten drive (of even one pass), then we'll talk.  Until then, I maintain that the vast majority of us don't have to worry about it.  It's hardly unethical to write a general "how-to" that is applicable to the needs of 99% of the readers.  It IS hyperbole to try to claim otherwise.

          Then who was it intended for?  People able to be misinformed?

          Audience and scope are the first things you should be thinking of.

          And I feel it was appropriate to the vast majority of readers.  If you feel otherwise, the request for citation above can easily settle the matter.

          by Jonathan on Thu Apr 12, 2007 at 08:30:52 PM PDT

          •  Finally...

            For the record, here's a Cambridge researcher who tried, and failed, to turn up any evidence that data recovery of the type you describe is being done:

            http://www.nber.org/...

            His conclusion was that it was an urban legend.

            While I'm not quite so sanguine (the techniques of certain three-letter non-law-enforcement agencies might be rather difficult to uncover), I've basically run into the same problem: lots of claims, nothing to back them up.

            Which is why I'm quite comfortable with one overwrite pass using DBAN.

            by Jonathan on Thu Apr 12, 2007 at 08:53:48 PM PDT

  •  Hi, BC

    As I noted to Demena, it really depends upon what you want to protect.

    If you want to prevent software recovery of any data (this includes what most investigators would use to recover data), then a single overwrite is sufficient.

    As for more exotic methods of data recovery:

    Several researchers have published papers discussing the recoverability of overwritten data.  It's clear that even when data is overwritten, some of it remains (both along the side of the data tracks and in the actual magnetic thresholds of the current data bits).  So even multiple over-writes can leave some remaining data.  Of course, I've yet to read of any court case, etc. that included such exotic methods of recovery in the production of evidence, so it's very probable such methods remain theoretical, for the most part.

    If you want to be safe, overwrite multiple times.  If you really want to be safe, drill your hard drive open and take a bench grinder to it.  It all depends upon what you think you have to be concerned about.

    by Jonathan on Thu Apr 12, 2007 at 08:08:34 PM PDT

    •  Most data wiping software

      Is hype. It's good enough to fill most people's privacy needs and to evade local law enforcement though (if they really need it they send the drive off to the FBI). The DOD doesn't use anything like that to delete the data because it's too time consuming and it doesn't do the job. When they want to make sure the data is unrecoverable they destroy the hard drives by either incineration or with the newer glass/ceramic substrates by basically shattering the platters into enough pieces they could never be reassembled and read.

      Bottom line: If the hard drive platter exists and it's in reasonably good shape any data stored on it can still be retrieved. All you need is the right technology and a clean room.

      You probably won't see any court cases where people have gone to that extreme to recover data (usually there's enough other evidence to convict) but you can bet the CIA, FBI, and NSA have the capability.

      by Cali Techie on Thu Apr 12, 2007 at 09:43:37 PM PDT

      •  Possibly

        The FBI, even now, is a law enforcement agency.  If they had the ability to do what you say and had ever used it, that fact would have turned up in court cases somewhere.

        And, as I noted above, most technology used by the government (e.g. AES, supercomputers) is sold to the government by contract vendors.  Aspects of that technology may certainly be classified.  However, I find it very difficult to believe that if this particular technology were actually available, some data recovery specialist firm somewhere wouldn't be marketing it.

        Bottom line is, though, most of us will never face it, even if it exists.  Such a sensational secret couldn't be kept secret otherwise.

        by Jonathan on Fri Apr 13, 2007 at 03:51:31 AM PDT

        •  I believe they've used it

          While investigating cybercrimes and child porn rings. It is also commercially available at a very high cost. It's not 100% effective and some files will be unrecoverable, but it's not guaranteed that if you use data wiping software the data cannot be recovered.

          by Cali Techie on Fri Apr 13, 2007 at 08:14:41 AM PDT

          •  Some extra reading

            It's true that there's no such thing as a guarantee.  NSA recommends multiple overwrites or physical destruction.  Of course, when you're tasked with protecting data (the release of which would cause exceptionally grave damage to national security), you don't take chances, even with theoretical security holes.

            I've been doing some research on this topic over the last day or so, reviewing documents I saw about a year ago when I was a hard-core forensics junkie.

            This one I mentioned previously: Can Intelligence Agencies Read Overwritten Data? A response to Gutmann.  Interesting in that it points out no one has yet demonstrated the ability to recover overwritten data.  The paper cites several reasons to believe that the possibility of such recovery belongs in the category of urban legend.

            Erasing hard disk drive data: How many passes are needed? -- discusses recovery of overwritten data.  Here are their main points, based upon actual research into such recovery:

              1. Nobody has ever shown they can actually do this. (This, to me, is the most important fact: No one has ever actually taken a hard disk drive, recorded data on it, overwritten the data, then attempted to recover it—let alone demonstrated that they can do this reliably.)

              2. No reputable data recovery expert believes this is possible or advertises that they can do it. (Do you want to end up in the position of having taken money for a service you can't provide?)

              3. If it isn't possible to do it commercially, there's a strong chance no intelligence agency can do it either.

            Secure Erase of Disk Drive Data -- this one's my favorite, as it speaks to my own research background in electronics and microelectronics fabrication/testing techniques.  This is a research paper by the Center for Magnetic Recording Research at the University of California San Diego, discussing a study in which hard drives were subjected to "exotic" recovery techniques.

            The researchers point out that even one overwrite pass leaves remnant signals that are far, far lower than the signal-to-noise level required for the standard digital hard drive read path to detect.  So far, not surprising.  I can, myself, verify this: I've examined the raw digital data of hard drives overwritten with DBAN using forensic software.  There's nothing left to see.

            What's interesting is that they found, through testing of the raw read head output signal using an oscilloscope, that after one overwrite pass they could barely detect the overwritten data.  The kicker is the conditions:

            1. they had to use multiple read passes (about 100) to decrease the noise sufficiently,

            2. they had to already know the data pattern they were looking for,

            3. the data had to be a repeating pattern,

            4. they had to know the overwrite pattern

            After two overwrite passes, the data was utterly unrecoverable no matter what crutches were available, as it was buried in the noise.  To quote them:

            Drive information can sometimes be recovered that has been erased using a single erasure pass on-track. It should be first pointed out that single-frequency squarewave overwrite tests are not meaningful indicators of information recovery. A spectrum analyzer can see - 60 dB overwritten signals but it can’t recover data. The CMRR technique requires reading a data block many times, computer averaging the playback waveforms, then erasing the block and re-recording the overwrite data to obtain its  averaged playback waveform data, which is subtracted from the first waveform. The demonstration shown below merely means that it is possible, not that it is practical or will work on any drive. It requires knowing the data pattern being looked for, and also knowing the overwriting data pattern. So it "begs the question." It can be defeated by using a random data overwrite pattern.

            To my hardware-oriented, engineering mind, that pretty much settles the matter.  But I do realize that each of us have our own ideas of what constitutes proof.

            by Jonathan on Fri Apr 13, 2007 at 09:19:46 PM PDT

    •  well I guess I'm not worried much.

      I'm more concerned about remnants of my (bad) crime novel ever coming to light... not to mention poems and letters, etc...

      heh...

      BC wipe was a free download back when I first started computing and it seemed a prudent means of cleaning my hard drive.

      by bigchin on Thu Apr 12, 2007 at 11:15:26 PM PDT

•  Pop out the hard drive...

  beat it with a hammer then sprinkle lighter fluid on it, light it then beat it with the hammer again.

  by Gator on Thu Apr 12, 2007 at 07:43:50 PM PDT

  •  To me...

    ...that's not a bad idea when you're done with the drive, and probably highly satisfying.

    My personal preference is to use a 20-ton shop press.  If anyone (including Demena, above) can recover data from the paper-thin mash that is the result, then they deserve it.

    by Jonathan on Thu Apr 12, 2007 at 07:56:33 PM PDT

    •  That's what it takes...

      Basically.. total annihilation of the drive.

      by Cali Techie on Thu Apr 12, 2007 at 09:45:31 PM PDT

  •  Then put it in a bag & throw it in the river! n/t

    by mike101 on Thu Apr 12, 2007 at 08:05:14 PM PDT

    •  Don't forget to give it the concrete shoes!

      Or wire it to the cinder block so you can be sure it will sleep with the fishes.

      by frostyinPA on Thu Apr 12, 2007 at 10:01:32 PM PDT

•  So if they used an overwrite program

  that should be evident right?  It should be obvious that the hard drive in question has been scrubbed, or at least attempted to be scrubbed based on the pattern of data.  A virgin drive OTOH should have unused sectors that only have formatting info ... correct?

  by kbman on Thu Apr 12, 2007 at 10:11:50 PM PDT

•  You can erase your HD for sure

  by dropping it into a vat full of molten steel like the cyborg at the end of Terminator 2.  The technical means to erase data will always exist.  The question is whether erasing it will result in getting sent to jail for obstruction of justice, and in judges giving juries instructions for adverse inference about the contents of the missing emails at your trial.  Let's hope they lock up Karl Rove and Abu Gonzales for a long, long time.

  by clyde on Thu Apr 12, 2007 at 10:42:00 PM PDT