Server Woes
My server is in peices in the Kitchen table. I don't like seeing it that way. Here's the story so far:
Ok, so last night when I went to access the /data2 partition on my fileserver, it came back and told me that it couldn't find it: ls returned an I/O error. I played with it for a few minutes, then (since I was at work), I called ertach over to take a glance at it, since he knows more about servers than I do. He poked and prodded it for several minutes, and then ran some deeper testing that appears to have revealed that the Superblock on the partition was bad. The rebuild command is dangerous to run, as it clould result in data corruption, so we decided to borrow a spare drive and go to my house after I got off work to backup the drive before we tried to repair it.
Sol we went home, and ran a bunch more tests, that appear to have shown up other issues (another drive that doesn't want to show up all the time...), and we went to copy the old drive over to the new one. We startede this, played some magic games, then went ahead and pulled out the Puzz3D Star Destroyer that belongs to him that has been sitting at my apartment, begging to be built. Several hours later, with the process not done, I sent him home to sleep, figuring he could walk me through the rest of the commands in the morning.
So I woke up & checked it today. The command was done. That was good. It had errored out. That was NOT good. The reason it errored out: it said it had run out of space to copy the disk to - it said it copied 100GB of data (the failing disk, /dev/hdd was 100GB ReiserFS partition). I went, "Hmm. That spare disk (/dev/hdc) is 120GB. How could it have run out of space copying a smaller drive over to a larger one?" ... The I looked at the command:
dd if=/dev/hdc of=/dev/hdd conv=noerrors
dd = the command we ran (disk to disk copy, or something along those lines)
"if" = input file (device)
"of" = output file (device)
"/dev/hdc" = WD 120 GB Drive, spare, empty, on borrow from somewhere...
"/dev/hdd" = WD 100 GB Drive, nearly full, 4.5 years old (purchased Summer 2002)
What did we do wrong?
Do you see it yet?
Yeah, we copied the contents of the nice, new empty drive over to the full, trying to fail drive. Ouch^32
Now I'm trying to find a way to try and recover the data from that disk. We found a program that will let me see what is on the partition, but only see - it's the trial version of the program, you have to pay $429 to get the version that will actually recover the data. Ouch.
Does anybody out there who I haven't talked to about this have any other ideas for me?
Right now the plan for getting files back is to find the list of what was on that drive then try to find those files from friends. About the only *good* thing is that of the four drives in that box that could fail, the stuff on this drive is the "least" important - it's not the music files and it's not the anime, and it's not the drive with the OS or the /home directories (which includes my windows My Documents folder).
The plan for the box itself it to take it over to Tom's house tomorrow, to figure out what is going on with the other drive, and maybe try it in a different case. i don't want to buy a new HDD for the server, but that might end happening as well.
Ok, so last night when I went to access the /data2 partition on my fileserver, it came back and told me that it couldn't find it: ls returned an I/O error. I played with it for a few minutes, then (since I was at work), I called ertach over to take a glance at it, since he knows more about servers than I do. He poked and prodded it for several minutes, and then ran some deeper testing that appears to have revealed that the Superblock on the partition was bad. The rebuild command is dangerous to run, as it clould result in data corruption, so we decided to borrow a spare drive and go to my house after I got off work to backup the drive before we tried to repair it.
Sol we went home, and ran a bunch more tests, that appear to have shown up other issues (another drive that doesn't want to show up all the time...), and we went to copy the old drive over to the new one. We startede this, played some magic games, then went ahead and pulled out the Puzz3D Star Destroyer that belongs to him that has been sitting at my apartment, begging to be built. Several hours later, with the process not done, I sent him home to sleep, figuring he could walk me through the rest of the commands in the morning.
So I woke up & checked it today. The command was done. That was good. It had errored out. That was NOT good. The reason it errored out: it said it had run out of space to copy the disk to - it said it copied 100GB of data (the failing disk, /dev/hdd was 100GB ReiserFS partition). I went, "Hmm. That spare disk (/dev/hdc) is 120GB. How could it have run out of space copying a smaller drive over to a larger one?" ... The I looked at the command:
dd if=/dev/hdc of=/dev/hdd conv=noerrors
dd = the command we ran (disk to disk copy, or something along those lines)
"if" = input file (device)
"of" = output file (device)
"/dev/hdc" = WD 120 GB Drive, spare, empty, on borrow from somewhere...
"/dev/hdd" = WD 100 GB Drive, nearly full, 4.5 years old (purchased Summer 2002)
What did we do wrong?
Do you see it yet?
Yeah, we copied the contents of the nice, new empty drive over to the full, trying to fail drive. Ouch^32
Now I'm trying to find a way to try and recover the data from that disk. We found a program that will let me see what is on the partition, but only see - it's the trial version of the program, you have to pay $429 to get the version that will actually recover the data. Ouch.
Does anybody out there who I haven't talked to about this have any other ideas for me?
Right now the plan for getting files back is to find the list of what was on that drive then try to find those files from friends. About the only *good* thing is that of the four drives in that box that could fail, the stuff on this drive is the "least" important - it's not the music files and it's not the anime, and it's not the drive with the OS or the /home directories (which includes my windows My Documents folder).
The plan for the box itself it to take it over to Tom's house tomorrow, to figure out what is going on with the other drive, and maybe try it in a different case. i don't want to buy a new HDD for the server, but that might end happening as well.