Urgent-Need to recover data off some HDDs but running in to errors

Ok so i had a server setup using Fedora 12 with Amahi loaded on it, which was running well enough, had Grayhole setup and running great. well we had a massive power outage and i hadn't setup a UPS yet so the server had a hard shutdown, well when the power came back on in a few days, the server was spiting out errors like crazy, turned out that the motherboard was shot, sata controller was not picking up all the drives and was deflating to 1.5gb/s speeds.

So now that leaves me with a couple of drives with data on them that i need to get back. lucky for me Grayhole just distributes the data around and does not strip it like RAID, so all the files are all there.

So i get a new server built to use with OpenIndiana + napp-it to try my hand with ZFS, but first i need to get the files off

Im using an ubuntu live cd, to try to copy the data over and this is where i run in to problems, i keep getting I/O errors which i think are caused by Permission problems.

I have tried copying it with both the GUI and the terminal, i have tried Rsync, i have tried changing the permissions still cant get it to copy the files over (some do but no the important ones)

so after about 3 days of googling i find this program called Safecopy, which ignores I/O errors, so i load it up on the live cd but every time i try to use it it just spits out the help info, i have tried compiling it from source and the apt-get commands, but i just cant get it to work, but it installs with out errors.

Im all out of ideas, i am no linux expert nor fluent in CLI but i do know my way around.

Please help, i need to get this data off quickly as i need to get access to it soon.

Thanks!

PS sorry for the long post.

 

I've used safecopy a couple of times quite some time ago and it is pretty much easy to use.

safecopy [OPTIONS] FILE_TO_RESCUE SAVE_AS_TARGET_FILE

Using one of these recommended options you can choose between maximum retries or maximum safe files only.
--stage1
--stage2
--stage3

The --help option should give you more detailed info or the man page too.

If you want to rescue the entire disk/partition - I think you can use that as the SOURCE and your DESTINATION would be an image of that disk/partition that would be created. The image is similar to an ISO image I believe that can be mounted with -o loop.

Could you describe the exact command you're trying and the problem you're facing.

 

see thats the thing i use the safecopy commands exactly like that with --stage2 but it just spits out the help menu no matter what i do.

I have tried the cp command, rsync -av, iv tried the chmod but it doesnt change the problem.

I dont know of anyway to tell it to ignore the I/O errors and thats why i tried safecopy.

EDIT: i read on another forum that some one was having a similar problem, it wouldn't transfer over 800mb or so and would give out I/O errors as well, but i dont know if that is the same problem im haivng

 

Yes cp, rsync, tar are all bound to fail on I/O errors. Is there any error message printed just before the start of the help message ?

 

very strange now its not spitting out the help menu but its still giving an error.

Error opening file for reading: stage1.badblocks : No such file or directory

and thats with the command safecopy --stage2 xxx xxx

 

Ok my bad I forgot how it used to work

First do a --stage1 where it writes a stage1.badblocks file. If you recover your file in stage1 it is good otherwise, you need to do a --stage2 where it would read this file and write a stage2.badblocks file. If you got your file now it is good otherwise a --stage3

 

ok that makes sense.

its running but as the thing flashes by it is only saying

Error: write to output life failed : Input/output error X ;-} 100%

is it because its a directory of files and not a single file?? do i have to make it in to an image?? would it be just making the drive in to a image the copying that over or is there something more that i have to do??

 

It looks like you're trying to recover files from a partition and writing the files to the same partition or you're trying to write to a directory you do not have permissions for.

Could you confirm that you're trying to recover files onto a separate partition which is on a totally different disk which doesnt experience any I/O errors ?

 

What im trying to do is copy files from one drive that is formatted in ext4 to a NTFS external hdd plugged in by esata so that when i setup the new sever i can copy the data over the network from my windows box.

im trying to copy a directory called gh.

I have been typing it in as

safecopy --stage1 /media/2518c3a9-4662-4734-913d-8fe86a54ad61/gh /media/Server/gh

 

Can you tell me the output of these two commands:

Code:

touch temp123
touch /media/Server/temp456

Basically I'm trying to find out if your user has permissions to create file in the current directory (which is what the safecopy program is for stagex.badblocks) and the destination directory.

If both of them fail then there might be something else which is a problem. Can you copy and paste the complete output of the command when it prints the error.

 

nope no error

when i do ls -l for /media/ this is the output

drwxr-xr-x. 4 root root 4096 2010-11-21 19:30 2518c3a9-4662-4734-913d-8fe86a54ad61
drwxr-xr-x. 4 root root 4096 2010-11-21 19:30 27f479d8-c57d-4000-8917-7e62d2357e03
lrwxrwxrwx 1 root root 6 2011-07-26 13:32 cdrom -> /cdrom
drwx------ 1 ubuntu ubuntu 4096 2011-07-26 19:48 Server

The long drive names are the two drives from the downed server and the drive Server is the external HDD im trying to copy to

When i do the touch command on one of the server drives i get

touch: cannot touch `/media/2518c3a9-4662-4734-913d-8fe86a54ad61/temp123': Permission denied

if thats any help

 

Okay just check in your current directory if any stage1.badblocks file was created with a non-zero byte file size, if so then try safecopy --stage2 command to see if it helps.

 

Another option would be to do it on the actual disk/partition itself. Could you post the output of:

Code:

df -h

and using that you can find out which devices are your bad hard drives, and then image them using safecopy.

Substitue source file name with /dev/sdaX (X is the number you find out using df command).

 

the things is that when i do safecopy --stage1 /media/2518c3a9-4662-4734-913d-8fe86a54ad61/gh /media/Server/gh it outputs a single file which i dont think is what i want it to do but when i but in safecopy --stage1 /media/2518c3a9-4662-4734-913d-8fe86a54ad61 /media/Server i give me an error saying Error opening destination: /media/Server : Is a directory, which should i use for copying the files from the gh folder and their sub folders to a similar folder in the Server drive.

EDIT: will do the command you said

 

heres the output:

Filesystem Size Used Avail Use% Mounted on
aufs 1.6G 504M 1.1G 32% /
none 1.6G 700K 1.6G 1% /dev
/dev/sr0 686M 686M 0 100% /cdrom
/dev/loop0 658M 658M 0 100% /rofs
none 1.6G 2.7M 1.6G 1% /dev/shm
tmpfs 1.6G 828K 1.6G 1% /tmp
none 1.6G 96K 1.6G 1% /var/run
none 1.6G 4.0K 1.6G 1% /var/lock
/dev/sdc1 932G 6.6G 925G 1% /media/Server
/dev/sdb1 917G 379G 493G 44% /media/2518c3a9-4662-4734-913d-8fe86a54ad61
/dev/sda2 819G 272G 506G 35% /media/27f479d8-c57d-4000-8917-7e62d2357e03

 

Yes you're right when you're trying to recover a single file using safecopy you should give the destination path to a file and not a directory - so the first command is the valid one.

But why do you say the file is not what you want ? Because of the file size ? BTW gh is a file and not a directory right ? AFAIK safecopy doesnt find and recover files recursively, so you should run this command for each file and then recover or try the option of imaging your entire partition.

So based on your output of df, I can figure out that /dev/sdb1 and /dev/sda2 are your corrupted hard drives, and these commands should help you create the image of those.

Code:

safecopy --stage1 /dev/sdb1 /media/Server/myimage1
safecopy --stage1 /dev/sda2 /media/Server/myimage2

I'm guessing you might need to run these commands as sudo BTW

 

gh is not a file but a folder that grayhole copies files in to on the drive(s) and im trying to copy all the folders and files over, the thing is though to do this one by one would take months as its about 500gb of data total.

the commands you listed, do they copy all the files and folders into a single file or a container of some sort??

Thanks again for all your help.

 

well i ran the commands you said and it still gives me an error

Code:

Error: write to /media/Server/myimage1 failed : Input/output error

i was using sudo and it does make the file myimage1 on drive Server so im lost on whats going one here

 

Ok so i think its still permissions errors and im wondering if its because im using a liveCD because no matter how i try to change permissions, GUI or terminal, it does not change them. does this mean i will have to install ubuntu?? or use another distribution?? i might try to use a PartedMagic live cd as it is meant for this kinda thing.

 

Yes parted magic is a good LiveCD as well that you can use for these tasks.

Yes if it is a directory unfortunately you need to repeat this step again and again. You can automate this with a script but first I need to know the directory structure of gh especially if it will contain subdirectories. But detecting from the script if the recovery finished in stage1 or not or which subsequent stage might be some problem.

This error looks like some permission issues in /media/Server. With sudo/root there should be no permission issues whatever.

One thing I can suggest is if you happen to have a hard drive formatted with a linux based filesystem ext2/ext3/ext4/reiserfs/xfs/brtfs etc. try to see if you're able to recover the files into that partition.

Ideally the entire partition should get imaged as a single file just like an ISO file for CDs/DVDs

 

I did use sudo and it gives me that error but if i dont use it give me an error about /dev/sda.

My problem is that i need the external drive to be ntfs because i will have to be reading it from my windows laptop, the only other computers are macs but they dont have esata so it could take hours to days to copy the 500gb or so. i guess i could load up another live cd for the copy but it would put me out of a computer for a while. i guess will try it but im going to move over to parted magic for it, it will just be easier.

As for the file structure thats a whole other problem, when i had set up the amahi server i had made directories for different things like movies, picture, backups, etc. but the files that where added came off an external hdd and they dont have a clear cut structure (something i was planing to change but never got to it), and then theres also the problem because i used Grayhole the files and their folder structure is spread out between the two drives. example being:

Code:

Drive 1 - 
  -gh
    -Pictures
      -some vacation some where
        -Somepcit1.jpg
Drive 2 - 
  -gh
    -Pictures
      -some vacation some where
        -Somepict2.jpg
        -otherpict1.jpg
      -Someotherfolder

not clean but this is how my parents kept it (it was there server).

Wouldn't an image have the permissions as the drives?? if not would it be as easy as mounting it and copying the data??

Thanks for all your help.

 

I'm pretty clueless as to how to make safecopy recurse all throughout other than writing a script which does the same.

Can you try recovering a single file if possible where you found cp to be failing earlier ?

Yeah only root can access /dev/sda or any other block device (hard disk partiitons, flash drives, etc.) directly at the lower most level.

 

Tried it with a DVD rip but it didnt work it spat out an error.

Code:

Error: write to /media/Server/88.Minutes.avi failed : Input/output error

the thing is when ever i do the safecopy, even if i get that error, it still makes the file on the drive it just does not copy over the data. and if i do --stage2 it spits ou this error

Code:

Destination filesize not reported by stat(), trying seek().
Error determining destination file size, cannot resume!

and the same thing on --stage3

i think i can narrow it down to permission problems on the to drives, that is why it wont copy i think but im not sure, and it wont let me change them in Ubuntu livecd i will have to try in parted magic.

 

I feel the problem is more of a read from hard disk error which safecopy seems to be complaining about - I tried to do a safecopy to a directory which didn't have permissions on my system and I get a permission denied error rather than the I/O error.

If it still makes the file on the drive, then it does have permission to write, besides you were able to create a file using touch. So permissions is not the problem.

I feel it is just complaining about reading the file from the source.

If all the 3 stages fail - then I feel there is nothing much safecopy can do to recover the file - sorry about that.

You might probably have luck with some other software if it uses a different method/algorithm to recover the file.

 

the thing is that the data is intact as i can read it, play the movies, view the photos, i dont know why it gives me I/O errors when copying the data. When i copy it over the error it gives me most of the time is this

Code:

Error splicing file: Input/output error

if that means anything to you.

The permissions for the two drives they say the owner is root but i figured out how to change that to the livecd user in ubuntu

Thanks

 

That is really interesting - if you're able to view the data as photos, videos - the data should be intact at that too to a very large extent.

I'm starting to suspect the live CD environment itself. Things you can do is:
1. Try some other non-Ubuntu based live cd - there are many including the parted magic you mentioned.
2. Try connecting these hard drives to a machine which runs Linux (not a LiveCD one) and see if you observe different results.

Permissions should not be an issue if you're running a command as root.

 

Thats what i thought.

i will try a different live cd first to see if that works. if i cant get it to copy it then i will install a Linux on there server hardware.will report back if it works or not.

Thanks

 

Hallelujah its a bloody miracle!

i loaded up a Fedora 15 live USB and its coping, its really coping no errors so far (cross your finger and knock on wood). it says i have about 2 hours for the first disk (the smaller one). I should have used Fedora first after all it was the base OS for the server, but one would think ext4 volumes would work across the board . hopefully by tomorrow morning i will have all the data back and read to start using OpenIndiana and ZFS.



TuxDude i cant thank you enough you dont know much you saved my a$$. Thank You!


EDIT:All the files copied over and looks like they are all there and working

 

Wow glad you got everything working

You got through all the steps and were not hesitant to try out new things and ways to do stuff

I always recommend people to switch over to RAID 5 or higher based storage when terabytes of data are at stake I myself invested in a mini-RAID-5 storage for all my movies, pictures, my important data and I dont regret it at all

 

Exactly what i doing now with ZFS and its zraid which is like raid5 but with out the write hole.

Thanks again