ubuntu input/output error

Question

I'm having a problem with Ubuntu that I'm finding hard to troubleshoot for reasons that will become clear:

reboot

-bash: /sbin/reboot: Input/output error

dmesg

-bash: /bin/dmesg: Input/output error

ps -e

ps: error while loading shared libraries: /lib/libproc-3.2.8.so: cannot read file data: Input/output error

lsof

-bash: /usr/bin/lsof: Input/output error

fsck

-bash: /sbin/fsck: Input/output error

badblocks

-bash: /sbin/badblocks: Input/output error

So I can't see what is going on, and I can't remotely reboot. What can I do to get to the bottom of this?

Interestingly:

init 0

Segmentation fault

I can cat /var/syslog but not /var/log/messages or several other important files. less and more don't work, neither do tail or head, etc.

Answer 1

The system is having severe trouble reading off of your hard disk. It's likely that the disk is dead (almost certain), but it could be something as simple as a loose/disconnected cable (don't count on it). There isn't anything you can do to troubleshoot it from here. Just power it off.

Check for loose connections on your hard disk. If everything is fine there boot from a rescue disk and run fsck or badblocks from there.

I hope you have a back up.

Answer 2

I'm having exactly the same problem. I've had problems 3 or 4 times, but what's weird is that it seems that the problems aren't exactly the same each time. eg the first two times I couldn't ssh in or log in at the console, the last time I was able to ssh in, but got many of the errors you listed above.

Even more strange:

1. I am running the system on a raid 1. The last time I had the problem and was able to ssh in, I did see that two of the partitions on one disk (each one in one raid) were faulty. But, although the other disk was fine I still got those errors!

2. When I re-boot everything is fine.

3. I don't see ANY errors in log files.

I'm wondering if this could be some kind of problem with the scsi bus (they are scsi drives), and it's totally messing with the disks. Or, is the system re-building fine because the failed disk is taking some small amount of empty/saved space and re-allocating it?

I have a disk on express order, and of course have backed up. Opinions are welcome (eg. bus error versis disk error vs anything else?), and why no errors logged??