I totally can't figure out any way now. I manage a server which runs Ubuntu 10.04 LTS AMD64. Several days ago, another sysadmin tried rebooting the machine and it never boot up again.
I'm sure that the ureadahead thing is harmless, and once I moved the ureadhead.conf to other place, the warning disappeared. (Other servers I'm in charge also show the same kind of warnings, but they are fine.)
The server has several hard disks, in order to exclude the posibility that the filesystem problem stuck the whole process, I commented out all the other disk entries, now only "/" "swap" "boot", and the /dev/xxx entries show up on screen. and they all said
/dev/xxx:clean, xxxx/xxxx files, xxxxxxxx/xxxxxxxxxx blocks
After removing the mount-tmp.conf in the /etc/init folder, I got rid of the
init:mounted-tmp main process (946) terminated with status 127
and then it showed that plymouth-log terminated with xx and I again move away the relative file in /etc/init and as expected, they also disappeared.
However, my server is still stuck there with a cursor though now it don't have so much warnings and errors displaying there.
By the way, I guess my system have a rootkit which stop all the call to /bin/ls, even if I copy a ls file from other normal Ubuntu to this server, and the new ls can't run either (even after renaming it)! Strangely, I can run the dir command which, in my opinion, works all the way like ls, but I did notice the dir file is much larger than the ls command in size.
As I mentioned before, I can enter the single-user mode, though the mode still can't run the ls command, but at least I can enter the system, that means the rookit didn't stuck the kernel, right?
I know many people would say "oh, you got a rootkit, go reinstall", but I'm curious, besides reinstalling, how can I debug this problem? If the problem is really caused by a rookit, as I use live-usb to inspect the server, I should, at least theoretically find out some trace of the rootkit. Can you guys tell me what process and steps I should follow?
Thank you all.
(Sorry, I'm a new user at Super User, and temporarily can't post my screenshot here.)
