A particularly special user of mine inadvertently ran a command to move files that looked something like this:

mv /* /home/ubuntu/GS14K/

This led to a series of errors:

mv: cannot move /bin' to/home/ubuntu/GS14K/bin': Permission denied

mv: cannot move /boot' to/home/ubuntu/GS14K/boot': Permission denied

mv: cannot move /dev' to/home/ubuntu/GS14K/dev': Permission denied

as you would hope, but then this error appeared:

mv: cannot move /mnt' to/home/ubuntu/GS14K/mnt': Device or resource busy

mv: cannot move /proc' to/home/ubuntu/GS14K/proc': Device or resource busy

SSH then stopped working and he couldn't get back in. I can't either and I can't access the box.

It's an AWS VM so I forced a stop and rebooted, but the machine won't come back up. I acknowledge the machine is probably dead, but I'm interested to know what the cause might be.

Edit: I was running this on Ububtu, and the user did not have root at the time, so I'm curious as to how he could run this command to do something like this at all.

  • Did this user have root privileges? A non-root user shouldn't be able to cause disruptive changes. If you do need the data off the machine, and it's an EBS backed instance, you could connect the storage device to another VM and recover it that way. – qasdfdsaq Jun 16 '15 at 15:50
  • Thanks, and indeed that is what I ended up doing. The user didn't have root so I'm surprised they were able to move as much as they did. Looking at the attached drive /home/ is now a complete mess of system folders. – monkeymatrix Jun 17 '15 at 9:40
  • 1
    +1 for the expression a particularly special user of mine. You really are the forgiving type. – MariusMatutiae Jun 26 '15 at 20:20
  • Without root access (or access to root via sudo), the user would not have been able to move any files under /. Are you 100% sure they didn't have such access? This of course assumes you didn't have any system files with odd permissions (ie write access to non-root users)... – mjturner Jul 3 '15 at 10:50

Its because if your boot device is disk, then the bios will expect the boot loader to be present in the MBR of your disk, which will try to load the kernel from the device location specified in the boot loader configuration. Most likely this will be /boot/kernel-image, now that you have moved everything to /home/, the boot loader will not longer find the kernel image. Also in case of grub, the boot loader, it loads in 2 stages, first stage will be in MBR and second will be again specified at a device location, so there is a good chance that even the 2nd stage of the boot loader will not load

You could read more here

  • you are right, but it needs a better explanation. MBR or EFS won't be relocated by the entered command, and the kernel wasn't relocated, so I suspect /etc/grub was moved, so the GRUB 2nd-stage loader doesn't work as it can't find the required configuration files. You can try booting the machine up over a network or using a livedvd to try to repair grub using boot-repair. I suspect that you have already tried that, and it didn't work. Note that /etc/ also has the nvram data, so it is extremely important. Check that once. – Tamoghna Chowdhury Jun 16 '15 at 12:35
  • Thanks for that explanation, @TamoghnaChowdhury . I didn't mean that MBR or EFS will be relocated. The kernel will reside in boot device, in this case disk, for the boot loader to load to memory. So if the boot loader is grub, grub.cfg will hold the information about list of kernels to load and location of each kernel. By default the location of kernel will be under /boot so that boot loader can load that compressed kernel to memory and uncompress it.(again all these are arch dependent). – ukesh upendran Jun 16 '15 at 15:22
  • 2
    Can this be achieved with a user that doesn't have root? – monkeymatrix Jun 26 '15 at 14:18
  • @monkeymatrix In line of principle, no, it shouldn't. Just try it yourself: try to move out as a regular user any command from, say, /bin,, and you will be covered by Permisison denied. This applies to both booting ile, and regular commands. – MariusMatutiae Jun 26 '15 at 21:28

Since the machine stopped responding before the reboot happened, I'm not convinced moving the kernel image was the cause. Users are not supposed to have access to this file in the first place, otherwise the whole system security is compromised: just create a patched kernel image which gives you root access when you call wait4() and reboot.

Could this special user of yours give himself permanent access to some of the system configuration files in /etc? Removing a couple of those can easily break the boot process, and also damage the running system.

It's hard to give a more precise answer without details. For example, when you say "SSH stopped working", do you mean the active session was forcibly closed, or did the user log out and try to log back in? And when you say "The machine won't come back", what happens exactly? Any messages on the console, or nothing at all?

Ultimately, you'll need to inspect the filesystem to see which files are missing. There can be no official source which says how exactly a particular user shot himself in the foot.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.