7

My web server (Linux, Debian Wheezy) has suddenly gone read-only, MySQL have crashed (but Apache didn't) - giving errors when pressing tab to complete file name as:

bash: cannot create temp file for here-document: Read-only file system

Also, when I restart Apache it won't turn on anymore.

What I tried to do : I've replaced the hard disk , moved web files again ; error repeated after 1 week.

Logs shows nothing, one notice that I've seen many failed attempts to access the server through ssh.

df output:

Filesystem                                             1K-blocks    Used Available Use% Mounted on
rootfs                                                 249773956 2328024 234758164   1% /
udev                                                       10240       0     10240   0% /dev
tmpfs                                                     406336     200    406136   1% /run
/dev/disk/by-uuid/e45e30eb-efa4-4cd9-aaf9-c6cbe46aa41c 249773956 2328024 234758164   1% /
tmpfs                                                       5120       0      5120   0% /run/lock
tmpfs                                                    2489760       0   2489760   0% /run/shm
/dev/sdb1                                              249773956 2303784 234782404   1% /mnt/sdb1

mount -n output:

sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
udev on /dev type devtmpfs (rw,relatime,size=10240k,nr_inodes=506431,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,noexec,relatime,size=406336k,mode=755)
/dev/disk/by-uuid/e45e30eb-efa4-4cd9-aaf9-c6cbe46aa41c on / type ext4 (ro,relatime,errors=remount-ro,user_xattr,barrier=1,data=ordered)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k)
tmpfs on /run/shm type tmpfs (rw,nosuid,nodev,noexec,relatime,size=2489760k)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
/dev/sdb1 on /mnt/sdb1 type ext4 (rw,relatime,user_xattr,barrier=1,data=ordered)
  • 2
    these many ssh attempts that you report, are they by you (or some other expected person)? If not, is it right that a third party can get ssh access to this machine? Any evidence that they have succeeded in logging in? If it isn't mindless (or worse mindful) vandalism by a third party, it seems there must be an error occurring on your / mount, then it remounts read-only. I would boot the system from CD/USB (e.g. systemrescuecd) and then do e2fsck -fp /dev/disk/by-uuid/e45e30eb-efa4-4cd9-aaf9-c6cbe46aa41c – gogoud May 3 '15 at 11:26
  • @gogoud No there are no successful attempts except for mine, and fsck won't be helpful since it's a new harddisk even – Ahmed Yusuf May 3 '15 at 15:25
10

Your root file system is mounted read-only. This likely happened on a reboot. There are a few options:

  • Configure the system to fix errors during reboot. On Ubuntu this is controlled by the FSCKFIX option in the file /etc/default/rcS.
  • Reboot in recovery mode and run fsck -f /dev/disk/by-uuid/e45e30eb-efa4-4cd9-aaf9-c6cbe46aa41c and reboot again.
  • Boot from a recovery disk and run fsck as shown above.

All options will require your system to be down for a bit. However, it appears it is effectively down now.

It may be possible to run fsck without rebooting and remount the system rw. I would run a test fsck without enabling fixing problems to see how broken the file system

| improve this answer | |
  • Alternatively, it can happen without a reboot if something breaks one of the kernel modules supporting the file-system journalling. Rebooting is a workaround in that case. – Thomas Dickey May 4 '15 at 21:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.