2

My MYSQL server has been stopping recently about once a week. It requires a reboot to get is working again. But once its rebooted, it will work again for another week or two.

The error in question is number 30. According to url in the error log MYSQL doesn't give my any more info than just saying my database drive is now in ready-only mode.

My database is on a mounted drive using UUID="c3c2527e-..." /media/database ext4 user,auto 0 2. And that mounted drive is accessible via ssh when this error occurs.

To my knowledge I haven't updated my OS or MYSQL recently or made changes on the server. I do remember having to adjust apparmor when I first added the drive containing MYSQL. However that was was done over a year ago.

So my question is: Does this issue have to do with my hard drive going bad, apparmor hiccuping, or something else?

OS: 12.04.2 LTS (GNU/Linux 3.5.0-23-generic x86_64)

MYSQL: 5.5.31-0ubuntu0.12.04.1 (Ubuntu)

MYSQL Drive: /dev/sdb1 2.0T 709M 2.0T 1% /media/database

The whole error is:

140301  8:00:29 [Note] Plugin 'FEDERATED' is disabled.
140301  8:00:29 InnoDB: The InnoDB memory heap is disabled
140301  8:00:29 InnoDB: Mutexes and rw_locks use GCC atomic builtins
140301  8:00:29 InnoDB: Compressed tables use zlib 1.2.3.4
140301  8:00:29 InnoDB: Initializing buffer pool, size = 128.0M
140301  8:00:29 InnoDB: Completed initialization of buffer pool
140301  8:00:29  InnoDB: Operating system error number 30 in a file operation.
InnoDB: Error number 30 means 'Read-only file system'.
InnoDB: Some operating system error numbers are described at
InnoDB: http://dev.mysql.com/doc/refman/5.5/en/operating-system-error-codes.html
InnoDB: File name ./ibdata1
InnoDB: File operation call: 'open'.
InnoDB: Cannot continue operation.
Improve this question

1 Answer 1

Reset to default
1

You definitely have disk-related problems. It could be the disk controller. Whatever the story the error is at the OS level:

sh-4.1# perror 30
OS error code  30:  Read-only file system

InnoDB will stop working because it requires lots of writes to take place, even if it is reading

Here is what InnoDB looks like

InnoDB Architecture

The active InnoDB components requiring writes are

  • Data dictionary (When creating, altering and dropping tables)
  • Double Write Buffer (Copy of data to reuse in crash recovery)
  • Insert Buffer (Mitigate changes in indexes from Buffer Pool)
  • Rollback Segments/Undo Logs (Transaction Isolation and MVCC)
  • Log Files (Receiving Changes From Log Buffer via Fuzzy Checkpointing)
  • .ibd files (Individual Tablespace files)

Once a file system turns read-only, all these moving parts stop.

You should back the data

  • I would do a mysqldump of all the data
  • Shutdown mysql and get an LVM snapshot (if applicable)
  • Shutdown mysql and copy the datadir

Once you have a backup, get right to the disk maintenance. Check the RAID, the disk controller, and the disk itself.

Here are some other ServerFault posts on read-only issues

Improve this answer
1
  • It ended up being my hard drive. I decided after this issue to move every thing into the cloud so I wont have to deal with 1 year old hard drives going bad :+1
    – Jon
    May 3, 2014 at 5:24

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .