There are lot of these kinds of questions on superuser, so I'll try and keep this quick by telling you what it's not:
- There is no application involved here. Just a user and a filesystem.
- I can login as root and my non-root user has full sudo access.
- As far as I know, this problem only exists in one directory (though given enough time I think I can recreate this behavior in any directory -- more on that below).
- No strange filesystem customizations, umasks or odd user or directory permission schemes -- I own this system, and no other people interact with it.
This directory is a destination path for backup scripts written in bash. These scripts are not fancy. Rsync files from elsewhere to a working directory. Tar up all the files in the working directory and store the resulting archives in this destination path. Pretty simple stuff.
Let's say this is a fresh install, for this example. The script works fine for a while. But after some period of weeks, no files can be created in the destination path anymore, by any user, using any method. I've tried touch, tar, zip, etc. Nothing works. Tar and zip will fail with I/O errors, touch acts fine but no file gets created. In that destination path, and only that path. File creation/modification/deletion works fine everywhere else on the filesystem (if it's happening elsewhere, it's asymptomatic).
permissions on the dir look like this via an ls -al: drwxrwxr-x. -- owned by my non-root user and its group.
Lastly, if I reboot the system after this condition presents itself, I can create files in that path again for a short time. Eventually though, the problem returns.
When I said I can probably recreate this in another dir, it's because I've changed the destination path to something else trying to fix this a few different times already, and it eventually happens in every location I've tried.
- CentOS 2.6.32-431.3.1.el6.x86_64
- Filesystem is ext4
- 400GB free on / (lvm)
Any ideas or thoughts on things to try would be greatly appreciated. I'm happy to go into more detail if needed, but I wanted to keep this brief to start with.
Thanks for your time!
EDIT 10/20/14: Still no answers here. Can I get anyone any more info?
EDIT 10/8/14: Adding information as requested in comments:
$ ls -l
$ sudo vgs
VG #PV #LV #SN Attr VSize VFree
vg_omega1 1 2 0 wz--n- 464.24g 0
EDIT 10/7/14: Adding information as requested in comments:
/dev/mapper/vg_omega1-lv_root on / type ext4 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
tmpfs on /dev/shm type tmpfs (rw,rootcontext="system_u:object_r:tmpfs_t:s0")
/dev/sda1 on /boot type ext4 (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
EDIT 10/6/14: Further testing indicates I can apparently create sub-directories and symlinks just fine. This problem appears to be limited to regular files.