I've got a bit of a problem here...
I've got a render-farm full of machines that are writing to a central disk, and, sometimes, after it's written an image sequence (DPX files, 12Mb per frame), about a minute after each file is written, it suddenly drops from 12Mb to 4kb.
I'm pretty certain that there is nothing running that would be doing this. .
The application that is creating the files is Shake.
This problem has cropped up before, very occasionally. I've certainly never figured out what actually causes it.
The central disk is a 10Tb raid, which the machines connect to via NFS.
df -h gives me the following for it:
clarke:/Volumes/projects 8.9Ti 7.7Ti 1.1Ti 88% /mount/projects
So it's not like it's running out of space (I was at 700Gb free when it started having the problem tonight, and I've freed up a load more just in case that was something to do with it...
Any help would be much appreciated! I really want to figure out what's causing this, as it's certainly causing me a lot of pain...
If you want more specs of the server:
- XServe running OSX Server 10.5.8
- Promise Raid VTrak E610f
Also: The data certainly was there, for a couple of minutes. I set off a process to copy it to another machine, and that other machine is seeing the images just fine.
Is there some kind of tool that will track every single file creation or modification in a folder (or whole file system) and tell me exactly what's going on? I want to find out if there is some kind of rogue process doing something naughty (it works most of the time, so I suspect not) or something lower down than that...