33 reputation
5
bio website tverniquet.com
location Brisbane, Australia
age 32
visits member for 2 years, 2 months
seen 3 hours ago

Jan
10
comment Git Clone imreachable. Timeout error
I tried them both and they work for me. Must be something wrong with your connection. Can you resolve and ping the servers?
Oct
29
awarded  Scholar
Oct
29
awarded  Supporter
Oct
29
accepted Thunderbird excessive disk
Oct
29
comment Thunderbird excessive disk
That'd be it. Thanks!
Oct
29
asked Thunderbird excessive disk
Oct
25
awarded  Teacher
Oct
23
answered Vim-like history in bash?
Oct
20
comment syncing large files FreeBSD
I think you might have misunderstood my purpose for the polling - it was just to show that nothing else can open/read the file while it is being sync'd. What I want is to allow other processes (maybe not controlled by me) to be able to open up and read the data while it is being sync'd. I can do that now with msync using MS_ASYNC. But thank you - I do make use of kqueue/kevent for monitoring file changes etc in other areas.
Oct
19
revised syncing large files FreeBSD
deleted 2552 characters in body
Oct
18
revised OpenSSH does not accept public key?
added 44 characters in body
Oct
18
answered OpenSSH does not accept public key?
Oct
17
revised syncing large files FreeBSD
added 77 characters in body
Oct
17
revised syncing large files FreeBSD
added 277 characters in body
Oct
17
revised syncing large files FreeBSD
added 995 characters in body
Oct
17
comment syncing large files FreeBSD
Update - I've discovered the problem is actually only when opening (or mmaping) or the file while it is being synced - it doesn't affect the ability to read the mapping if the file is already opened and mapped. However this is still a problem for me. (Also hangs if calling 'read'/'pread'. I'm guessing any file-related system calls.
Oct
17
comment syncing large files FreeBSD
The problem is the os exclusive lock on the file when syncing. To show that disk io shouldn't be a problem - imagine having 2 identical mmaped files. Write the data to both of them. Have all reads happening on the one file. Sync the other file to disk - users can still access the non-syncing one as it isn't being synced. Once the first is synced, swap the pointers around and remove the temporary file. Application runs at memory speed regardless of the sync. Downside - complexity and using 2x memory!
Oct
17
comment syncing large files FreeBSD
The periodic flushes can be avoided by mapping with NO_SYNC. The problem is that at some stage the data will need to be sync'd to disk. When that happens, it does it all in one go and locks the file for the duration of the sync. In the case of large files this can easily be a few minutes. If I purposly sync very often so that the file isn't locked for as long - then the application is limited by disk speed even when there is lots of memory available.
Oct
17
comment syncing large files FreeBSD
Thanks for the links. I think my question is better asked with my second edit - I am quite happy for the whole file to be cached, that is essentially what I want - the problem is that if it is all cached then at some stage it needs to be written to disk, and when this happens the file is locked out for the whole time. The file should still be available for reading at any time. Flushing after every few writes means that the application is limited by disk, even if there is lots of memory available.
Oct
17
revised syncing large files FreeBSD
added 923 characters in body