1

I want to call a shell script regularly, and in it I need to process the files present in a directory tree that are new or that have been modified since the previous call of the script.

My 1st thought is to call the script from cron and use find -type f -mmin ... with the same time interval.

But I'd prefer if the script was self sufficient and allowed for manual or irregular calls. I can't just do a plain diff because I also need the list of modified files.

Maybe store the date of the call somewhere ? Any ideas ?

Improve this question

1 Answer 1

Reset to default
2

You could use the -newer argument of find :

-newer reference
          Time of the last data modification of the current file is
          more recent than that of the last data modification of the
          reference file.

Your script could then end with a command such as echo >timestamp.
To list the changed files do find -newer timestamp.

Improve this answer
7
  • Warning: there's a time window between processing a file and running echo >timestamp. Updating the timestamp after processing many files will possibly extend this window to significant length for some file(s). A change during the window will be missed by the next find -newer. Updating the timestamp before processing (but obviously after getting a list of files from find -newer) leaves a window for changes that may cause a processed file seem not processed, I guess. I think in general it's hard to do this flawlessly if files may change when the script runs.
    – Kamil Maciorowski
    May 24 at 16:32
  • @KamilMaciorowski: What kind of time window? Disk writes in the memory cache do not count here, since they were already time-stamped.
    – harrymc
    May 24 at 16:35
  • Imagine the script needs one second per new (or modified) file. There are 10 new files to process. After processing the first file there will be 9 seconds until echo >timestamp runs. This is the time window when a third party program may change the first file and the change will be "invisible" to the next find -newer timestamp. It will be "invisible" to every future -find newer timestamp until yet another change happens outside unfortunate windows.
    – Kamil Maciorowski
    May 24 at 16:49
  • @KamilMaciorowski: Third-party modifications can certainly falsify everything with racing conditions. As long as this script is the only actor, the poster is fine.
    – harrymc
    May 24 at 16:54
  • The script is "to process the files present in a directory tree that are new or that have been modified since the previous call of the script" [emphasis mine]. How can the script be the only actor?
    – Kamil Maciorowski
    May 24 at 16:56

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .