I am writing a bash script to periodically download a setup file using cURL. The problem is the following, the file is around 110 MB and always have the same filename.
I can download the file, check its hash (md5, sha1) and compare it to a previously downloaded file:
- if the hash is equal, the file is the same and has not been updated: in this case, I have downloaded it for no reason.
- if the hash is different, the file on the webserver is a new version: in this case, I use this file and update to the new version.
I would like to run this script every half-an-hour, and downloading 110MB for nothing (same file version) bothers me. Is there a way to download the first 500KB using curl in command line ?
My current workaround is to compare the filesize from the HTTP header but I still would like to have an answer on the question above and know if this is feasible. Many thanks.
curl -sI 'http://domain.tld/uri/updater/getLatest.etcenter?c=var&asi=allright' -H 'Host: domain.tld' -H 'User-Agent: Mozilla/5.0 (Windows NT 6.3; Trident/7.0; rv:11.0) like Gecko' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8' -H 'Accept-Language: en-US,en;q=0.5' -H 'Accept-Encoding: gzip, deflate, br' -H 'DNT: 1' -H 'Connection: keep-alive' | grep 'Content-Length:'