1

I am trying to unzip/gunzip a file from the terminal and it works absolutely fine.

When I put the same in a sh file and call it from terminal it works fine too!

 echo "started"
for file in $filepath/*; do
echo "filename ",$file
if [[ $file == *.zip ]]; then
        #unzip -o $path$file -d $path
      unzip $file
      #jar xvf $path$file -d $path
      #tar -xvf {$path$file} -C $path
      #rm $file
elif [[ $file == *.gz ]]; then
        #unzip -o $path$file -d $path
      gunzip $file
      #jar xvf $path$file -d $path
      #tar -xvf {$path$file} -C $path
  else
        echo "file name is not csv or zip ", $file
fi
done

But when I call this from another shell script(which is part of inotifywait), it gives the following errors. For gz

gzip: /usr/apps/{somepath}/6206553653146575.csv.gz: unexpected end of file

for zip

Archive:  /usr/apps/{somepath}/3020393677273733265.csv.zip
End-of-central-directory signature not found.  Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive.  In the
latter case the central directory and zipfile comment will be found on
the last disk(s) of this archive.
unzip:  cannot find zipfile directory in one of 
/usr/apps/{somepath}/3020393677273733265.csv.zip or
/usr/apps/{somepath}/3020393677273733265.csv.zip.zip, and cannot find 
/usr/apps/{somepath}/3020393677273733265.csv.zip.ZIP, period.

What did I try already:

Changed execution to #!/bin/bash

Tried to add .zip/.gz at the end of the file name.

Thanks for the help. Appreciate that.

  • Maybe its a problem withthe shell being used. What happens if you add "#! /bin/bash" as the dfirst line of the script to force it to use the bash interpreter rathere then possibly something else? – davidgo Jun 3 '19 at 19:53
  • It gives same error even when i add it – Sureshkumar Neelamegam Jun 3 '19 at 20:22
  • Clutching at straws here but could it be related to selinux? (Does /var/log/audit.log give.any clues / what happens if you do setenforce 0 ?) – davidgo Jun 3 '19 at 20:31
  • setenforce 0 says command not found – Sureshkumar Neelamegam Jun 3 '19 at 21:32
1

You mentioned inotifywait. I guess you're trying to extract files as soon as they appear. It looks to me in some cases unzip or gunzip is run when the file it is told to extract is incomplete yet.

Possible causes:

  • You catch wrong events (e.g. open, create or modify) instead of these that would suggest the file is now complete (moved_to, close_write).
  • Your script gets no information about the file that triggered it. The script uses for and tries to extract all files with suitable names. Even if you properly catch a file when it's complete, the script will also try to extract other files (if any), including incomplete ones.

What to improve:

  • Catch right events.
  • Rebuild the logic so the script knows which file it's supposed to process. Let each file trigger its own instance of the script, so any single instance of the script processes just one file. It may do little (e.g. it may exit because the name of the file is not what you want) but don't let it do too much (process other files).
  • Implement some way(s) to confirm the given file is complete. Few ideas:
    • checking if the file is no longer open (fuser, lsof);
    • checking if the file hasn't grown for several seconds;
    • silently testing the file integrity (gunzip -t, unzip -t).
| improve this answer | |
  • Wow. It was a wonderful explanation and perfect catch of my issue. I changed create to close_write and now everything is good. Thanks a ton. – Sureshkumar Neelamegam Jun 4 '19 at 21:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.