Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

My question is same as in Create multiple zip files that are not dependent on each other? but my idea is to add files to zip until it is of required size and continue with a new zip file for others and so on.

Any idea how to do this in a bas script? The script here, Creating a BASH Script - Zipping? seems to of much help. Needs some customizations though.

share|improve this question

2 Answers 2

up vote 3 down vote accepted

A basic script would just check the zip file's size and switch zip files accordingly. Something like this:


#!/usr/bin/env bash

## This counter is used to change the zip file name
COUNTER=0;
## The maximum size allowed for the zip file
MAXSIZE=1024;
## The first zip file name
ZIPFILE=myzip"$COUNTER".zip;
## Exit if the zip file exists already
if [ -f $ZIPFILE ]; then
    echo $ZIPFILE exists, exiting...
    exit;
fi
## This will hold the zip file's size, initialize to 0
SIZE=0;

## Go through each of the arguments given in the command line
for var in "$@"; do
    ## If the zip file's current size is greater than or
    ## equal to $MAXSIZE, move to the next zip file
    if [[ $SIZE -ge $MAXSIZE ]]; then
    let COUNTER++;
    ZIPFILE=myzip"$COUNTER".zip;
    fi
    ## Add file to the appropriate zip file
    zip -q $ZIPFILE $var;
    ## update the $SIZE
    SIZE=`stat -c '%s' $ZIPFILE`;
done

CAVEATS:

  • The script expects files, not directories, if you want it to run on directories add -r to the zip command. However, it will not check the file size until each directory has been compressed.
  • The zip file's size is checked after each compression. This means that you will get files that are larger then your limit. This because it is hard to guess what a file's compressed size will be so I can't check before adding it to the archive.
share|improve this answer
    
Well, that was cool. I just hacked a script inspired from this answer. Will update my question with the new script. –  Aczire Jul 4 '13 at 14:21
1  
@Aczire please don't. Post your script as an answer instead. –  terdon Jul 4 '13 at 14:23

EDIT: Any idea about optimizing the below script for huge file counts??

#!/usr/bin/env bash

format="%b-%m.zip"    # the default format
# see http://unixhelp.ed.ac.uk/CGI/man-cgi?date
dateformat="%Y-%b-%d" # the default date format
zipsize=2048; #256901120 #245MB
zipfile="" # the zip file name to use.

if [[ $# -lt 1 ]]; then
  echo "Usage: $0 directory [format] [zip size] [dateformat]"
  echo
  echo "Where format can include the following variables:"
  echo " %f file"
  echo " %b file with no extention"
  echo " %e file extention"
  echo " %c file created date (may be 1-Jan-1970 UTC if unknown)"
  echo " %m file modified date"
  echo " %t current date"
  echo
  echo "And dateformat uses the same format specifiers as the date command."
  echo
  echo " Example: $0 zip-1 %f-%m.zip %Y"
  echo " Example: $0 zip-1 %f-%m.zip %Y-%b"
  echo
  echo "And zipsize is the maximum zip size allowed per zip file in bytes."
  echo
  echo " Example: $0 zip-1 256901120 %f-%m.zip %Y"
  echo " Example: $0 zip-1 256901120 %f-%m.zip %Y-%b"
  exit 1
fi

if [[ $# -ge 2 ]]; then
  zipsize="$2"
fi

if [[ $# -ge 3 ]]; then
  format="$3"
fi

if [[ $# -ge 4 ]]; then
  dateformat="$4"
fi


dozip()
{
  filepath=$1
  parent_path=$(dirname "$filepath")
  file=$(basename "$filepath")
  ext=${file##*.}
  body=${file%.*}

  date=$(date +$dateformat)
  mdate=$(date --date="@$(stat -c %Y "$filepath")" +$dateformat)
  cdate=$(date --date="@$(stat -c %W "$filepath")" +$dateformat)

    if [ -z "$zipfile" ]; then
    zipfile=$(echo $format | sed -e "s/%f/$file/g" -e "s/%b/$body/g" -e "s/%e/$ext/g" -e "s/%t/$date/g" -e "s/%m/$mdate/g" -e "s/%c/$cdate/g")
    else
    size=`stat -c '%s' $zipfile`
    if [[ $size -ge $zipsize ]]; then
      zipfile=$(echo $format | sed -e "s/%f/$file/g" -e "s/%b/$body/g" -e "s/%e/$ext/g" -e "s/%t/$date/g" -e "s/%m/$mdate/g" -e "s/%c/$cdate/g")
    fi
    fi

  pushd "$parent_path" > /dev/null
  zip "$zipfile" "$file" > /dev/null
  popd > /dev/null
}

#files=$(find $1 -type f)
files=$(find $1 -type f | sed -e '/zip$/d') # exclude zip files
IFS=$'\n';
for file in $files; do
  dozip $file
done
share|improve this answer
    
The best thing to do for huge file counts is probably adding everything to a single zip and then spliting. –  terdon Jul 4 '13 at 14:48
    
@teredon if that was feasible in this case, my life would be much easier :) Sadly, each zip file goes to different users. I was trying to add a checkpoint count like 1000, add files till the count, check the size etc.. But without much performance gain. –  Aczire Jul 5 '13 at 5:53

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.