Why do some (but not all) zip-Files greater than ~3.7GB fail to extract

Question

I am dealing with a lot of zip-files (DCP Films for a Film-Festival) between 4GB and 40GB that are sent to me by various individuals that use various programs to compress the folder that they are sending. There are usually 5-10 files per folder with 1-5 files exceeding 3.7GB.

Some of this archives can be extracted without any problems but for some I get the following errors:

trying to extract with 7z:

7z e ORE_SHR-1-30_F_XX-XX_AT-XX_20_2K_XX_20190312_FF_SMPTE_OV.zip 

7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,64 bits,8 CPUs Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz (506E3),ASM,AES-NI)

Scanning the drive for archives:
1 file, 8033862438 bytes (7662 MiB)

Extracting archive: ORE_SHR-1-30_F_XX-XX_AT-XX_20_2K_XX_20190312_FF_SMPTE_OV.zip
ERROR: ORE_SHR-1-30_F_XX-XX_AT-XX_20_2K_XX_20190312_FF_SMPTE_OV.zip
Can not open the file as archive


Can't open as archive: 1
Files: 0
Size:       0
Compressed: 0

trying to extract with unzip:

unzip ORE_SHR-1-30_F_XX-XX_AT-XX_20_2K_XX_20190312_FF_SMPTE_OV.zip 
Archive:  ORE_SHR-1-30_F_XX-XX_AT-XX_20_2K_XX_20190312_FF_SMPTE_OV.zip
warning [ORE_SHR-1-30_F_XX-XX_AT-XX_20_2K_XX_20190312_FF_SMPTE_OV.zip]:  4294967296 extra bytes at beginning or within zipfile
  (attempting to process anyway)
file #1:  bad zipfile offset (local header sig):  4294967296
  (attempting to re-compensate)
   creating: ORE_SHR-1-30_F_XX-XX_AT-XX_20_2K_XX_20190312_FF_SMPTE_OV/
  inflating: ORE_SHR-1-30_F_XX-XX_AT-XX_20_2K_XX_20190312_FF_SMPTE_OV/ASSETMAP.xml  
  inflating: ORE_SHR-1-30_F_XX-XX_AT-XX_20_2K_XX_20190312_FF_SMPTE_OV/ORE_SHR-1-30_F_XX-XX_AT-XX_20_2K_XX_20190312_FF_SMPTE_OV_cpl.xml  
  inflating: ORE_SHR-1-30_F_XX-XX_AT-XX_20_2K_XX_20190312_FF_SMPTE_OV/ORE_SHR-1-30_F_XX-XX_AT-XX_20_2K_XX_20190312_FF_SMPTE_OV_pkl.xml  
  inflating: ORE_SHR-1-30_F_XX-XX_AT-XX_20_2K_XX_20190312_FF_SMPTE_OV/ORE_SHR-1-30_F_XX-XX_AT-XX_20_2K_XX_20190312_FF_SMPTE_OV_Reel_1_j2c.mxf  
  error:  invalid compressed data to inflate
file #6:  bad zipfile offset (local header sig):  3671207290
  (attempting to re-compensate)
  inflating: ORE_SHR-1-30_F_XX-XX_AT-XX_20_2K_XX_20190312_FF_SMPTE_OV/ORE_SHR-1-30_F_XX-XX_AT-XX_20_2K_XX_20190312_FF_SMPTE_OV_Reel_1_pcm.mxf  
  inflating: ORE_SHR-1-30_F_XX-XX_AT-XX_20_2K_XX_20190312_FF_SMPTE_OV/VOLINDEX.xml 

trying to rescue (on MacOS):

zip -FF ORE_SHR-1-30_F_XX-XX_AT-XX_20_2K_XX_20190312_FF_SMPTE_OV.zip --out ore.zip
Fix archive (-FF) - salvage what can
 Found end record (EOCDR) - says expect single disk archive
Scanning for entries...
 copying: ORE_SHR-1-30_F_XX-XX_AT-XX_20_2K_XX_20190312_FF_SMPTE_OV/  (0 bytes)
 copying: ORE_SHR-1-30_F_XX-XX_AT-XX_20_2K_XX_20190312_FF_SMPTE_OV/ASSETMAP.xml  (588 bytes)
 copying: ORE_SHR-1-30_F_XX-XX_AT-XX_20_2K_XX_20190312_FF_SMPTE_OV/ORE_SHR-1-30_F_XX-XX_AT-XX_20_2K_XX_20190312_FF_SMPTE_OV_cpl.xml  (631 bytes)
 copying: ORE_SHR-1-30_F_XX-XX_AT-XX_20_2K_XX_20190312_FF_SMPTE_OV/ORE_SHR-1-30_F_XX-XX_AT-XX_20_2K_XX_20190312_FF_SMPTE_OV_pkl.xml  (613 bytes)
 copying: ORE_SHR-1-30_F_XX-XX_AT-XX_20_2K_XX_20190312_FF_SMPTE_OV/ORE_SHR-1-30_F_XX-XX_AT-XX_20_2K_XX_20190312_FF_SMPTE_OV_Reel_1_j2c.mxf 
        zip warning: no end of stream entry found: ORE_SHR-1-30_F_XX-XX_AT-XX_20_2K_XX_20190312_FF_SMPTE_OV/ORE_SHR-1-30_F_XX-XX_AT-XX_20_2K_XX_20190312_FF_SMPTE_OV_Reel_1_j2c.mxf
        zip warning: rewinding and scanning for later entries

after the rescue (MacOS):

unzip ore.zip 
Archive:  ore.zip
warning [ore.zip]:  4294967296 extra bytes at beginning or within zipfile
  (attempting to process anyway)
file #1:  bad zipfile offset (local header sig):  4294967296
  (attempting to re-compensate)
   creating: ORE_SHR-1-30_F_XX-XX_AT-XX_20_2K_XX_20190312_FF_SMPTE_OV/
  inflating: ORE_SHR-1-30_F_XX-XX_AT-XX_20_2K_XX_20190312_FF_SMPTE_OV/ASSETMAP.xml  
  inflating: ORE_SHR-1-30_F_XX-XX_AT-XX_20_2K_XX_20190312_FF_SMPTE_OV/ORE_SHR-1-30_F_XX-XX_AT-XX_20_2K_XX_20190312_FF_SMPTE_OV_cpl.xml  
  inflating: ORE_SHR-1-30_F_XX-XX_AT-XX_20_2K_XX_20190312_FF_SMPTE_OV/ORE_SHR-1-30_F_XX-XX_AT-XX_20_2K_XX_20190312_FF_SMPTE_OV_pkl.xml  
file #5:  bad zipfile offset (local header sig):  2432
  (attempting to re-compensate)
file #5:  bad zipfile offset (local header sig):  2432
  inflating: ORE_SHR-1-30_F_XX-XX_AT-XX_20_2K_XX_20190312_FF_SMPTE_OV/ORE_SHR-1-30_F_XX-XX_AT-XX_20_2K_XX_20190312_FF_SMPTE_OV_Reel_1_pcm.mxf  
  inflating: ORE_SHR-1-30_F_XX-XX_AT-XX_20_2K_XX_20190312_FF_SMPTE_OV/VOLINDEX.xml 

So far I only managed to extract the archive using 7z on Windows10.

I do get...

Headers-Error
Warning: 32-Bit overflow in headers

...but still, using 7z on Windows10 is the only way I can extract the archive - despite the Header-Error/Warning

I am using a Linux computer but I also tried using "unzip" on a MacOS and I want to be able to extract the archive there instead of walking over to the Windows machine every time this happens.

• Why does the same file work on 7z Windows10 but not with 7z on MacOS or 7z on Linux?
• How can I be able to extract the archive on a Linux machine?

Additional information:

All are 64-bit environments. On the Linux System I use ext4 and on the MacOS I don't know for now (no access for the upcoming 2 weeks) but guess it's the MacOS-native filesystem if there is such a thing.

A colleague of mine was able to extract several of the archives that didn't work on my machine on MacOS by simple 'right click'->'extract here' - but they have no idea what program they are using - just that they are on an Apple computer.

Answer 1

Something else to try, if the ZIP file is > 4 GB and was created by OneDrive or Windows, is a utility designed to fix-up the headers:
https://github.com/pmqs/Fix-OneDrive-Zip

Apparently, there's some disagreement about how to handle the "file count" header for Zip64 files. Microsoft products expect that the file count header should be 0 while other systems expect it to be 1. Newer versions of 7-zip for Windows (versions 19+) handle the issue fine but Linux only has an older POSIX port, p7zip, that is version 16 from 2016, and it does not have a way to handle this issue. The utility linked to above is a Perl-based script that changes the header information so that the file count changes from 0 to 1, so utilities can handle the files.

If you're curious, someone did a deep-dive on the issue here:
https://www.bitsgalore.org/2020/03/11/does-microsoft-onedrive-export-large-ZIP-files-that-are-corrupt