Zip64 Format: Crossing the Limits of File Sizes and Number of Files and Segments
Introduction
The Zip64 format is an extension to the standard zip format that practically removes limits in sizes and the number of files
inside of a zip archive.
The ZipArchive Library automatically uses the Zip64 extensions when the regular zip archive
limits are exceeded. The ZipArchive Library will not otherwise include the extensions in the archive, even if the
extensions are enabled (see the paragraph below). The extensions will be removed from the archive when due to modifications
they are not needed anymore. The library also automatically detects an archive in the Zip64 format.
To detect if an archive is in Zip64 format, request the central directory information with the CZipArchive::GetCentralDirInfo()
method and call the
CZipCentralDir::CInfo::IsZip64() method.
Enabling Zip64 Extensions in the ZipArchive Library
The Zip64 extensions are enabled by default in the ZipArchive Library. You should keep the extensions disabled
if you don't use them, otherwise the library will use 64-bit types and perform some
additional processing, which may slow down your application.
With CMake: Control Zip64 extensions using the ZIP_ENABLE_ZIP64 option:
- -DZIP_ENABLE_ZIP64=ON - Enable Zip64 extensions (default)
- -DZIP_ENABLE_ZIP64=OFF - Disable Zip64 extensions
Without CMake: Make sure _ZIP_ZIP64 is defined and set to 1
in the _features.h file (default is 1). Rebuild the ZipArchive Library and your
application if you modify this definition.
Limits Compared: the Standard Format Versus Zip64 Format
The maximum values allowed in each format are summarized below:
|
Number of Files Inside an Archive
|
65,535
|
2^64 - 1
|
|
Size of a File Inside an Archive [bytes]
|
4,294,967,295
|
2^64 - 1
|
|
Size of an Archive [bytes]
|
4,294,967,295
|
2^64 - 1
|
|
Number of Segments in a Segmented Archive
|
999 (spanning)
65,535 (splitting)
|
4,294,967,295 - 1
|
|
Central Directory Size [bytes]
|
4,294,967,295
|
2^64 - 1
|
Large Collections
The maximum number of items a collection can hold, limits the maximum number of files in an archive that the ZipArchive
Library can process. This limit can be determined by examining the
ZIP_ARRAY_SIZE_TYPE definition.
This is defined as:
size_t in the STL version
INT_PTR in the MFC version
The MFC version also depends on
size_t type when sorting.
These types are usually defined as unsigned
32-bit integer types. You may need to use a
64-bit compiler for a larger collections support.
Additional Considerations
- Under Windows, when your system utilizes large amount of memory while extensive file operations (especially on large files),
see Modification of Archives: Replacing, Renaming, Deleting and Changing Data for a possible solution.
- If you have problems reading very large files, it may be the case that they have incorrect format and they do not conform
to Zip64 specifications (it can happen with some archives under macOS). To deal with the problem, try using the CZipArchive::SetSpecialFlags() method with the CZipArchive::sfLargeNotZip64
flag.
- Under Windows, the built-in Compressed Folders shell extension (also known as Windows Zip Folders) can severely slow down
opening and browsing Zip64 archives that contain a large number of files. This is because the shell extension attempts to
parse the archive contents in the background whenever such a file is accessed in Windows Explorer. To avoid this performance
issue, disable the built-in Compressed Folders support by unregistering the zipfldr.dll shell extension
(run
regsvr32 /u zipfldr.dll from an elevated command prompt). You can re-enable it later by running
regsvr32 zipfldr.dll. This issue is most pronounced on older Windows versions (such as Windows XP), but can
also affect later versions of Windows when dealing with archives containing a very large number of files.