std/zip
import std/zip to use it. Source: src/std/zip.iyi at v0.18.0.
ZipReader reads an archive the way the format means it to be read: from its end. The end of central directory record is found by scanning back over the comment that may follow it (APPNOTE 4.3.16; at most 65,535 bytes, so the scan is bounded), and the central directory it points at is the list of entries - the local headers in front of each entry's data are read only when that entry is opened, and only to find where its data begins. A record whose comment length reaches exactly to the end of the archive is taken over one that does not, so a comment that happens to hold the record's four signature bytes does not fool the scan; failing that, the last record that fits, as Go's archive/zip reads an archive with bytes after it. Each entry carries its name, both sizes, its CRC-32, its method, its modification time (MS-DOS date and time, two-second steps, no zone - answered as a Time whose UTC wall clock is what was written, as Go answers it), and its flags. A name is UTF-8 when general purpose bit 11 says so (APPNOTE appendix D), else code page 437, which is what the format says and what Python's zipfile decodes; Info-ZIP's Unicode Path extra field (0x7075), a UTF-8 name carried beside one written in another code page, is taken when its CRC-32 matches the name it stands for, and passed over when it is stale, as Python and unzip take it. The ZIP64 extended information field (0x0001) and the ZIP64 end records (4.3.14, 4.3.15) carry the sizes, counts and offsets that do not fit in 16 or 32 bits.
An entry is read as a stream: ZipReader#open answers a ZipEntryReader whose read gives the content a piece at a time through its method's decoder - Inflater for deflate (8), Bzip2Reader for bzip2 (12), XzReader for LZMA (14) and xz (95), ZstdReader for Zstandard (93) - or straight from the archive for stored (0), and checks the CRC-32 and the size against the central directory before it hands out the last piece. An LZMA entry carries a header of its own before its data (APPNOTE 5.8.8): the LZMA SDK's version, which is passed over, and the five bytes of properties, which are given to std/xz with the size as the 13-byte header of a .lzma file, the size "unknown" when general purpose bit 1 says an end marker ends the data (5.8.9), as Python's zipfile writes it. An entry written with a data descriptor (bit 3) has its sizes in the central directory, which is what is read by; the descriptor after the data is read only for the CRC-32 it repeats, which must agree, as Go checks it. ZipReader.open takes the archive as a String or as an IyiIO that can seek - a file - which is read only where it is needed, never whole.
Bytes from outside are not trusted, and what is wrong with them is a ZipError, a value: no end record; a central directory that lies past where the archive ends; a split (multi-disk) archive; an encrypted entry (bit 0) or one compressed with a method this module does not read (deflate64, PPMd, implode and the rest), named; a decoder's own refusal, its sentence after the entry's name; a local header that is missing or names another file; entries whose data overlaps the next entry's header, which is how the overlapping-file zip bomb (Fifield, 2019) gets a kernel inflated once per entry - every entry's span is checked against the next before anything is read, and Python's zipfile has refused them since CVE-2024-0450; a total uncompressed size past the limit the caller gives open; and content that does not match its CRC-32 or size, or inflates past the size the directory declares, which is refused the moment it does, not after the memory is spent. ZipReader#extract(dir) writes the entries under a directory and refuses, before it writes anything, a name that is absolute, climbs out with .., or uses \ or a drive (:) - the "zip slip" (Snyk, 2018) - and so never writes outside dir.
ZipWriter writes to any IyiIO, and needs no seeking: an entry from a String is compressed first - deflated, stored, bzip2 or Zstandard, with the version needed to extract each asks for, 4.6 and 6.3 - and its header written knowing the sizes and CRC-32; one streamed from an IyiIO is written with a data descriptor (bit 3) after its data, as Python's zipfile writes to a stream it cannot seek, deflated or stored, as std/bzip2 and std/zstd compress a whole input and not a stream. Directories are entries named with a trailing /. A name that is not ASCII is written as UTF-8 with bit 11 set, as Python writes it. ZIP64 is written where a value needs it: an offset or a size past 0xFFFFFFFF gets the extended field in the central directory, and more than 65,535 entries or a directory past 4 GiB gets the ZIP64 end records, the 32-bit fields left at their maximum where they overflow. ZipWriter.new(io, zip64: true) writes every entry and the end that way, which is how ZIP64 is exercised without writing four gigabytes. A streamed entry that turns out past 4 GiB without it has 8-byte sizes in its descriptor and the extended field in the central directory, which every reader that reads by the directory reads, as Go's archive/zip writes one. finish writes the central directory and the end; the stream is flushed, not closed.
Measured on the machine this was written on, a release build: 50 MB of text written deflated as one entry in 145 ms (to 3.2 MB) and read back, CRC-32 checked, in 65 to 70 ms; Python's zipfile, over C zlib, reads the same archive in 35 ms and writes it at level 1 in 56 ms. 50 MB of numbered lines Python 3.14 wrote as one entry are read in 300 ms from bzip2 (Python 297), 85 ms from LZMA (57) and 64 ms from Zstandard (32); written, they take 1.40 s as bzip2 (an archive of 1,690,084 bytes; Python 2.93 s, 1,690,195) and 72 ms as Zstandard (455,939 bytes; Python at its default level 29 ms, 667,026).
Left out: encryption (traditional and AES), the methods other than these six, writing LZMA and xz (std/xz only reads), split archives, the extended timestamp fields, Unix permissions on extraction, symbolic links (an entry is written as a file holding its bytes), and appending to an existing archive. Archives with bytes in front of them (self-extracting) are read only if their offsets count from the start of the file.
ZipEntryclass
One entry of the central directory.
pub class ZipEntry
def comment : String
def compressed_size : Int64
def crc32 : UInt32
# Bit 3: the CRC-32 and sizes follow the data, in a data descriptor.
def descriptor? : Bool
def directory? : Bool
# Bit 0: the data is encrypted, which this module does not read.
def encrypted? : Bool
# The general purpose bit flags.
def flags : Int32
def initialize(name : String, raw_name : String, size : Int64, compressed_size : Int64, crc32 : UInt32, method : Int32, flags : Int32, modified : Time, offset : Int64, comment : String, zip64 : Bool)
# The method number: 0 stored, 8 deflate, 12 bzip2, 14 LZMA, 93
# Zstandard, 95 xz; any other is listed and refused when the entry is
# opened.
def method : Int32
# The method's name, for a sentence about it.
def method_name : String
def modified : Time
# The name as the archive's directory holds it, decoded to UTF-8.
def name : String
# Where the entry's local header is, from the start of the archive.
def offset : Int64
# The name's bytes as written, before decoding.
def raw_name : String
# The uncompressed size.
def size : Int64
# Bit 11: the name is UTF-8 rather than code page 437.
def utf8? : Bool
# Whether the central directory carried this entry's sizes or offset
# in a ZIP64 extended information field.
def zip64? : Bool
endZipEntryReaderclass
One entry's content, read a piece at a time; ZipReader#open makes them. The CRC-32 and the size are checked against the central directory before the last piece is handed out, so a caller that stops at the size it was told has still had every byte checked; and content that inflates past the size the directory declares is refused at the read that would pass it.
Each method's decoder is the module's that has it: Inflater from std/compress, Bzip2Reader from std/bzip2, ZstdReader from std/zstd, and XzReader from std/xz for xz and, as a .lzma file, for LZMA. The window a Zstandard frame names may be up to 512 MiB, the most ZstdReader takes, and the dictionary an LZMA or xz stream names up to XzReader::DICT_MAX, rather than their own defaults: both readers grow their buffers with the output, not to the window, and the output is held to the size the directory declares - which ZipReader.open's limit bounds - so the memory an entry costs is its own size at most, and an entry another tool wrote with a large window is read.
pub class ZipEntryReader
# Whether every byte has been read and checked.
def finished? : Bool
# *lzma_head* is the .lzma header `ZipReader#open` made of an LZMA
# entry's own, which the decoder is given before the data after it.
def initialize(source : ZipSource, entry : ZipEntry, data_start : Int64, slot_end : Int64, lzma_head : String)
# Up to *max* bytes of the content, "" at its end; or what is wrong
# with it, after which every read answers the same. A *max* below 1 is
# a panic.
def read(max : Int32 = 65536) : String | ZipError
endZipErrorstruct
What is wrong with an archive or an entry: message is the sentence, which says which entry where there is one.
pub struct ZipError
def initialize(message : String)
def to_s : String
endZipMethodenum
The methods this module writes, by their number in the format.
pub enum ZipMethod : Int32
Stored = 0
Deflated = 8
Bzip2 = 12
Zstandard = 93
endZipReaderclass
An archive's entries, read from its central directory.
zip = ZipReader.open(File.read("a.zip"))!
zip.entries.each { |e| puts "#{e.name} #{e.size}" }
if readme = zip.find("README")
text = zip.read(readme)!
endpub class ZipReader
# The archive's comment, its bytes as written.
def comment : String
def entries : Array(ZipEntry)
# Every entry written under *dir*, directories made as needed, files
# that are there already written over. Every name is checked before
# anything is written: one that is absolute, has a `..` part, or holds
# a `\` or a `:` (a separator and a drive on Windows) would land outside
# *dir*, and the archive is refused whole. A file whose content fails
# its check is deleted and the error answered.
def extract(dir : String) : Nil | ZipError
# The entry named *name*, or nil.
def find(name : String) : ZipEntry | ::Nil
# *entry*'s content as a stream; see `ZipEntryReader`. An encrypted
# entry, one in a method this module does not read, and one whose
# local header is not where and what the directory says are refused,
# and so is an LZMA entry whose own header is not LZMA's.
def open(entry : ZipEntry) : ZipEntryReader | ZipError
# The archive in *data*. *limit* is the most the entries may hold
# uncompressed, all together: an archive whose directory says more is
# refused here, and an entry that inflates past what the directory says
# it holds is refused as it reads.
def self.open(data : String, limit : Int64 = NO_LIMIT) : ZipReader | ZipError
# The archive in a file open for reading, which is read where the
# directory and the entries are and not whole. It is moved about with
# `seek`; the reader and its entry readers own its position.
def self.open(io : IyiIO, limit : Int64 = NO_LIMIT) : ZipReader | ZipError
# *entry*'s whole content, checked as `open`'s stream checks it.
def read(entry : ZipEntry) : String | ZipError
# Why *name* may not be written under a directory, or nil when it may.
def self.unsafe_name(name : String) : String | ::Nil
endZipWriterclass
An archive written to an IyiIO, an entry at a time, the central directory at finish.
File.open("out.zip", "w") do |io|
zip = ZipWriter.new(io)
zip.add("README", text)
zip.add("photo.jpg", jpeg, ZipMethod::Stored)
zip.add_directory("empty")
zip.finish
endWhat is wrong with the calls is a panic: a name that is empty, longer than 65,535 bytes, absolute or holding a \ (APPNOTE 4.4.17), or already in the archive; an add after finish, and finish twice; a comment longer than 65,535 bytes; bzip2 or Zstandard asked of an entry streamed from an IyiIO, which those two compress only whole.
pub class ZipWriter
# *data* as the entry *name*, compressed with *method* - deflate,
# bzip2 (blocks of 900,000 bytes, bzip2's -9, as Python's `zipfile`
# writes it), Zstandard, or stored as it is - and modified at
# *modified*'s wall clock.
def add(name : String, data : String, method : ZipMethod = ZipMethod::Deflated, modified : Time = Time.local) : Nil
# What *source* holds until its end as the entry *name*, written as it
# is read: the local header first, without the sizes and CRC-32, and a
# data descriptor after the data with them. *source* is not closed.
# Deflated or stored only: the bzip2 and Zstandard compressors take
# their input whole, and an entry for them is added from a `String`.
def add(name : String, source : IyiIO, method : ZipMethod = ZipMethod::Deflated, modified : Time = Time.local) : Nil
# A directory entry: *name* with a `/` after it if it has none, stored
# and empty.
def add_directory(name : String, modified : Time = Time.local) : Nil
# The central directory and the end, with *comment*; then *io* is
# flushed. Once only.
def finish(comment : String = "") : Nil
# *zip64* writes every entry with the ZIP64 extended field and the end
# with the ZIP64 records, needed or not. Offsets count from where *io*
# is when the writer starts.
def initialize(io : IyiIO, zip64 : Bool = false)
endError for ZipErrorimpl
impl Error for ZipError
def message : String
end
ZIP archives (PKWARE's APPNOTE.TXT 6.3.10), read and written in iyi, with the DEFLATE of
std/compress, the bzip2 ofstd/bzip2, the Zstandard ofstd/zstd, the LZMA and xz ofstd/xz, and the CRC-32 ofstd/digest.