std/tar
import std/tar to use it. Source: src/std/tar.iyi at v0.18.0.
TarReader reads an archive as the format means it to be read: from the front, a 512-byte header and then the entry's data padded to a block, entry after entry, over any IyiIO - a file, a socket, a pipe - with no seeking. next_entry answers the next TarEntry, and the entry's read and read_all give its data; data an entry's caller did not read is read past by the next next_entry, so a listing reads every byte once and keeps none. The first bytes say whether the stream is gzip (RFC 1952's 1f 8b), bzip2 ("BZh" and a block-size digit 1 to 9) or xz (FD 37 7A 58 5A 00): if they do, everything after is read through Inflater, Bzip2Reader or XzReader, as Python's tarfile.open(mode="r:*") and GNU tar -x without -z, -j or -J take one.
A header is believed only once its checksum is: the sum of its bytes with the checksum field counted as spaces, unsigned as POSIX sums it, or signed as some old writers summed it, which Python's tarfile accepts too. Its magic says which header it is: "ustar\0" "00" is POSIX, whose prefix field (155 bytes) is the front of a name longer than the 100-byte name field; "ustar " " \0" is GNU, whose bytes at that place are something else and are not read; anything else is v7, where a regular file whose name ends in / is a directory (as Python and Go read it). Numbers are octal, ended by a NUL or a space; a field whose first byte has its high bit set is GNU's base-256, which Python's GNU_FORMAT writes for a size past 8 GiB or an id past 2,097,151, and which is read, negative numbers included. Two headers come before the one they describe and are folded into it: a pax extended header (typeflag x), records "<length> <key>=<value>\n" whose path, linkpath, size, mtime (seconds with a decimal fraction, which is kept to the nanosecond), uid, gid, uname and gname stand in for the header's fields; and GNU's ././@LongLink (L for the name, K for the link's target). A global pax header (g) holds for every entry after it until another says otherwise. Every record an entry was given is in TarEntry#pax, the ones this module does not read too. A name is its bytes as written, UTF-8 by every writer this was checked against; a directory's trailing / is kept, as Go keeps it.
The end is a block of zeros, as POSIX writes two of them; a stream that ends at a block boundary without one is taken as ended too, as Python and GNU tar take it. What is wrong with an archive is a TarError, a value, and every call after it answers the same one: a header whose checksum does not add up; a number field that is not a number; a header, or an entry's data, cut short; a pax record that is not one, or a pax size, mtime, uid or gid that is not a number; an extended header (pax or GNU) past 1 MiB, which is held in memory whole and so is bounded, as Go bounds it; an extended header with no entry after it; and a gzip, bzip2 or xz stream that Inflater, Bzip2Reader or XzReader refuses. In a compressed archive the rest of the stream after the zero block is read, so the gzip trailer's CRC-32 and length, bzip2's block and stream CRCs, or xz's block checks and index, vouch for every byte handed out: content that changed is refused by the next_entry that finds the end. An xz stream's dictionary is held to XzReader's 64 MiB default.
TarWriter writes ustar headers to an IyiIO, POSIX's magic, as Python's default PAX_FORMAT and GNU tar --format=posix do, and puts a pax header in front of an entry only when a field does not fit its place: a name past 100 bytes that no / splits into a prefix of at most 155 and a name of at most 100 (the split Python makes), a link target past 100 bytes, a user or group name past 32, a size past 8 GiB - 1 (eleven octal digits), an id past 2,097,151, a modification time before 1970 or after 2242; and a name, target, user or group name that is not ASCII, which POSIX's portable character set does not hold and pax's UTF-8 records do (Python writes those the same way). A field a pax record stands in for holds what fits of it, a number zero. Modification times are written in whole seconds. finish writes the end - two zero blocks - and pads the archive to a multiple of 10,240 bytes, the record size Python and GNU tar write. TarWriter.new(io, gzip: true) writes the archive through Deflater in the gzip envelope, 128 KiB at a time.
TarReader#extract(dir) writes the entries under dir as they come: directories, regular files, symbolic links and hard links, with the permission bits of their mode (setuid, setgid and sticky are not given to what an archive made) and their modification time. It refuses "tar slip" (the zip slip of Snyk, 2018, in a tar): a name that is absolute or has a .. part; a symbolic link whose target is absolute, climbs above dir from the link's own directory, or goes down and back up (a/../b), which could pass through a link the archive made; a hard link whose target is not a regular file the archive already put under dir; and any entry that would be written through a symbolic link on the way to it - so a link the archive made cannot carry a later entry outside. On Windows a name with \ or : is refused too. A file that is there already is replaced, not written through, and a device or FIFO is refused by name. The archive is read once, so what came before a refused entry has been written: a stream cannot be checked whole first, as ZipReader#extract checks a zip's directory.
Left out: writing GNU or v7 headers, sparse files (GNU S and pax's GNU.sparse records are read as an entry whose data is what was stored), access and change times (kept in pax, not read), ACLs and extended attributes (likewise), multi-volume archives, appending, writing .tar.bz2 (Bzip2.compress takes a whole string, and a writer streams), writing .tar.xz (std/xz decompresses only), and devices made on extraction.
TarEntryclass
One entry's header, read by TarReader#next_entry; its data is read with read and read_all until the reader moves on.
pub class TarEntry
# How many bytes of data follow the header: the size, or none for a
# type that carries none.
def data_size : Int64
def devmajor : Int64
def devminor : Int64
def directory? : Bool
def file? : Bool
def format : TarFormat
def gid : Int64
def gname : String
def hardlink? : Bool
def index : Int32
def initialize(reader : TarReader, index : Int32, name : String, linkname : String, size : Int64, mode : Int32, uid : Int64, gid : Int64, uname : String, gname : String, mtime : Time, typeflag : UInt8, devmajor : Int64, devminor : Int64, pax : Hash(String, String), format : TarFormat, offset : Int64)
# A link's target, the same way: pax `linkpath`, GNU `K`, the field.
def linkname : String
def mode : Int32
def mtime : Time
# The name: a pax `path`, a GNU long name, the ustar prefix and name
# joined by `/`, or the name field, the first of these there is.
def name : String
# Where its header is, from the start of the (uncompressed) archive.
def offset : Int64
# Every pax record given to this entry, the global ones under the ones
# of its own.
def pax : Hash(String, String)
# Up to *max* bytes of the data, "" at its end; or what is wrong with
# the archive. Only until the reader's next `next_entry`: a read after
# it is a panic, as is a *max* below 1.
def read(max : Int32 = 65536) : String | TarError
# The whole of the data (what is left of it).
def read_all : String | TarError
# The size the header (or a pax `size`) declares. Links, devices,
# FIFOs and directories carry no data whatever it says, as POSIX
# specifies and Go reads them.
def size : Int64
def symlink? : Bool
def type : TarType
# What the typeflag names, for a sentence: "regular file", "FIFO",
# "GNU sparse file", "type 'Q'".
def type_name : String
# The typeflag byte as written.
def typeflag : UInt8
def uid : Int64
def uname : String
endTarErrorstruct
What is wrong with an archive: message is the sentence, which names the entry, or the header's offset in the (uncompressed) stream.
pub struct TarError
def initialize(message : String)
def to_s : String
endTarFormatenum
Which header an entry was read from: Pax when a pax record was folded into it, else by the header's magic.
pub enum TarFormat : Int32
V7 = 0
Ustar = 1
Gnu = 2
Pax = 3
endTarReaderclass
An archive read entry by entry.
tar = TarReader.new(File.open("src.tar.gz"))
while entry = tar.next_entry.or_panic
puts "#{entry.name} #{entry.size}"
text = entry.read_all.or_panic if entry.name == "README"
endpub class TarReader
# Whether the stream is bzip2; known once the first entry is asked for.
def bzip2? : Bool
# For `TarEntry#read`.
def entry_read(entry : TarEntry, max : Int32) : String | TarError
# For `TarEntry#read_all`.
def entry_read_all(entry : TarEntry) : String | TarError
# Every entry written under *dir* (made if it is not there), as it is
# read; see the module's comment for what is refused. Answers the first
# error, the archive's or the refusal's; a file cut short is deleted.
def extract(dir : String) : Nil | TarError
# Whether the stream is gzip; known once the first entry is asked for.
def gzip? : Bool
# The archive *io* holds from where it is, plain or gzip.
def initialize(io : IyiIO)
# The next entry, its header read and checked; nil at the end. The
# data of the entry before, as much as was not read, is read past.
def next_entry : TarEntry | Nil | TarError
# *path*'s parts, without empty ones and `.`.
def self.parts(path : String) : Array(String)
# A pax header's records into *into*. A global record with no value
# takes the key away; an entry's keeps it, empty, which stands for the
# header's own field.
def self.records(data : String, at : Int64, into : Hash(String, String), global : Bool) : Nil | TarError
# Why a symbolic link *name* to *target* may not be made under a
# directory, or nil when it may: the target must be relative, may climb
# with `..` no higher than the link's own directory, and only before
# its first other part - after one, the part could be a link itself and
# the `..` would climb from wherever that points.
def self.unsafe_link(name : String, target : String, windows : Bool = false) : String | ::Nil
# Why *name* may not be written under a directory, or nil when it may.
# *windows* refuses the separator and the drive Windows reads in it.
def self.unsafe_name(name : String, windows : Bool = false) : String | ::Nil
# A path Windows reads from a root rather than from where it is put:
# `\x`, or a drive with its separator, `C:\x` and `C:/x`. Refused anyway
# for its `\` or its `:`, but the sentence names what it is. `C:x`, the
# drive's own working directory, is not one, and is refused for its `:`.
def self.windows_absolute?(path : String) : Bool
# Whether the stream is xz; known once the first entry is asked for.
def xz? : Bool
endTarTypeenum
What an entry is, by its typeflag. Other is any typeflag this module has no name for; TarEntry#type_name names it.
pub enum TarType : Int32
Regular = 0
Hardlink = 1
Symlink = 2
CharDevice = 3
BlockDevice = 4
Directory = 5
Fifo = 6
Contiguous = 7
Other = 8
endTarWriterclass
An archive written to an IyiIO, an entry at a time.
File.open("out.tar.gz", "w") do |io|
tar = TarWriter.new(io, gzip: true)
tar.add("README", text)
tar.add_directory("bin")
tar.add("bin/run", script, mode: 0o755)
tar.add_symlink("latest", "bin/run")
tar.finish
endWhat is wrong with the calls is a panic: a name that is empty, holds a NUL or is absolute; a link without a target; a negative size, id or a mode outside 0 to 0o7777; a source that ends before the size it was said to hold; an add after finish, and finish twice.
pub class TarWriter
# *data* as the regular file *name*.
def add(name : String, data : String, mode : Int32 = 420, mtime : Time = Time.utc, uid : Int64 = 0_i64, gid : Int64 = 0_i64, uname : String = "", gname : String = "") : Nil
# *size* bytes of *source* as the regular file *name*, copied as they
# are read. *source* is not closed.
def add(name : String, source : IyiIO, size : Int64, mode : Int32 = 420, mtime : Time = Time.utc, uid : Int64 = 0_i64, gid : Int64 = 0_i64, uname : String = "", gname : String = "") : Nil
# A directory: *name* with a `/` after it if it has none.
def add_directory(name : String, mode : Int32 = 493, mtime : Time = Time.utc, uid : Int64 = 0_i64, gid : Int64 = 0_i64, uname : String = "", gname : String = "") : Nil
# A hard link *name* to the entry *target*, earlier in the archive.
def add_hardlink(name : String, target : String, mode : Int32 = 420, mtime : Time = Time.utc, uid : Int64 = 0_i64, gid : Int64 = 0_i64, uname : String = "", gname : String = "") : Nil
# A symbolic link *name* to *target*, which is written as it is given.
def add_symlink(name : String, target : String, mode : Int32 = 511, mtime : Time = Time.utc, uid : Int64 = 0_i64, gid : Int64 = 0_i64, uname : String = "", gname : String = "") : Nil
# The end: two zero blocks, then zeros to a 10,240-byte record; then
# the gzip trailer, if gzip; then *io* is flushed. Once only.
def finish : Nil
# *gzip* writes the archive through `Deflater` in the gzip envelope.
def initialize(io : IyiIO, gzip : Bool = false)
# One pax record: its length counts its own digits.
def self.record(key : String, value : String) : String
endError for TarErrorimpl
impl Error for TarError
def message : String
end
Tar archives - POSIX.1-2001 pax and ustar (IEEE Std 1003.1, the
paxutility's "ustar Interchange Format" and "pax Interchange Format"), read and written in iyi, with GNU's long names and the old v7 header read as well,.tar.gzthrough the DEFLATE ofstd/compress,.tar.bz2read throughstd/bzip2, and.tar.xzread throughstd/xz.