std/http
import std/http to use it. Source: src/std/http.iyi at v0.18.0.
**The client.** HTTP.get, head, post, put, delete and the request under them speak http:// and https://, to a host name (resolved as IyiSocket.connect resolves one, through iyi's own resolver), an IPv4 address, an IPv6 one in brackets (http://[::1]:8080/) or localhost. https:// is std/tls's TLS 1.3, the server's certificate checked for the host against the system's roots or HTTP.tls_config's. Each answers the whole Response; HTTP.open(...) { |response| ... } answers the block's value instead, the body still on the wire: response.body_io.read takes it a piece at a time, so a download of any size costs what one piece costs. A request's body is a String, sent with its length, or an IyiIO, sent chunked as it is read. Redirects (301, 302, 303, 307, 308) are followed, ten at most by default, the method becoming GET where a browser's does; credentials and cookies a caller wrote are not carried to another host. A body coded gzip, deflate or zstd (RFC 8878, its window held to the 8 MB RFC 9659 sets) is decoded as it arrives, unless the caller asked for a coding itself; a jar: (std/cookie's CookieJar) keeps what Set-Cookie sets and sends it back. Each HTTP request is a connection of its own; an HTTPClient keeps the connections its answers leave open and sends the next request to that server over one. Response headers are read by name without regard to case, a field on several lines is one field with its values joined by ", " (header_values keeps the lines apart, as a Set-Cookie must be read), and interim 1xx answers are read past.
**The server.** Server.serve(listener) { |request| Response }: a fiber per connection under a group, each connection kept alive across requests until the client or the block's answer says Connection: close, the client closes, or sends something that is not HTTP (answered 400 and closed; a body past Server::MAX_BODY 413, a head past Server::MAX_HEAD 431). The block gets a Request - method, path (an absolute-form target's own), query, headers, body - and answers a Response. By default the body is read whole before the block runs; with stream_bodies: true the block reads request.body_io itself, and what it leaves unread is read past, up to a megabyte, or the connection closed. An answer is a String, written with its length; an IyiIO, copied out and closed (with its length when the block gave a Content-Length, chunked otherwise); a Response.stream { |w| ... } whose block writes as it goes; or a Response.upgrade, after whose 101 the connection belongs to the block (std/websocket's). With compress: true an answer whose type is worth compressing goes zstd or gzip to a client that accepts one: zstd for a whole body when the client's q for it is at least its q for gzip, gzip for a stream (Server.coding_for). Beside it: std/router routes requests by pattern as Go's ServeMux does, std/fileserver answers them from a directory, std/sse streams events, and std/websocket takes a connection over. The socket parks under the runtime, so a slow client costs one fiber; the listener's close ends serve. With tls: a TlsServerConfig (std/tls's server: TLS 1.3, an Ed25519 or P-256 certificate) every connection is TLS, its handshake first and close_notify last; inside it, as in the clear, a connection that opens with HTTP/2's preface - as one whose ALPN chose h2 does - is served HTTP/2, any other HTTP/1.1. A config with client_roots asks each client for its certificate, and a request carries the chain it presented in peer_certificates; an https:// request presents HTTP.tls_config's identity when asked.
**Errors are values.** What the other end does wrong - a malformed head, a body cut short, a broken chunk, a corrupt gzip - is an HTTPError beside Cancelled and SocketError; ! passes it up and .or_panic refuses it with its sentence. What a program does wrong - a header with a line break in it, a status of 2000 - is a panic, before anything is written. format_request, parse_request, format_response and parse_response are the text of whole messages, pure, for a program and for the gate; both halves are this one module, so what one writes the other reads, and bench/std_http_exercise.sh runs the client against the server and both against Python's.
**HTTP/2** (RFC 9113, the section near the end of this file): the same requests and answers as frames over one connection. serve takes a connection that opens with HTTP/2's preface as HTTP/2 (h2c with prior knowledge) beside HTTP/1.1 on the same listener; the client speaks it where TLS's ALPN chose h2 - offered once a program lists it in HTTP.tls_config.alpn - and over http:// from an HTTPClient.new(prior_knowledge: true). A server's block, a request's body and an answer's are what they are over HTTP/1.1; version says HTTP/2. bench/http2_exercise.iyi and the gate hold it against curl, Node's own HTTP/2 both ways (TLS with ALPN included, the server's own TLS too) and RFC 9113's error rules frame by frame.
**Through a proxy.** proxy: on HTTP.request, HTTP.open and HTTPClient.new is an http:// proxy's URL (host:port alone will do), credentials and all: http:// goes to it as an absolute-form request (RFC 9112 §3.2.2), https:// through a CONNECT tunnel (RFC 9110 §9.3.6) with TLS to the server inside, and the URL's user and password go as Proxy-Authorization: Basic to the proxy alone, never inside a tunnel. HTTP.upgrade(proxy:) - std/websocket's WebSocket.connect - goes through a tunnel for http:// too, its handshake inside (upgrade says why). A CONNECT the proxy refuses (407, 403, 502) is an HTTPError naming its status; kept connections are kept by proxy, and a tunnel by proxy and server. HTTP.proxy_for(url, http_proxy, https_proxy, no_proxy) says which proxy a URL takes under those variables' values, NO_PROXY read as Go's httpproxy reads it, and std/http_proxy's HTTPProxy.from_env reads them from the environment. bench/std_http_exercise.sh holds it against a proxy written in Python - a WebSocket through it, ws:// and wss://, among the rest - curl through the same proxy, and Go's net/http for NO_PROXY.
**What it reaches.** What std/socket reaches, and for https:// what std/tls does - the operating system's trusted roots and its entropy - and nothing of its own: no file, and no clock or environment beyond those. Cookies with their expiry are std/cookie's, a directory served is std/fileserver's, a multipart boundary is made from the parts' own hash, and a proxy named by the environment is std/http_proxy's to read.
Not here: HTTP/2's server push, its upgrade from HTTP/1.1 (which RFC 9113 retired) and RFC 8441's WebSocket over HTTP/2 - an upgrade offers ALPN http/1.1 alone - pipelining requests on a kept HTTP/1.1 connection; a proxy reached over TLS (https://) or SOCKS, which are refused by name; and a streamed answer in zstd, std/zstd's compressor taking its input whole.
BodyReaderclass
A body as it arrives: from a connection, framed by its length, by chunks or by the connection's close, or from a string already in hand. read answers the next piece, "" once the body is done; what the other end does wrong is an HTTPError, and a connection that fails or is cancelled answers that. The first failure is kept: every read after it answers the same. A body coded gzip, deflate or zstd is decoded as it is read when the client asked for that.
A body has a limit: past it a read answers an HTTPError with status 413. The server's is Server::MAX_BODY; a client's is none.
pub class BodyReader
# The body's coding undone from here on: `Envelope::Gzip` for gzip,
# `Envelope::Zlib` for HTTP's deflate - which some servers send as the
# bare stream instead, without zlib's two bytes of header, so the first
# two bytes are looked at first, and a pair that is not a zlib header
# (RFC 1950 §2.2: method 8, a multiple of 31) is read as the bare
# stream. A body that does not decode is an `HTTPError` naming the
# coding and why.
def decode(envelope : Envelope) : Nil
# The body's `zstd` coding (RFC 8878) undone from here on, by std/zstd's
# `ZstdReader` as the pieces arrive, frames one after another. A frame
# that asks for a window past *max_window* bytes is refused rather than
# given the memory (a window outside 1 KiB to 512 MiB is a panic), and a
# body that does not decode is an `HTTPError` naming the coding and why.
def decode_zstd(max_window : Int32 = Zstd::WINDOW_LIMIT) : Nil
# Reads past the rest of the body, *limit* bytes at most: whether it
# ended within them. A server reads past what a block left unread, so
# the next request on the connection starts where it should.
def drain(limit : Int64) : Bool
# Whether every byte of the body has been read, the framing's end too.
def finished? : Bool
# A body that arrives as an HTTP/2 stream's DATA: refused past *limit*.
protected def self.frames(h2 : H2Connection, id : Int32, limit : Int64) : BodyReader
def initialize(conn : Conn | ::Nil, kind : Int32, length : Int64, limit : Int64, text : String = "", hello : Proc(Bool) | ::Nil = nil)
protected def on_frames(h2 : H2Connection, id : Int32) : Nil
# The body's bytes as the framing delivers them, before any decoding.
protected def raw(max : Int32) : String | Cancelled | SocketError | HTTPError
# The next piece of the body, at most *max* bytes; "" when it is done.
def read(max : Int32 = 65536) : String | Cancelled | SocketError | HTTPError
# The rest of the body as one string, refused past *limit* bytes (413).
# The pieces are kept as they arrive and joined once: a builder of the
# length a head declared was made before any of it had come, and a head
# that declared 64 MB and sent two bytes cost the server 64 MB.
def read_all(limit : Int64 = 2147483647_i64) : String | Cancelled | SocketError | HTTPError
protected def remember(fault : HTTPError | Cancelled | SocketError) : HTTPError | Cancelled | SocketError
# A body that is a string already in hand.
def self.text(text : String) : BodyReader
# Whether the body ends where the connection does: nothing can follow
# it on the connection.
def to_close? : Bool
endBodyWriterclass
What a streamed body is written to: an answer's Response.stream block's, and a client's upload from an IyiIO. Each write goes out as one chunk (RFC 9112 §7.1), or as it is where the connection's close ends the body, so a block that writes a line a second is read a line a second. Under a server's compress, each write is a gzip piece flushed whole (std/compress's Deflater), which decodes as it arrives too. The first failure is kept, and every write after it answers it; finish, which the server calls when the block returns, writes the end.
pub class BodyWriter
# The end of the body: the gzip trailer, and the last chunk with the
# blank line after it. Once; the server's call after a block that
# finished its own body writes nothing.
def finish : Nil | Cancelled | SocketError
def initialize(socket : IyiSocket | TlsStream | H2Out, chunked : Bool, deflater : Deflater | ::Nil = nil)
def write(text : String) : Nil | Cancelled | SocketError
# Bytes of the body written so far, before any framing or coding.
def written : Int64
endCookieStoretrait
What HTTP.request's jar: keeps cookies in: the Cookie header a request to a URL carries, and an answer's Set-Cookie lines taken in. std/cookie's CookieJar is one; std/http itself reads no clock, so the expiry of a cookie is the store's to keep.
pub trait CookieStore
abstract def header_for(url : String) : String | ::Nil
abstract def update(url : String, response : Response) : Nil
endHTTPstruct
pub struct HTTP
# For a request target in absolute form (RFC 9112 §3.2.2), where its
# authority starts: after `http://` or `https://`, in any case.
def self.absolute_form(target : String) : Int32 | ::Nil
# Whether a list of codings a client accepts (`Accept-Encoding`, RFC
# 9110 §12.5.3) takes *coding*: named, or `*`, with a `q` above 0.
def self.accepts?(list : String, coding : String) : Bool
# A 1xx, 204 or 304: an answer with no content (RFC 9110 §15).
def self.bodiless?(status : Int32) : Bool
# Where a chunked body that starts at *from* ends in `text` — the byte
# after the last chunk's terminating CRLF (trailers included) — nil when
# it has not all arrived, or why it is not a chunked body. What
# `decode_chunked` will then read whole.
def self.chunked_end(text : String, from : Int32 = 0) : Int32 | String | Nil
# How much a list of codings a client accepts wants *coding*, as its
# `q` in thousandths: the weight it is named with, else `*`'s, else 0 -
# and 0 refuses it.
def self.coding_weight(list : String, coding : String) : Int32
# A `Content-Length` as a number, or nil when it is not one. Repeated
# lines arrive joined, and are one length only when every one says the
# same (RFC 9112 §6.3). A length past 2147483647 is still a length, too
# large for either side, and is read as one past it: `to_i?` answered
# nil for it, and 3000000000 was refused as "not a number".
def self.content_length(value : String) : Int64 | ::Nil
# A control character other than a tab, or DEL: what a field value may
# not hold (RFC 9110 §5.5), nor a request target or a reason phrase.
def self.control?(text : String) : Bool
# The pairs of a request's `Cookie` header, `a=1; b=2` (RFC 6265 §4.2),
# a value's quotes taken off; the first of a name is kept, as a server
# reads the most specific cookie first.
def self.cookie_pairs(value : String) : Hash(String, String)
# A chunked body (RFC 9112 §7.1) as one string: each chunk's size line
# is hexadecimal, extensions after `;` are ignored, and a zero-length
# chunk ends it. What follows the last chunk is trailers, which nothing
# here reads.
def self.decode_chunked(text : String) : String
def self.delete(url : String, headers : Hash(String, String) = {} of String => String) : Response | Cancelled | SocketError | HTTPError
def self.find_headers_end(text : String, from : Int32 = 0) : Int32 | ::Nil
# The request as it goes on the wire, whole. `Host`, `Connection` and
# `Content-Length` are the request's own lines; a caller's header of
# that name is refused rather than sent twice. A body's length is
# always written for the methods a body belongs to, so `POST` with
# nothing to say still says `Content-Length: 0`. A byte of the path past
# 127 goes as `%` and two hex digits: a request target is ASCII (RFC
# 9112 §3.2, RFC 3986 §2.1), and `/ü` went out as its UTF-8 bytes, raw.
def self.format_request(method : String, path : String, host : String, body : String = "", headers : Hash(String, String) = {} of String => String) : String
def self.get(url : String, headers : Hash(String, String) = {} of String => String) : Response | Cancelled | SocketError | HTTPError
# *text* in gzip, whole: an answer's body under a server's `compress`.
def self.gzip(text : String) : String
# A writer whose body goes out in gzip as it is written.
def self.gzip_writer(socket : IyiSocket | TlsStream | H2Out, chunked : Bool) : BodyWriter
def self.has_break?(text : String) : Bool
# Whether a list of tokens (RFC 9110 §5.6.1), its repeated lines joined
# with ", ", holds *token*, in any case.
def self.has_token?(list : String, token : String) : Bool
def self.head(url : String, headers : Hash(String, String) = {} of String => String) : Response | Cancelled | SocketError | HTTPError
# The value of a chunk size's hex digits, nil for anything else or for
# one past what an Int32 holds.
def self.hex(text : String) : Int32 | ::Nil
# *n* in lowercase hexadecimal: a chunk's size line.
def self.hex_text(n : Int64) : String
# A Host value - the `a` of `Host: a` - or an absolute-form target's
# authority, made of what a host and its port are spelled with (RFC 3986
# §3.2.2, §3.2.3): a name, an IPv4 address or an IPv6 one in brackets,
# and digits after a colon. Empty is one too, for a target with no
# authority (RFC 9112 §3.2). A blank, a `/`, an `@` - the bytes another
# host or a path begins with - are not.
def self.host?(text : String) : Bool
def initialize
# One request, and the block's answer to the response, its body still
# arriving: `response.body_io.read` takes it a piece at a time, and
# `response.body` all of it. The connection closes when the block
# returns. A body of *body* is a string, sent with its length, or an
# `IyiIO`, sent chunked as it is read - which a redirect that keeps the
# method (307, 308) cannot send again, so that answer is handed to the
# block as it is.
#
# The redirects followed are RFC 9110 §15.4's: 301 and 302 make a POST a
# GET, 303 makes anything but a HEAD a GET, and 307 and 308 keep the
# method and the body. `Authorization`, `Proxy-Authorization` and
# `Cookie` written by the caller are not sent to another host. One more
# than *redirects* is an `HTTPError`; with 0 the first answer is the
# block's.
#
# *timeout_ms* bounds the connect, and each read and write after it -
# a pause, not the whole exchange, as Python's `timeout` is: a body that
# keeps arriving is read for as long as it takes. One that runs out is a
# `SocketError` whose `timed_out?` is true. 0, the default, waits as
# long as the server does, as Go's and Python's clients do; a client
# that talks to servers it does not control should give one.
#
# *proxy* is a proxy's URL - `http://user:password@host:port`, or
# `host:port` alone - and every request goes through it, every redirect
# too: `http://` as an absolute-form request to the proxy (RFC 9112
# §3.2.2), which carries `Proxy-Authorization: Basic` when the URL has
# a user; `https://` through a tunnel the proxy opens with `CONNECT
# host:port` (RFC 9110 §9.3.6), the credential on the CONNECT alone and
# TLS to the server inside it, its certificate checked as without a
# proxy. A caller's own `Proxy-Authorization` is the proxy's too: sent
# on the CONNECT when the URL has no user, and never inside the tunnel.
# A CONNECT answered with anything but a 2xx - 407 when the proxy wants
# credentials, 403, 502 - is an `HTTPError` naming the proxy and its
# answer. `http://` through a proxy is HTTP/1.1 whatever
# *prior_knowledge* says. Which proxy, if any, a URL's environment
# names is `proxy_for`'s, and std/http_proxy's `HTTPProxy.from_env`.
def self.open(method : String, url : String, headers : Hash(String, String) = {} of String => String, body : String | IyiIO = "", redirects : Int32 = 10, jar : CookieStore | ::Nil = nil, decompress : Bool = true, client : HTTPClient | ::Nil = nil, timeout_ms : Int32 = 0, proxy : String | ::Nil = nil, &block : (Response -> R)) : R | Cancelled | SocketError | HTTPError forall R
# The field lines after the first, or why they are not fields. A name
# repeated, in any case, is one field, its values joined in order with
# ", " (RFC 9110 §5.3): into a Hash by name, a second line spelled the
# same replaced the first and one spelled otherwise was an entry
# `header` never reached, so two different `Content-Length`s were read
# as whichever came first or last. A value with a control character in
# it - a CR or LF that ends no line, a NUL - is refused (§5.5) rather than
# handed to whoever reads it or echoes it into a header.
#
# *unfold*, for an answer: a line that begins with a blank continues the
# field above it (obs-fold, RFC 9112 §5.2), and a user agent joins it on
# with one space; read as a field of its own, `HTTP.get` panicked with
# "header without a colon: \" b\"". A server may refuse one, and this
# one does.
#
# *apart*, when given, is handed each repeated field's values by the
# name in the answer, a line each, before they are joined.
def self.parse_fields(lines : Array(String), unfold : Bool = false, apart : Hash(String, Array(String)) | ::Nil = nil) : Hash(String, String) | String
# The response as it came off the wire, whole. `head` says the request
# was a `HEAD`, whose answer carries the headers of a body it does not
# send. An interim answer - `100 Continue`, `103 Early Hints` - is read
# past to the final one after it (RFC 9110 §15.2). A `101` switches
# protocols, and HTTP ends with it. What is not a response panics with
# why: this is the text a program has in hand; the client answers the
# same sentences as `HTTPError`s.
def self.parse_response(text : String, head : Bool = false) : Response
# A response's head - the text before its blank line - read: status,
# reason, the fields, and the lines of those on more than one. A line
# that begins with a blank continues the field above it (obs-fold, RFC
# 9112 §5.2), which a user agent joins on with one space.
def self.parse_response_head(text : String) : ::Tuple(Int32, String, Hash(String, String), Hash(String, Array(String)) | ::Nil) | HTTPError
# Where *text* goes on at *from* past the empty lines there: what a
# server skips before a request line (RFC 9112 §2.2).
def self.past_empty_lines(text : String, from : Int32) : Int32
def self.post(url : String, body : String | IyiIO = "", headers : Hash(String, String) = {} of String => String) : Response | Cancelled | SocketError | HTTPError
# The proxy a request for *url* goes through, given the values of
# `HTTP_PROXY`, `HTTPS_PROXY` and `NO_PROXY` - nil when it goes
# direct. Go's `golang.org/x/net/http/httpproxy` is the rule, line for
# line, as its documentation and tests state it:
#
# * `https://` takes *https_proxy*, `http://` *http_proxy*, any other
# scheme none. A proxy written without a scheme is `http://`'s, and is
# answered with it, ready for `open`'s *proxy*.
# * `localhost` (in lower case: Go's comparison) and a loopback address
# (127.0.0.0/8, ::1, the first as IPv4-mapped too) never go through a
# proxy.
# * *no_proxy* is a comma-separated list, each entry trimmed and read
# without regard to case: `*` alone sends everything direct; an IP
# address or a CIDR range (`10.0.0.0/8`, `2001:db8::/32`) matches the
# addresses in it, IPv4 and its IPv4-mapped form alike, and a CIDR
# range of one family never the other's; a domain matches itself and
# every name under it (`foo.com`: `foo.com`, `bar.foo.com`, not
# `barfoo.com`); with a leading dot, or `*.`, only the names under it
# (`.y.com`: `x.y.com`, not `y.com`). An address or domain may carry
# a port (`1.2.3.4:80`, `[::1]:443`, `foo.com:8080`), and matches
# then on that port alone, the URL's own or its scheme's default.
# An entry that is none of these (`:80`) is passed over.
# * A host is compared as IDNA's ASCII form, as Go compares it.
#
# Pure: the values are the caller's, so std/http reaches no environment;
# std/http_proxy reads them from it.
def self.proxy_for(url : String, http_proxy : String | ::Nil, https_proxy : String | ::Nil, no_proxy : String | ::Nil = nil) : String | ::Nil
def self.put(url : String, body : String | IyiIO = "", headers : Hash(String, String) = {} of String => String) : Response | Cancelled | SocketError | HTTPError
# The reason phrase a status carries (RFC 9110 §15), for a response the
# server writes; a status this table does not know gets its class's.
def self.reason_for(status : Int32) : String
# One request and its whole answer: the answer as `open` hands it, its
# body read. Redirects are followed, *redirects* at most; *jar* keeps
# the cookies the answers set and sends them back; *decompress* asks
# for gzip, deflate and zstd and undoes them. A refused or reset
# connection answers `SocketError`; a task cancelled answers
# `Cancelled`; what the server does wrong answers `HTTPError`. `!`
# passes any of them up, `.or_panic` refuses it.
def self.request(method : String, url : String, headers : Hash(String, String) = {} of String => String, body : String | IyiIO = "", redirects : Int32 = 10, jar : CookieStore | ::Nil = nil, decompress : Bool = true, timeout_ms : Int32 = 0, proxy : String | ::Nil = nil) : Response | Cancelled | SocketError | HTTPError
# A request's head: the request line, `Host`, `Connection` (*connection*:
# `close`, or `Upgrade`), the caller's headers, and the body's framing -
# *length*, chunks, or none.
def self.request_head(method : String, path : String, host : String, headers : Hash(String, String), length : Int64 | ::Nil, chunked : Bool, connection : String) : String
# Two field names for the same field: ASCII case is the only difference
# a name may carry.
def self.same_name?(a : String, b : String) : Bool
def self.split_crlf(text : String) : Array(String)
# A status line read: the status and its reason, or why it is not one.
def self.status_line(line : String) : ::Tuple(Int32, String) | HTTPError
# What an `https://` request is checked against and offers: the
# system's roots, and ALPN's `http/1.1`, unless a program sets its own -
# roots of its own for a server it runs, say.
def self.tls_config : TlsConfig
def self.tls_config=(config : TlsConfig) : TlsConfig
# A token (RFC 9110 §5.6.2): what a method or a field name is made of.
def self.token?(text : String) : Bool
# Without the blanks at either end (RFC 9112 §5.1: field values are
# trimmed of the whitespace around them).
def self.trim(s : String) : String
# The 101 an upgrade to *protocol* is answered with, and the connection,
# which is the caller's from then on: std/websocket's handshake. Any
# other answer is an `HTTPError` naming it.
#
# *proxy* is `open`'s, and so is its credential, on the CONNECT alone;
# but an upgrade goes through a tunnel for `http://` as for `https://`,
# the handshake inside it in origin form. `Upgrade` belongs to the
# connection it is sent on (RFC 9110 §7.8), and a proxy forwarding a
# request removes `Connection` and every field it names (§7.6.1), so a
# handshake handed to a forward proxy in absolute form reaches the
# server, at best, as a plain GET: the gate's proxy, which drops
# `Connection`, leaves std/websocket's server a request it answers 400.
def self.upgrade(url : String, protocol : String, headers : Hash(String, String) = {} of String => String, timeout_ms : Int32 = 0, proxy : String | ::Nil = nil) : Upgraded | Cancelled | SocketError | HTTPError
endHTTPClientclass
Requests that keep their connections: an HTTPClient holds the connections its answers left open, by host and port, and the next request to that server goes over one of them instead of a new one - the TCP handshake, and the slow start after it, paid once rather than per request. Its methods are HTTP's, redirects, cookies and decoding included:
client = HTTPClient.new
client.get("http://127.0.0.1:8080/a")
client.get("http://127.0.0.1:8080/b") # the same connection
client.closeA connection goes back once its answer has been read to the end (inside open, up to 64 KiB left unread is read past for it) and the answer allows it; at most idle wait per server, and the rest close. A server may close an idle connection whenever it likes (RFC 9112 §9.3.1), so a request whose kept connection turns out closed - the write fails, or it ends before a byte of an answer - is sent again once on a new one, when its method is idempotent (RFC 9110 §9.2.2) and its body a string; anything else answers the failure. One task at a time per request: two tasks sending at once take two connections, and a client is not for sharing across threads.
Over HTTP/2 - https:// where ALPN chose h2, or http:// with prior_knowledge (h2c, RFC 9113 §3.3) - a kept connection carries the next request on a new stream, until the server says GOAWAY.
pub class HTTPClient
# Every idle connection closed. The client may be used after; it opens
# new ones.
def close : Nil
# Connections this client has opened, and requests sent over one it
# kept: what a program reads to see that its connections are kept.
def connections_made : Int32
def connections_reused : Int32
def delete(url : String, headers : Hash(String, String) = {} of String => String) : Response | Cancelled | SocketError | HTTPError
def get(url : String, headers : Hash(String, String) = {} of String => String) : Response | Cancelled | SocketError | HTTPError
protected def give_back(key : String, conn : Conn) : Nil
def head(url : String, headers : Hash(String, String) = {} of String => String) : Response | Cancelled | SocketError | HTTPError
# *timeout_ms* and *proxy* are `HTTP.open`'s, for every request the
# client sends.
def initialize(idle : Int32 = 4, timeout_ms : Int32 = 0, prior_knowledge : Bool = false, proxy : String | ::Nil = nil)
protected def made : Nil
def open(method : String, url : String, headers : Hash(String, String) = {} of String => String, body : String | IyiIO = "", redirects : Int32 = 10, jar : CookieStore | ::Nil = nil, decompress : Bool = true, &block : (Response -> R)) : R | Cancelled | SocketError | HTTPError forall R
def post(url : String, body : String | IyiIO = "", headers : Hash(String, String) = {} of String => String) : Response | Cancelled | SocketError | HTTPError
# `http://` spoken as HTTP/2 from the first byte, without asking: for a
# server known to take it.
def prior_knowledge : Bool
# The proxy every request goes through, `HTTP.open`'s *proxy*; its
# connections are kept under it.
def proxy : String | ::Nil
def put(url : String, body : String | IyiIO = "", headers : Hash(String, String) = {} of String => String) : Response | Cancelled | SocketError | HTTPError
def request(method : String, url : String, headers : Hash(String, String) = {} of String => String, body : String | IyiIO = "", redirects : Int32 = 10, jar : CookieStore | ::Nil = nil, decompress : Bool = true) : Response | Cancelled | SocketError | HTTPError
# A kept connection to *key*, nil when none waits. An HTTP/2 connection
# that can take no more requests is closed rather than handed out.
protected def take(key : String) : Conn | ::Nil
endHTTPErrorstruct
What the other end did that HTTP does not allow, as a value: reason is the sentence, status the answer a server gives a request that did it (400, or 413 and 431 for the limits). message says it is HTTP's.
pub struct HTTPError
def initialize(reason : String, status : Int32 = 400)
def reason : String
def status : Int32
def to_s : String
endMultipartclass
A multipart/form-data body (RFC 7578) read a part at a time, as it arrives: next_part answers the next Part, nil after the last, and a part's read answers its bytes up to the boundary, "" at its end. A part not read to its end is read past by the next next_part. The body of a request is Request#multipart's; any BodyReader and boundary will do. A part's head is refused past 16 KiB, and a body that ends before its closing boundary is an HTTPError.
pub class Multipart
# The boundary a `Content-Type` of `multipart/...` names, nil when it
# names none.
def self.boundary(content_type : String) : String | ::Nil
# Where *pattern* is in what is held, -1 when it is not there yet.
protected def find(pattern : String) : Int32
def initialize(body : BodyReader, boundary : String)
# The next part, nil after the last.
def next_part : Part | Nil | Cancelled | SocketError | HTTPError
# Up to *max* bytes of the part being read, "" at its boundary. What is
# held back is what may yet be the start of the boundary.
protected def part_read(max : Int32) : String | Cancelled | SocketError | HTTPError
# One more piece of the body onto what is held: false at its end.
protected def pull : Bool | Cancelled | SocketError | HTTPError
endMultipartBuilderclass
A multipart/form-data body to send: fields and files in order. Its boundary is chosen when the body is first asked for - finish, content_type or boundary, whichever comes first, after which no part is added - as iyi- and 32 hex digits of the parts' SHA-1, made again from itself in the rare case a part holds it: so it is none of the parts' bytes by construction rather than by chance, the same fields make the same body, and std/http reaches no source of randomness for it. A boundary given to new is used as it is, and a part that holds it is refused.
pub class MultipartBuilder
def boundary : String
def content_type : String
def field(name : String, value : String) : self
def file(name : String, filename : String, data : String, content_type : String = MIME.from_extension(filename)) : self
# The body, closing boundary and all.
def finish : String
# The boundary ends a part where it follows a CRLF, the part's first
# bytes included, which follow the head's blank line.
protected def self.holds?(data : String, mark : String) : Bool
def initialize(boundary : String | ::Nil = nil)
# A name inside quotes, as browsers write one: a quote and a line break
# percent-encoded (the WHATWG form encoding), so no field can end the
# parameter or the line early.
protected def self.quoted(text : String) : String
endPartclass
One part of a multipart body: its head's fields, its name and file name from Content-Disposition, and its bytes, read up to its boundary.
pub class Part
def content_type : String | ::Nil
# The file name a browser sent the part under, `filename=`.
def filename : String | ::Nil
def finished? : Bool
def header(name : String) : String | ::Nil
def headers : Hash(String, String)
def initialize(owner : Multipart, headers : Hash(String, String))
# The form field's name, `name=` of `Content-Disposition`.
def name : String | ::Nil
# The part's next bytes, at most *max*; "" at its end.
def read(max : Int32 = 65536) : String | Cancelled | SocketError | HTTPError
# The rest of the part as one string, refused past *limit* bytes.
def read_all(limit : Int64 = 2147483647_i64) : String | Cancelled | SocketError | HTTPError
endRequestclass
What a server's block is handed: one request. Its body is body, read whole before the block ran, or under stream_bodies read by the block from body_io as it arrives; body then reads the rest itself, and a body that cannot be read - the client went, or sent one that is not HTTP - ends the connection with that sentence rather than be answered as if it were whole.
pub class Request
def body : String
def body_io : BodyReader
# The pairs of the request's `Cookie` header, the first of a name kept.
def cookies : Hash(String, String)
# The fields of a form: the query's, and an
# `application/x-www-form-urlencoded` body's after them. The body is
# read for it.
def form : Params
def header(name : String) : String | ::Nil
def headers : Hash(String, String)
# A request whose body is in hand: what `parse_request` answers.
def initialize(method : String, path : String, query : String | ::Nil, headers : Hash(String, String), body : String, version : String = "HTTP/1.1", remote_address : String | ::Nil = nil)
# A request whose body is still on the connection.
def initialize(method : String, path : String, query : String | ::Nil, headers : Hash(String, String), version : String, reader : BodyReader, remote_address : String | ::Nil = nil)
def method : String
# The body as `multipart/form-data` parts, read as they arrive; nil when
# the `Content-Type` is not multipart or names no boundary.
def multipart : Multipart | ::Nil
def path : String
# The client's certificate chain, leaf first, as std/tls checked it:
# under `serve(tls:)` with `client_roots` set, the certificate the
# client presented; empty otherwise.
def peer_certificates : Array(Certificate)
# The client's certificate chain, leaf first, as std/tls checked it:
# under `serve(tls:)` with `client_roots` set, the certificate the
# client presented; empty otherwise.
def peer_certificates=(value : Array(Certificate)) : Array(Certificate)
def query : String | ::Nil
# The client's address, as the server's socket saw it (no port): nil
# for a request a program made, or when the client had already gone. A
# proxy in front is what this names; `X-Forwarded-For` is the proxy's
# word, and a header any client can write.
def remote_address : String | ::Nil
def version : String
endResponseclass
A response: what a server's block answers, and what a client is handed.
To write one: Response.new(200, "hello"), a status and a string; Response.new(200, io), a stream the server copies out and closes, with the caller's Content-Length when it gives one and chunked otherwise; Response.stream(200) { |w| w.write("...") }, whose block writes as it goes; Response.upgrade(headers) { |socket, leftover| ... }, a 101 after which the connection is the block's. add_header writes a field on a line of its own, as each Set-Cookie must be.
What a client is handed: the status, reason and headers, and the body - body whole, or body_io as it arrives inside HTTP.open's block.
pub class Response
# One more line of the field *name*, written on a line of its own: what
# two `Set-Cookie`s need, since the ", " a field's lines are joined with
# is part of a cookie's date.
def add_header(name : String, value : String) : Nil
# The lines of each field written on more than one, by its name in
# `headers`; nil when every field is on one line.
def apart : Hash(String, Array(String)) | ::Nil
# The body: whole. Inside `HTTP.open`'s block it is read here, the first
# time it is asked for, and a body that cannot be read panics with why;
# `body_io` answers that as a value.
def body : String
# The body as it arrives. For a response a program made, its string.
def body_io : BodyReader
# Whether the client undid the body's `gzip`, `deflate` or `zstd`
# coding; the `Content-Encoding` and `Content-Length` that described it
# are gone from the headers then.
def decoded? : Bool
# A header by name, as the server spelled it or otherwise: a field
# name is case-insensitive (RFC 9110 §5.1), and `content-length` is as
# common on the wire as `Content-Length`.
def header(name : String) : String | ::Nil
# A field's lines apart, in the order they came: `header` joins them
# with ", ", and `Set-Cookie` is the field that join breaks (RFC 9110
# §5.3), since a cookie's `Expires` date has a comma in it - two cookies
# read as "a=1; Expires=Wed, 21 Oct 2026 07:28:00 GMT, b=2", which no
# split takes back apart. Empty for a field that is not there.
def header_values(name : String) : Array(String)
def headers : Hash(String, String)
# A response a client read: its body comes from *reader*, and *url* is
# where it came from.
def initialize(status : Int32, reason : String, headers : Hash(String, String), apart : Hash(String, Array(String)) | ::Nil, reader : BodyReader, url : String, decoded : Bool)
def initialize(status : Int32, reason : String, headers : Hash(String, String), body : String, apart : Hash(String, Array(String)) | ::Nil = nil)
# A response to write: the reason is the status's own, the headers the
# caller's. `Response.new(200, "hello")` is the common shape.
def initialize(status : Int32, body : String = "", headers : Hash(String, String) = {} of String => String)
# A response whose body is *source*, copied out by the server and
# closed after.
def initialize(status : Int32, source : IyiIO, headers : Hash(String, String) = {} of String => String)
# The body read whole into the response, which is answered; why not,
# when it cannot be.
def load : Response | Cancelled | SocketError | HTTPError
def reason : String
def source : IyiIO | ::Nil
def status : Int32
# A response whose body the block writes as it goes, chunked.
def self.stream(status : Int32, headers : Hash(String, String) = {} of String => String, &writer : (BodyWriter -> Nil)) : Response
def success? : Bool
def takeover : Proc(IyiSocket | TlsStream, String, Nil) | ::Nil
protected def takeover=(takeover : Proc(IyiSocket | TlsStream, String, Nil)) : Proc(IyiSocket | TlsStream, String, Nil)
# A 101 Switching Protocols with *headers* (`Upgrade` and the
# protocol's own), after which the connection is the block's: it is
# handed the connection - the socket, or under `serve(tls:)` the
# `TlsStream` over it - and the bytes the client sent after its request,
# and the connection closes when it returns.
def self.upgrade(headers : Hash(String, String), &takeover : (IyiSocket | TlsStream, String -> Nil)) : Response
# The URL the answer came from, after any redirects; nil for one a
# program made.
def url : String | ::Nil
def writer : Proc(BodyWriter, Nil) | ::Nil
protected def writer=(writer : Proc(BodyWriter, Nil)) : Proc(BodyWriter, Nil)
endServerstruct
pub struct Server
# The headers a coded body goes with: the coding, and `Vary` so that a
# cache keeps the answers apart. A `Content-Length` the block gave is
# the plain body's, and goes.
protected def self.coded(response : Response, coding : String) : Response
# The coding *response* goes in to the client of *req*, or nil: one the
# client takes (RFC 9110 §12.5.3, a `q` of 0 refusing it), for an
# answer with a body and a type worth compressing, and no coding of its
# own. A whole body goes `zstd` when the client's `q` for it is at least
# its `q` for gzip, and `gzip` otherwise. The tie goes to zstd for the
# server's time: measured on this machine, release builds, on
# src/std/http.iyi (199,134 bytes) `Gzip.compress` wrote 54,484 bytes in
# 5 ms and `Zstd.compress` 61,753 in 1 ms, and on CHANGELOG.md (1.5 MB)
# 552,446 in 58 ms against 589,785 in 13 ms - a quarter of the time, for
# an eighth more bytes at most. A stream - an `IyiIO` or a
# `Response.stream` - goes gzip or as it is: std/compress's `Deflater`
# writes each piece as it comes, where std/zstd's compressor takes its
# input whole, and a frame a write would code each piece alone.
def self.coding_for(req : Request, response : Response) : String | ::Nil
# A string body compressed whole in *coding*, `zstd` or `gzip`, with
# the headers that say so. `Zstd.compress`'s frame needs a window of
# 1 MiB at most, inside the 8 MB RFC 9659 §3 holds an encoder to.
protected def self.compressed(response : Response, coding : String) : Response
# A response's head. `Content-Length` is *length*'s - the caller's own
# line is checked against it, and kept only where no body follows it: a
# HEAD, or a 304 that names the length a 200 would have (RFC 9110 §8.6) -
# or `Transfer-Encoding: chunked` when *chunked*, or neither for a body
# the connection's close ends. `Connection` says what the connection will
# do next, except on an *upgrade*, whose own line says it. A 1xx, 204 or
# 304 has no content (RFC 9110 §15), and writes neither a body nor a
# length of its own (§8.6): the body was read by the client as the start
# of the next answer, and `Content-Length: 0` on a 304 told a cache that
# the resource had become empty. A field added on lines of its own
# (`add_header`) is written a line each.
def self.format_head(response : Response, head : Bool, close : Bool, length : Int64 | ::Nil, chunked : Bool, upgrade : Bool = false) : String
# The response as it goes on the wire, whole: `format_head` for its
# string body, and the body.
def self.format_response(response : Response, head : Bool, close : Bool) : String
# One connection: requests until the client is done. A request that is
# not HTTP is answered with why and the connection closed, since the
# bytes after it cannot be trusted to be a request either. Requests that
# came in one read are read where they lie, one after another. A
# connection that opens with HTTP/2's preface is served as HTTP/2. Under
# *tls* the handshake comes first, and the connection ends with
# close_notify.
def self.handle(client : IyiSocket, handler : Proc(Request, Response), listener : IyiSocket | ::Nil = nil, idle : Hash(UInt64, IyiSocket) | ::Nil = nil, stream_bodies : Bool = false, compress : Bool = false, timeout_ms : Int32 = 0, tls : TlsServerConfig | ::Nil = nil) : Nil
def initialize
# A request's head - the text before its blank line - read and checked,
# or why it is not one.
def self.parse_head(text : String) : RequestHead | HTTPError
# A request at *from* in `text`, if a whole one is there. No request and
# no error means "not yet". `consumed` counts from the start of `text`.
def self.parse_request(text : String, from : Int32 = 0) : ParsedRequest
# Serve `listener` until it is closed. The block runs on the
# connection's fiber, once per request; what it answers is written back.
# The close ends a connection waiting on its client too, for its next
# request or for the rest of one, and one that is answering ends when its
# answer is written: a keep-alive client that sat idle kept `serve` from
# returning for as long as it stayed, and so did one that had sent
# what has arrived of a request so far - one byte of it was enough.
#
# *stream_bodies*: the block reads each request's body from `body_io`
# rather than being handed it whole. *compress*: an answer whose type
# std/mime calls compressible goes zstd or gzip to a client that
# accepts one (`coding_for`).
#
# *timeout_ms*: a request's head must have arrived within it, counted
# from when the server starts waiting for one - a kept-alive connection's
# idle time included - and no read of a body nor write of an answer may
# wait longer; a connection that does is closed. Without it, a client
# that sends a byte of its head every few seconds (Slowloris) holds a
# connection and its fiber for as long as it likes, and so does one that
# reads none of its answer. A minute by default, nginx's
# `client_header_timeout`; 0 for none. A connection handed over by
# `Response.upgrade` waits as its protocol likes.
#
# *tls*: each connection is TLS 1.3 (std/tls's `TlsServer`) under this
# configuration, its handshake held to *timeout_ms* too; one whose
# handshake fails is closed, the client told why by an alert. Inside
# it a connection that opens with HTTP/2's preface - as a client whose
# ALPN chose `h2` does, when `tls.alpn` offers it - is served HTTP/2,
# and any other HTTP/1.1. A `Response.upgrade` block over TLS is handed
# the `TlsStream`, so a WebSocket is `wss://`.
def self.serve(listener : IyiSocket, stream_bodies : Bool = false, compress : Bool = false, timeout_ms : Int32 = 60000, tls : TlsServerConfig | ::Nil = nil, &handler : (Request -> Response)) : Nil
protected def self.writer_for(socket : IyiSocket | TlsStream, chunked : Bool, gzip : Bool) : BodyWriter
endTlsStreamclass
A TLS connection used as a socket is: read into a buffer, written to, closed, its timeouts set - by a Conn, a BodyWriter, and a connection handed over by an upgrade (wss:// at either end). What TLS refuses is a SocketError carrying TLS's own sentence (layer_reason), so answers keep the types they have over a plain socket.
pub class TlsStream
def close : Nil
def closed? : Bool
def initialize(tls : TlsConnection, host : String)
# The peer's certificate chain, leaf first, as the handshake checked it:
# the server's, at a client; the client's, at a server that asked.
def peer_certificates : Array(Certificate)
# Up to *count* bytes into *buffer*: 0 once the peer closed with
# close_notify.
def read(buffer : Pointer(UInt8), count : Int32) : Int32 | Cancelled | SocketError
# Up to *max_bytes* as a string, what is held first: "" once the peer
# closed with close_notify. `IyiSocket#read`'s form, for a block an
# upgrade hands either.
def read(max_bytes : Int32 = 4096) : String | Cancelled | SocketError
# How long a read or a write of the connection under it may wait.
def read_timeout_ms=(ms : Int32 | ::Nil) : Int32 | ::Nil
def write(text : String) : Int32 | Cancelled | SocketError
def write_timeout_ms=(ms : Int32 | ::Nil) : Int32 | ::Nil
endUpgradedclass
What HTTP.upgrade answers: the 101, and the connection, which is the caller's now - the socket, and the bytes the server sent after its head.
pub class Upgraded
def initialize(response : Response, socket : IyiSocket | TlsStream, leftover : String)
def leftover : String
def response : Response
# The connection: the socket, or for `https://` the `TlsStream` over it.
def socket : IyiSocket | TlsStream
endError for HTTPErrorimpl
impl Error for HTTPError
def message : String
end
HTTP/1.1 and HTTP/2 over
IyiSocket, written in iyi: a client, a server, and the bodies between them as streams.