std/websocket
import std/websocket to use it. Source: src/std/websocket.iyi at v0.18.0.
**The server.** Inside Server.serve's block, WebSocket.upgrade(req) { |ws| ... } answers the request. A request that is a version 13 handshake - a GET over HTTP/1.1 whose Upgrade names websocket, whose Connection holds upgrade, and whose Sec-WebSocket-Key is 16 bytes in base64 - is answered 101 with the key's Sec-WebSocket-Accept, and the block runs with the connection. Any other request is answered 400 with why, and a handshake of another version 426 with Sec-WebSocket-Version: 13 (RFC 6455 §4.2.2, §4.4). With protocols the server names the first of its own list the client offered in Sec-WebSocket-Protocol, and none when it offered none of them: the server's order is the preference, since it is the server that knows which it serves best. The connection is closed when the block returns; a block that did not close it is closed for, 1000. An Origin is not checked here: the block reads req.header("Origin") before it calls upgrade when it cares. Closing the listener does not end a connection that has been upgraded, and serve returns once those have ended, so a server that stops closes its WebSockets itself.
**The client.** WebSocket.connect("ws://host:port/path") sends the handshake through HTTP.upgrade with a key of 16 bytes from Random::Secure and checks what comes back: a 101 whose Upgrade and Connection say websocket, whose Sec-WebSocket-Accept is the key's, that agrees to no extension but the permessage-deflate it offered, and names no protocol the client did not offer. Each of these refused is a WebSocketError; a server that does not answer 101 is the HTTPError HTTP.upgrade names it with. wss:// is the same over std/tls, its certificate checked against HTTP.tls_config and ALPN offering http/1.1; a server under Server.serve(tls:) hands upgrade's block the TLS stream in the same way. With proxy: the handshake goes through an http:// proxy's CONNECT tunnel, for ws:// as for wss://, the proxy's credential on the CONNECT alone (std/http's HTTP.upgrade).
**The connection.** send(text), send_binary(bytes) and ping write one frame each: no message is split into fragments on the way out. receive answers the next whole message, a Message whose data is its bytes, and nil once the close handshake is done. On the way it reassembles fragments, with pings and pongs between them; answers a ping with its pong; and reads past pongs. close(code, reason) sends a close and waits up to ten seconds for the peer's. A close the peer starts is answered with its own code and ends receive; close_code and close_reason say what it carried (1005 when it carried no code, 1006 when the connection ended without one).
**Compression.** permessage-deflate (RFC 7692) is on at both ends unless deflate: false is given to connect or upgrade, since browsers offer it. connect offers permessage-deflate; client_max_window_bits. upgrade accepts the first permessage-deflate offer in the list that it can keep and declines the others, as §7 has it: one that names a parameter not among RFC 7692 §7.1's four, or one twice, or with a value it may not have (server_no_context_takeover and client_no_context_takeover take none, server_max_window_bits takes 8 to 15 in decimal, client_max_window_bits that or none), is declined, and a header that is not a list of extensions at all is a 400 that says so. The 101 names server_no_context_takeover, client_no_context_takeover and server_max_window_bits as the offer did. The client refuses, as a WebSocketError that names it, an answer with another extension, permessage-deflate twice, or a parameter wrong in any of those ways. deflate is what was agreed, nil for nothing.
Once it is agreed every text and binary message goes compressed, RSV1 set on its frame: through a raw-DEFLATE Deflater (std/compress) at deflate_level, 0 to 9 and 1 unless the caller says otherwise, whose sync flush ends it, the flush's last four bytes, 00 00 ff ff, left off (§7.2.1); an empty message is the one byte 00 (§7.2.3.6). A message past 256 KiB is compressed 256 KiB at a time, a flush after each, so the compressor's window does not grow to the largest message and stay that size. The window carries over from one message to the next unless this end's no_context_takeover was agreed, and a match reaches no farther back than this end's max_window_bits allows: the compressor holds itself to a smaller window exactly, so no offer is declined for one. Pings, pongs and closes are never compressed. A compressed message from the peer is inflated when its last frame is in, 00 00 ff ff put back, by an Inflater.flushed whose window is the whole 32 KiB, so whatever max_window_bits the peer writes with reads; its window carries over unless the peer's no_context_takeover was agreed. max_message holds on both sides of the decoder - on the compressed bytes as their frames declare them, and on the inflated ones as they come out - so a deflate bomb, kilobytes that would inflate to gigabytes, is refused 1009 once it is 64 KiB past the limit, never inflated whole. Compressed bytes that are not DEFLATE, or that do not end between blocks, are 1007. A compressed message of no bytes at all, which nothing built on zlib writes, is read as empty. Compression of secrets beside text an attacker chooses, over TLS, can give the secrets away by the sizes (CRIME; RFC 7692 §8): a server that sends both turns it off.
Measured here (--release, one machine, both ends in one process over loopback): eight 16 MiB messages of this repository's CHANGELOG.md, repeated, from a client to a server in 1.5 s at level 1 and 5.2 s at level 6, where they took 74 ms uncompressed. Nearly all of it is the compressor: one of those messages, 16.6 MB, goes to 7.2 MB in 121 ms at level 1 and to 6.1 MB in 589 ms at level 6, and inflates in 60 to 67 ms either way - which is why the default is 1, as gorilla/websocket's is. 64 MiB of zeros compressed at level 1, 292,849 bytes, are refused by the decoder at a limit of 1 MiB in 0.13 ms.
**What the other end does wrong** fails the connection as §7.1.7 has it: a close frame with the code that names the fault is sent, the socket is closed, and receive answers a WebSocketError with that code. A frame with RSV2 or RSV3 set, or RSV1 when permessage-deflate was not agreed, on a control frame or on a continuation (§6.1), or a reserved opcode, a client's frame without a mask or a server's with one (§5.1), a control frame that is fragmented or longer than 125 bytes (§5.5), a continuation with no message to continue or a new message before the last one's final frame (§5.4), a 64-bit length with its high bit set (§5.2), a close of one byte or with a code no peer may send (§7.4) are 1002. Text that is not UTF-8 - checked as the fragments arrive, a character split between two of them being whole, and an unfinished one at the end not - and a close reason that is not, are 1007. A message past max_message bytes (16 MiB unless the caller says otherwise) is 1009, refused on the length its frame declares, before any of it is read or anything is made to hold it: a frame that claims 2^62 bytes costs the 14 bytes of its head. A connection that fails below the protocol - SocketError, Cancelled, or the peer gone without a close frame (1006) - is closed too, since a frame read part way leaves nothing after it that can be trusted to be a frame.
**What a program does wrong** is a panic, before anything is written: text that is not UTF-8 handed to send, a ping past 125 bytes or a close reason past 123, a close code that is not one an endpoint sends (1000 to 1003, 1007 to 1014, 3000 to 4999), a caller's header that is the handshake's own, a protocol name that is not a token.
Masks come from Random::Secure, four fresh bytes a frame (§5.3 asks that they not be predictable), and are applied, like the UTF-8 check's run over ASCII, eight bytes a time. A message in one frame is read straight into the string receive answers; fragments into one buffer that grows as they come and is let go after the message. Measured here (--release, one machine, both ends in one process over loopback): 128 MiB as eight 16 MiB messages from a client to a server, masked and unmasked, in 70 ms as binary and 85 ms as text.
Frames are written whole, one at a time per connection: a task that sends while another receives (whose pongs and close replies are writes too) waits, a millisecond at a time, for the frame being written to finish. The runtime refuses a second write to a socket another task's write is parked on ("two fibers writing one fd"), and a frame let in between would have split the first.
Not here: extensions other than permessage-deflate, TLS, and sending a message in fragments. bench/std_websocket_exercise.sh runs both halves against each other, against hand-made frames, and against Python and Node peers, compressed and not.
Messagestruct
One whole message: its bytes, and whether it came as text (UTF-8, checked) or binary.
pub struct Message
def binary? : Bool
def data : String
def initialize(data : String, binary : Bool)
def text? : Bool
endPerMessageDeflatestruct
The permessage-deflate (RFC 7692) a handshake agreed to: whether each end starts every message it compresses with an empty window (*_no_context_takeover), and how far back each end's matches may reach, as a base-2 logarithm (*_max_window_bits, 15 where the answer did not say). server_ is what the server sends, client_ what the client sends. level is this end's own, not agreed: the DEFLATE level its messages are compressed at, std/compress's 0 (stored) to 9, 1 by default. A window outside 8 to 15, or a level outside 0 to 9, is a panic.
pub struct PerMessageDeflate
def client_max_window_bits : Int32
def client_no_context_takeover : Bool
def initialize(server_no_context_takeover : Bool = false, client_no_context_takeover : Bool = false, server_max_window_bits : Int32 = 15, client_max_window_bits : Int32 = 15, level : Int32 = 1)
def level : Int32
def server_max_window_bits : Int32
def server_no_context_takeover : Bool
endWebSocketclass
One end of a connection whose handshake is done: what upgrade hands its block and connect answers. WebSocket.new(socket, leftover, client) takes over a socket some other code did the handshake on; leftover is what arrived after the handshake's head, read first, and deflate the permessage-deflate that handshake agreed to.
pub class WebSocket
# The `Sec-WebSocket-Accept` that answers *key* (§4.2.2): the base64 of
# the SHA-1 of the key and the GUID.
def self.accept(key : String) : String
# The protocol a server's 101 agreed to (nil for none), or why the
# answer is not one to the handshake *key* offering *protocols* (§4.1).
protected def self.agreed(up : Upgraded, key : String, protocols : Array(String)) : String | Nil | WebSocketError
# The permessage-deflate a server's `Sec-WebSocket-Extensions`
# (*value*) agreed to, nil for none, or why it is not an answer to an
# offer of `permessage-deflate; client_max_window_bits` - or, without
# *offered*, to no offer (RFC 7692 §5, §7). This end compresses at
# *level*.
protected def self.answered(value : String | ::Nil, offered : Bool, level : Int32) : PerMessageDeflate | Nil | WebSocketError
# Past the spaces and tabs from *i*.
protected def self.blanks(p : Pointer(UInt8), n : Int32, i : Int32) : Int32
# The close handshake from this end: a close with *code* and *reason*,
# then the peer's, read past whatever messages it sent first, for up
# to ten seconds. Nil once it is done, or when the connection was
# already closed.
def close(code : Int32 = 1000, reason : String = "") : Nil | Cancelled | SocketError | WebSocketError
# The code the connection closed with: the peer's close's (1005 when it
# had none), the one this end failed it with, or 1006 when it ended
# without a close. Nil while it is open.
def close_code : Int32 | ::Nil
# Whether *code* is one a close frame may carry (§7.4): 1000 to 1003,
# 1007 to 1014 (1012 to 1014 as IANA's registry added them), and the
# ranges for libraries and applications. 1004 is reserved, and 1005,
# 1006 and 1015 are the codes an endpoint reports, never sends.
def self.close_code?(code : Int32) : Bool
protected def self.close_payload(code : Int32, reason : String) : String
# The reason the peer's close carried; "" for none.
def close_reason : String
# Whether the connection is closed: after the close handshake, after a
# failure, and after `close`.
def closed? : Bool
# The client's half: the handshake to a `ws://` URL, and the connection
# it opens. *headers* go with the request (an `Origin`, a cookie); the
# handshake's own are refused. *protocols* are offered in their order.
# *proxy* is `HTTP.upgrade`'s: an `http://` proxy's URL, through whose
# CONNECT tunnel the handshake goes, `ws://` and `wss://` alike; the
# environment's is `HTTPProxy.from_env` of the URL with `http` for `ws`
# and `https` for `wss`, the schemes whose variables it reads. With
# *deflate* permessage-deflate is offered, and messages are compressed
# at *deflate_level*.
def self.connect(url : String, headers : Hash(String, String) = {} of String => String, protocols : Array(String) = [] of String, max_message : Int32 = MAX_MESSAGE, proxy : String | ::Nil = nil, deflate : Bool = true, deflate_level : Int32 = 1) : WebSocket | Cancelled | SocketError | HTTPError | WebSocketError
# The permessage-deflate agreed; nil for none.
def deflate : PerMessageDeflate | ::Nil
# What a permessage-deflate offer's or answer's *params* say (RFC 7692
# §7.1), or why they are not what one may say: each of the four at most
# once, the two `no_context_takeover`s without a value, the two
# `max_window_bits` with 8 to 15 in decimal - and `client_max_window_bits`
# without one in an offer, as a client says it can be held to one.
protected def self.deflate_terms(params : Array(::Tuple(String, String | ::Nil)), answer : Bool) : DeflateTerms | String
# *value* as a list of extensions (RFC 6455 §9.1): each a token, with
# parameters after `;`, a parameter's value a token or a quoted string
# that is one once unescaped. Why it is not one, otherwise.
protected def self.extensions(value : String) : Array(Extension) | String
def initialize(socket : IyiSocket | TlsStream, leftover : String, client : Bool, protocol : String | ::Nil = nil, max_message : Int32 = MAX_MESSAGE, deflate : PerMessageDeflate | ::Nil = nil)
# Whether *key* is what §4.1 has a client send: 16 bytes in base64,
# which is 22 characters of the alphabet and `==`.
protected def self.key?(key : String) : Bool
# *count* bytes at *p* XORed with *key*'s four bytes in turn, from the
# first (§5.3): eight bytes a time, then one.
protected def self.mask(p : Pointer(UInt8), count : Int32, key : UInt32) : Nil
# A ping, which the peer answers with a pong `receive` reads past.
def ping(data : String = "") : Nil | Cancelled | SocketError | WebSocketError
def protocol : String | ::Nil
# The next whole message, or nil when the connection is closed: the
# close handshake is done, or the connection was already closed.
def receive : Message | Nil | Cancelled | SocketError | WebSocketError
# Why *req* is not a handshake this server answers 101, as the answer;
# nil when it is one.
protected def self.refusal(req : Request) : Response | ::Nil
# *text* as one text message. It must be UTF-8.
def send(text : String) : Nil | Cancelled | SocketError | WebSocketError
# *data* as one binary message.
def send_binary(data : String) : Nil | Cancelled | SocketError | WebSocketError
# The server's half of the handshake, answering *req*: a 101 whose
# block is handed the connection, or the 400 or 426 that says why the
# request is not a handshake this server takes. *protocols* are the
# subprotocols the server speaks, in its order of preference. With
# *deflate* the first permessage-deflate offer this end can keep is
# accepted, and messages are compressed at *deflate_level*.
def self.upgrade(req : Request, protocols : Array(String) = [] of String, max_message : Int32 = MAX_MESSAGE, deflate : Bool = true, deflate_level : Int32 = 1, &handler : (WebSocket -> Nil)) : Response
# The UTF-8 check's state after the *count* bytes at *p*, from *state*:
# 0 between characters; the continuation bytes still owed and the range
# the next must fall in (§3 of RFC 3629: no overlong forms, no
# surrogates, nothing past U+10FFFF) packed into one number otherwise;
# -1 once a byte is wrong. A text message is whole when it ends at 0.
protected def self.utf8_scan(p : Pointer(UInt8), count : Int32, state : Int32) : Int32
# 8 to 15 for those numbers written in decimal, without a leading zero;
# 0 for anything else.
protected def self.window_bits(value : String) : Int32
# The end of the word from *i*: up to a blank or one of `,;="`.
protected def self.word(p : Pointer(UInt8), n : Int32, i : Int32) : Int32
endWebSocketErrorstruct
What the other end did that RFC 6455 does not allow, or a handshake that is not one, as a value: reason is the sentence, code the close code it stands for (1002 a protocol error, 1007 bytes that are not UTF-8, 1009 a message too large, 1006 a connection that ended without a close). message says it is WebSocket's.
pub struct WebSocketError
def code : Int32
def initialize(reason : String, code : Int32 = 1002)
def reason : String
def to_s : String
endError for WebSocketErrorimpl
impl Error for WebSocketError
def message : String
end
WebSocket (RFC 6455) over std/http: the server's handshake, the client's, and the frames after them, written in iyi.