std/idna
import std/idna to use it. Source: src/std/idna.iyi at v0.18.0.
Punycode.encode writes any string as the ASCII Punycode of its code points, the basic ones first and a - after them when there are any, exactly as RFC 3492 section 6.3 does ("bücher" is "bcher-kva"); Punycode.decode undoes it and answers a PunycodeError for input that is not Punycode: a byte that is not ASCII, a byte in the extended part that is not a base-36 digit, input that ends inside a number, a number that overflows, and a code point past U+10FFFF or a surrogate, which no UTF-8 string holds. The arithmetic is 64-bit with the overflow checks of section 6.4, so whatever encode writes decode reads back, however long; Python's codec has unbounded integers and refuses the same inputs another way, as a code point out of range or a number cut short.
IDNA.to_ascii turns a host name into the one DNS looks up: "bücher.example" is "xn--bcher-kva.example". IDNA.to_unicode goes the other way. Both split at . and the three other full stops UTS #46 and Python's codecs treat as one (U+3002, U+FF0E, U+FF61), keep one trailing dot, and check every label whichever form it came in:
* an ASCII label is lower-cased, and holds letters, digits, `-` and
`_` (the underscore labels of SRV and DMARC records, which RFC 5890
section 2.3.1 leaves outside IDNA rather than forbids);
* no label begins or ends with a hyphen, and only an `xn--` label has
`--` at its third and fourth characters (RFC 5891 section 4.2.3.1);
* an `xn--` label is an A-label: it decodes, what it decodes to is not
all ASCII, and that passes the checks a U-label does (section 5.4);
* a label with other characters is a U-label: in NFC (section 5.3),
not beginning with a combining mark (4.2.3.2), and every code point
PVALID under RFC 5892, or CONTEXTO with its context rule met
(appendix A.3 to A.9). The RFC 5892 property is derived here, by the
RFC's own rules (section 3), from the general categories, case
folding and normalization `std/unicode` carries, rather than read
from a table: `IDNA.property` answers it;
* a label is at most 63 octets in its ASCII form, the name at most
253 without its trailing dot (RFC 1034 section 3.1).What fails is an IdnaError naming the label and the code point.
What is not done, plainly:
* No UTS #46 mapping. A label is checked as it is given, its ASCII
letters lower-cased and nothing else: `"BÜCHER.de"`, a full-width
`"ｅｘａｍｐｌｅ"` and `"fi.example"` are refused, naming the code
point UTS #46 would have mapped and what case folding and NFKC make
of it, where a browser maps them first. Nor is a name normalized:
`"u" + U+0308` is refused as not NFC rather than composed to `ü`.
* No transitional processing: `ß` and `ς` are themselves, as IDNA2008
and UTS #46 non-transitional have them (`"faß.de"` is
`"xn--fa-hia.de"`; IDNA2003, Python's `idna` codec, made it
`"fass.de"`).
* CONTEXTJ (ZWNJ U+200C, ZWJ U+200D): their rules need the Joining_Type
and Virama properties, which `std/unicode` does not carry, so a label
holding either is refused, including the ones those rules accept.
* The Bidi rule of RFC 5893 is not checked: it needs Bidi_Class, which
`std/unicode` does not carry either. A right-to-left label that RFC
5893 refuses (`"0א"` beginning with a European digit) is accepted.
* Symbols and emoji are DISALLOWED in IDNA2008 and refused (`"☃.net"`,
even written `"xn--n3h.net"`), where a WHATWG URL parser takes them.Pure computation: no platform, no clock, no randomness; std/uri builds URI.ascii_host on it.
IDNAstruct
Host names to and from the ASCII form DNS looks up, IDNA2008's way, label by label; the header says what is checked and what is not.
pub struct IDNA
def initialize
# RFC 5892's derived property of *char* (section 3), computed in the
# order the RFC gives: the exceptions (section 2.6), unassigned code
# points, the LDH characters, the joiners, and then a letter, digit or
# mark that neither case folding nor NFKC changes, that is not default
# ignorable, white space or a noncharacter, and not in the ignorable
# blocks or old Hangul jamo. Everything else is Disallowed.
def self.property(char : Char) : IdnaProperty
# The name as DNS looks it up: every U-label as its `xn--` A-label,
# every ASCII label lower-cased.
def self.to_ascii(name : String) : String | IdnaError
# The name as a person reads it: every A-label decoded, every ASCII
# label lower-cased; checked as `to_ascii` checks it.
def self.to_unicode(name : String) : String | IdnaError
# `U+00FC`: four hex digits at least, upper case.
def self.u_plus(cp : Int64) : String
endIdnaErrorstruct
What IDNA.to_ascii, IDNA.to_unicode and URI.ascii_host answer for a name they cannot carry: the sentence names the label and why.
pub struct IdnaError
def initialize(message : String)
def to_s : String
endIdnaPropertyenum
The derived property of RFC 5892 section 2: whether a code point may appear in a U-label (PValid), only in a context (ContextJ, ContextO), not at all (Disallowed), or is unassigned in the Unicode std/unicode holds.
pub enum IdnaProperty : Int32
PValid = 0
ContextJ = 1
ContextO = 2
Disallowed = 3
Unassigned = 4
endPunycodestruct
RFC 3492's bootstring with its Punycode parameters (section 5).
pub struct Punycode
# The string *text* is the Punycode of (section 6.2), or the reason it
# is none. Digits are read in either case; basic code points are kept
# in the case they were written.
def self.decode(text : String) : String | PunycodeError
# The Punycode of *text*'s code points (section 6.3). Total: every
# string has one. Bytes that are not UTF-8 are read as `each_char` reads
# them, U+FFFD each.
def self.encode(text : String) : String
def initialize
endPunycodeErrorstruct
What Punycode.decode answers for input that is not Punycode.
pub struct PunycodeError
def initialize(message : String)
def to_s : String
endError for PunycodeErrorimpl
impl Error for PunycodeError
def message : String
endError for IdnaErrorimpl
impl Error for IdnaError
def message : String
end
Internationalized domain names: Punycode (RFC 3492) and host names carried to and from their ASCII form label by label (IDNA2008: RFC 5890, 5891 and 5892).