std/string_scanner
import std/string_scanner to use it. Source: src/std/string_scanner.iyi at v0.18.0.
scan matches at the offset and moves past what matched; skip does the same and answers how far it moved; check and matches? look without moving. Their _until forms search forward from the offset instead and take everything up to the end of the match. A pattern is a Regex, a String or a Char compared byte for byte, or for scan, skip and check a count of characters. Each records its match, or that there was none, and [] / []? read the last match's groups, by number or by name; a string, a character or a count is a match with group 0 alone.
A regex is matched at the offset by std/regex's own machine, Regex#match_at: no thread starts after the offset, so a failed scan costs what the pattern read, not the rest of the text. The text before the offset is still the text, as in the other library and in Python's pattern.match(text, pos): ^ and \A mean its start, so /^b/ does not match at offset 1 of ab, and \b sees the byte before the offset.
Offsets are bytes, as every position in iyi is (Regex#match's pos, MatchData#byte_begin, String#[](start, count)), and as the other library keeps its own internally; its offset answers characters, and for ASCII the two are the same number. skip answers bytes too, so offset moves by what skip says. A count given to scan, check, skip and peek is characters, as there: scan(2) on 文字列 is 文字. A character is what String#each_char reads, a byte that begins none being one U+FFFD.
Refused, as mistakes the program made: an offset that is negative, past the end, or inside a character (offset = 1 in é); a negative count; a group the last match does not have, one that took no part in it, and any group when the last scan matched nothing. []? answers nil for those instead. What the scanner reads is never refused: a pattern that does not match is nil, and whether input is malformed is the tokenizer's to say with an error value of its own, as bench/std_string_scanner_exercise does.
Left out: peek_behind, rewind, current_char, previous_byte and beginning_of_line?, which read string and offset in a line each.
StringScannerclass
pub class StringScanner
# Group *n* of the last match, the whole match being 0; a negative *n*
# counts from the last group. No last match, a group it does not have and
# one that took no part are mistakes, and panic with which.
def [](n : Int32) : String
# The group called *name* of the last match.
def [](name : String) : String
# Group *n* of the last match, or nil for any of the reasons `[]` panics.
def []?(n : Int32) : String | ::Nil
# The group called *name* of the last match, or nil.
def []?(name : String) : String | ::Nil
# What `scan` would answer, without moving; the match is still the last.
def check(pattern : Regex | String | Char | Int32) : String | ::Nil
# What `scan_until` would answer, without moving.
def check_until(pattern : Regex | String | Char) : String | ::Nil
# Whether the offset is at the end of the string.
def eos? : Bool
def initialize(string : String)
# `#<StringScanner 5/16 …s ‣is …>`, as the other library writes it: the
# offset, the length, and five characters around the offset, two before
# it where there are, with `‣` at the offset and `…` where the string
# goes on. The two numbers are bytes here, as `offset` is.
def inspect : String
# Whether the last scan, skip, check or match matched.
def matched? : Bool
# How many bytes *pattern* matches at the offset, without moving, or nil.
def matches?(pattern : Regex | String | Char) : Int32 | ::Nil
# The offset, in bytes.
def offset : Int32
# Moves the offset to byte *position*: the end of the string is the
# furthest it goes, and a position inside a character is refused, as
# every read after it would start on half of one.
def offset=(position : Int32) : Int32
# Up to *count* characters from the offset, without moving: fewer at the
# end of the string. The last match is left as it was.
def peek(count : Int32) : String
# Back to the start, with no last match.
def reset : Nil
# The string from the offset on.
def rest : String
# Matches *pattern* at the offset and moves past it, answering what
# matched, or nil without moving.
def scan(pattern : Regex | String | Char | Int32) : String | ::Nil
# Searches for *pattern* from the offset and moves past the match,
# answering everything from the offset to the match's end.
def scan_until(pattern : Regex | String | Char) : String | ::Nil
# `scan`, answering how many bytes the offset moved.
def skip(pattern : Regex | String | Char | Int32) : Int32 | ::Nil
# `scan_until`, answering how many bytes the offset moved.
def skip_until(pattern : Regex | String | Char) : Int32 | ::Nil
# The string being scanned.
def string : String
# On to the end, with no last match.
def terminate : Nil
end
A lexical scanner over one string: an offset that moves forward as patterns match at it, and the last match kept for its groups - the other library's
StringScanner, which is Ruby'sstrscan, for the tokenizer a small language or a wire format needs.