iyi

Standard library, 0.18.0

std/shlex

Shell-style words, read and written the way Python's shlex reads and writes them in POSIX mode: Shlex.split is shlex.split(s, comments), Shlex.quote is shlex.quote and Shlex.join is shlex.join, rule for rule, so a command line a program builds here is the one a Python tool beside it would build, and the one /bin/sh reads back. It is a reader of words, not a shell: there is no expansion of $, ~, * or backquotes, no operators (;, |, && and > are letters of a word, as in shlex.split), and nothing is run.

import std/shlex to use it. Source: src/std/shlex.iyi at v0.18.0.

Splitting. Words are separated by runs of space, tab, carriage return and line feed - those four, as Python's shlex.whitespace - and a word is any run of the following, joined with no space between:

  • a byte that is not one of those, a quote or a backslash, kept;
  • a backslash and the byte after it, the byte kept and the backslash dropped (POSIX 2.2.1). A backslash before a line feed keeps the line feed, as Python keeps it, where sh would drop both: shlex.split reads one line of words, not a script's continuation lines;
  • '...', everything up to the next ' kept as it stands (POSIX 2.2.2): no backslash means anything inside single quotes;
  • "...", where a backslash before " or \ stands for that byte and a backslash before anything else is kept with it. POSIX 2.2.3 also lets a backslash take $, ` ` and a line feed inside double quotes; Python's shlex does not (its escapedquotes is " and its escape is \` alone), and this follows Python, the oracle the gate holds it to.

A word that is all quotes is still a word: '' and "" are one empty word each, a'' is a. With comments: true a # that starts a word or falls inside an unquoted one ends the word and everything up to the next line feed is skipped, as Python skips it (Python ends a#b at a where sh would read a#b - again Python is followed). Without it, # is a letter. Bytes are read one at a time; every byte with meaning is ASCII, so UTF-8 text passes through whole and splits as Python's characters do.

What is wrong with the text is a ShlexError value naming the byte where the trouble starts: a quote with no closing quote ("No closing quotation" in Python), or a backslash with nothing after it, outside or inside double quotes ("No escaped character"). There is no misuse to panic over - every string is a text to read or an argument to quote.

Quoting. An argument made only of ASCII letters, digits and %+,-./:=@_ is written as it is, an empty one as '', and any other in single quotes with each ' inside written '"'"' - Python 3.14's quote byte for byte, which sh reads back as the argument whatever it holds but a NUL, which no argument can hold. join is the quoted arguments with one space between, and split(join(args)) == args for every array of arguments; the gate checks that on a generated corpus, and checks /bin/sh reading join's output as printf arguments gives the arguments back.

Shlexstruct

POSIX shell words: split, quote and join, as Python's shlex.

std/shlexiyi doc src/std/shlex.iyi
pub struct Shlex
  def initialize
  # The arguments quoted and joined with spaces: a command line that
  # `split` and `sh` read back as *args*.
  def self.join(args : Array(String)) : String
  # *arg* as one shell word: as it is when every byte is a letter, a
  # digit or one of `%+,-./:=@_`, `''` when empty, else single-quoted.
  def self.quote(arg : String) : String
  # The words of *text*, quotes and backslashes taken off as `sh` takes
  # them; with *comments*, `#` to the end of the line is skipped.
  def self.split(text : String, comments : ::Bool = false) : Array(String) | ShlexError
end

ShlexErrorstruct

Why a text does not split: an unclosed quote or a final backslash. offset is the byte where the quote or the backslash stands.

std/shlexiyi doc src/std/shlex.iyi
pub struct ShlexError
  def initialize(message : String, offset : Int32)
  def offset : Int32
  def to_s : String
end

Error for ShlexErrorimpl

std/shlexiyi doc src/std/shlex.iyi
impl Error for ShlexError
  def message : String
end