text
Everyday string tools that core doesn't have:
- case conversion between identifier styles
- Levenshtein distance and "did you mean" suggestions
- terminal display width, with padding and truncation that respect it
- POSIX shell-word splitting and quoting
- word wrapping, indent and dedent
- a Myers line diff with unified (
diff -u) output
Each output from unified was checked by applying it with patch(1).
Effects: none.
fn main(args: [str]): int! {
print(text.snake("XMLHttpRequest")) // xml_http_request
print(text.camel("user_id")) // userId
print(text.title("the-quick_brown fox")) // The Quick Brown Fox
let cmds = ["status", "stash", "commit", "push"]
match text.closest("stauts", cmds) {
some(c) => print("unknown command; did you mean `{c}`?")
none => print("unknown command")
}
let argv = text.shell_words("grep -n 'hello world' \"my file.txt\"")?
print(argv) // ["grep", "-n", "hello world", "my file.txt"]
print(text.join(["rm", "it's here"])) // rm 'it'"'"'s here'
print(text.width("日本語")) // 6 columns
print(text.pad_right("日本", 6) + "|") // "日本 |"
print(text.truncate("a very long title", 10)) // a very lo…
for line in text.wrap("the quick brown fox jumps over the lazy dog", 16) { print(line) }
let patch = text.unified("a\nb\nc\n", "a\nB\nc\n")
print("--- a/notes.txt\n+++ b/notes.txt\n" + patch) // @@ -1,3 +1,3 @@ ...
0
}
API
Case (ASCII letters only change case; other chars are kept):
words(s): [str]: how the case functions split their input. Every char that isn't a letter or digit separates words. A word also ends before an upper-case letter that follows a lower-case letter, a digit or a non-ASCII letter (fooBar,v2Api). Inside an acronym, the last capital starts a new word when lower case follows it (XMLHttp→XML,Http). Digits stay with the word they follow (utf8Decode→utf8,Decode).snake,kebab,camel,pascal,title,constant(SCREAMING_SNAKE), andcapitalize(word)
Similarity:
distance(a, b): int: Levenshtein distance over chars. It is case-sensitive and takes O(len(a)·len(b)) time.similarity(a, b): float:1 - distance / max(len), from 0.0 to 1.0.suggest(s, options, n): [str]andclosest(s, options): str?: options within distancemax(len(s), 3) / 3ofs, ignoring ASCII case, best first. Ties keep the input order. This is the cutoff compilers use for "did you mean".
Display width:
char_width(c): int(0, 1 or 2),width(s): int: terminal columns.pad_right(s, w),pad_left(s, w): pad with spaces towcolumns."{s:<10}"counts chars, which misaligns CJK and emoji.truncate(s, w)cutssto at mostwcolumns, ending with…when it cuts.truncate(s, w, ellipsis)uses your own ending. A wide char is never split.
Shell words (POSIX sh quoting, no expansion):
shell_words(s): [str]!ShellErr: whitespace separates words.'…'is literal. Inside"…", a backslash escapes only$ ` " \and newline. Outside quotes,\xgivesx, and a backslash before a newline joins the lines. A#at the start of a word starts a comment.$VAR, globs,~and| ; &stay as ordinary text.quote(s): str: returnssunchanged if it's safe, otherwise single-quoted.shell_words(quote(s)) == [s]holds for every string.join(words): stris the inverse ofshell_words.enum ShellErr { UnclosedQuote(at: int), TrailingBackslash(at: int) },message(e)
Layout:
wrap(s, w): [str]breaks text greedily at spaces by display width. Existing line breaks and blank lines are kept. Runs of spaces collapse, and leading indentation is dropped. A word wider thanwis split.fill(s, w): striswrap(...).join("\n").indent(s, prefix)prefixes the non-blank lines.dedent(s)removes the leading spaces and tabs that all non-blank lines share, and empties whitespace-only lines.
Diff:
enum Edit { Keep(line), Del(line), Ins(line) }.diff(old: [str], new: [str]): [Edit]returns a shortest edit script, using Myers' O(ND) algorithm after trimming the common prefix and suffix, with deletions before insertions in each change.unified(old: str, new: str): strreturnsdiff -uhunks with 3 lines of context, or""when the texts are equal.unified(old, new, context)sets the context. Only the hunks are returned, so prepend--- a/x\n+++ b/x\nyourself. A missing final newline shows up as\ No newline at end of file, as git prints it.
Limits
- Width is an approximation per code point, not per grapheme. It counts 0 for controls, combining marks, zero-width joiners and spaces, variation selectors and emoji skin-tone modifiers. It counts 2 for East Asian Wide and Fullwidth chars (CJK, Hangul, kana, fullwidth forms) and emoji with default emoji presentation. Everything else counts 1. What it gets wrong:
- ZWJ sequences (
👨👩👧counts 6, a terminal draws 2) and flags made of regional indicators (2 chars, so 2 columns, which may differ by terminal). - Text-default symbols turned into emoji with U+FE0F (
❤️counts 1). - East Asian Ambiguous chars (
±,Ω, box drawing) count 1, which is right outside CJK locales. - The tables roughly follow Unicode 15, so newer emoji count 1.
- ZWJ sequences (
- No grapheme segmentation or Unicode normalization.
distanceandtruncatework on code points."é"written aseplus a combining accent counts as 2 chars fordistance, andtruncatecan cut between theeand its accent. - Case mapping is ASCII-only, like core's
upper/lower.snake("Ärger")keeps theÄ. diffuses memory proportional to D² (D = the number of differing lines), so it suits source files and config, not huge, mostly different inputs.