6.4 – String Manipulation

This library provides generic functions for string manipulation, such as finding and extracting substrings, and pattern matching. When indexing a string in Lua, the first character is at position 1 (not at 0, as in C). Indices are allowed to be negative and are interpreted as indexing backwards, from the end of the string. Thus, the last character is at position -1, and so on.

The string library provides all its functions inside the table string. It also sets a metatable for strings where the __index field points to the string table. Therefore, you can use the string functions in object-oriented style. For instance, string.byte(s,i) can be written as s:byte(i).

The string library assumes one-byte character encodings.

string.byte (s [, i [, j]])

Returns the internal numeric codes of the characters s[i], s[i+1], …, s[j]. The default value for i is 1; the default value for j is i. These indices are corrected following the same rules of function string.sub.

Numeric codes are not necessarily portable across platforms.

string.char (···)

Receives zero or more integers. Returns a string with length equal to the number of arguments, in which each character has the internal numeric code equal to its corresponding argument.

Numeric codes are not necessarily portable across platforms.

string.dump (function [, strip])

Returns a string containing a binary representation (a binary chunk) of the given function, so that a later load on this string returns a copy of the function (but with new upvalues). If strip is a true value, the binary representation may not include all debug information about the function, to save space.

Functions with upvalues have only their number of upvalues saved. When (re)loaded, those upvalues receive fresh instances. (See the load function for details about how these upvalues are initialized. You can use the debug library to serialize and reload the upvalues of a function in a way adequate to your needs.)

string.find (s, pattern [, init [, plain]])

Looks for the first match of pattern (see §6.4.1) in the string s. If it finds a match, then find returns the indices of s where this occurrence starts and ends; otherwise, it returns fail. A third, optional numeric argument init specifies where to start the search; its default value is 1 and can be negative. A value of true as a fourth, optional argument plain turns off the pattern matching facilities, so the function does a plain “find substring” operation, with no characters in pattern being considered magic.

If the pattern has captures, then in a successful match the captured values are also returned, after the two indices.

string.format (formatstring, ···)

Returns a formatted version of its variable number of arguments following the description given in its first argument, which must be a string. The format string follows the same rules as the ISO C function sprintf. The only differences are that the conversion specifiers and modifiers *, h, L, l, and n are not supported and that there is an extra specifier, q.

The specifier q formats booleans, nil, numbers, and strings in a way that the result is a valid constant in Lua source code. Booleans and nil are written in the obvious way (true, false, nil). Floats are written in hexadecimal, to preserve full precision. A string is written between double quotes, using escape sequences when necessary to ensure that it can safely be read back by the Lua interpreter. For instance, the call

     string.format('%q', 'a string with "quotes" and \n new line')

may produce the string:

     "a string with \"quotes\" and \
      new line"

This specifier does not support modifiers (flags, width, length).

The conversion specifiers A, a, E, e, f, G, and g all expect a number as argument. The specifiers c, d, i, o, u, X, and x expect an integer. When Lua is compiled with a C89 compiler, the specifiers A and a (hexadecimal floats) do not support modifiers.

The specifier s expects a string; if its argument is not a string, it is converted to one following the same rules of tostring. If the specifier has any modifier, the corresponding string argument should not contain embedded zeros.

The specifier p formats the pointer returned by lua_topointer. That gives a unique string identifier for tables, userdata, threads, strings, and functions. For other values (numbers, nil, booleans), this specifier results in a string representing the pointer NULL.

string.gmatch (s, pattern [, init])

Returns an iterator function that, each time it is called, returns the next captures from pattern (see §6.4.1) over the string s. If pattern specifies no captures, then the whole match is produced in each call. A third, optional numeric argument init specifies where to start the search; its default value is 1 and can be negative.

As an example, the following loop will iterate over all the words from string s, printing one per line:

     s = "hello world from Lua"
     for w in string.gmatch(s, "%a+") do
       print(w)
     end

The next example collects all pairs key=value from the given string into a table:

     t = {}
     s = "from=world, to=Lua"
     for k, v in string.gmatch(s, "(%w+)=(%w+)") do
       t[k] = v
     end

For this function, a caret ‘^‘ at the start of a pattern does not work as an anchor, as this would prevent the iteration.

string.gsub (s, pattern, repl [, n])

Returns a copy of s in which all (or the first n, if given) occurrences of the pattern (see §6.4.1) have been replaced by a replacement string specified by repl, which can be a string, a table, or a function. gsub also returns, as its second value, the total number of matches that occurred. The name gsub comes from Global SUBstitution.

If repl is a string, then its value is used for replacement. The character % works as an escape character: any sequence in repl of the form %*d*, with d between 1 and 9, stands for the value of the d-th captured substring; the sequence %0 stands for the whole match; the sequence %% stands for a single %.

If repl is a table, then the table is queried for every match, using the first capture as the key.

If repl is a function, then this function is called every time a match occurs, with all captured substrings passed as arguments, in order.

In any case, if the pattern specifies no captures, then it behaves as if the whole pattern was inside a capture.

If the value returned by the table query or by the function call is a string or a number, then it is used as the replacement string; otherwise, if it is false or nil, then there is no replacement (that is, the original match is kept in the string).

Here are some examples:

     x = string.gsub("hello world", "(%w+)", "%1 %1")
     --> x="hello hello world world"
     x = string.gsub("hello world", "%w+", "%0 %0", 1)
     --> x="hello hello world"
     x = string.gsub("hello world from Lua", "(%w+)%s*(%w+)", "%2 %1")
     --> x="world hello Lua from"
     x = string.gsub("home = $HOME, user = $USER", "%$(%w+)", os.getenv)
     --> x="home = /home/roberto, user = roberto"
     x = string.gsub("4+5 = $return 4+5$", "%$(.-)%$", function (s)
           return load(s)()
         end)
     --> x="4+5 = 9"
     local t = {name="lua", version="5.4"}
     x = string.gsub("$name-$version.tar.gz", "%$(%w+)", t)
     --> x="lua-5.4.tar.gz"

string.len (s)

Receives a string and returns its length. The empty string "" has length 0. Embedded zeros are counted, so "a\000bc\000" has length 5.

string.lower (s)

Receives a string and returns a copy of this string with all uppercase letters changed to lowercase. All other characters are left unchanged. The definition of what an uppercase letter is depends on the current locale.

string.match (s, pattern [, init])

Looks for the first match of the pattern (see §6.4.1) in the string s. If it finds one, then match returns the captures from the pattern; otherwise it returns fail. If pattern specifies no captures, then the whole match is returned. A third, optional numeric argument init specifies where to start the search; its default value is 1 and can be negative.

string.pack (fmt, v1, v2, ···)

Returns a binary string containing the values v1, v2, etc. serialized in binary form (packed) according to the format string fmt (see §6.4.2).

string.packsize (fmt)

Returns the size of a string resulting from string.pack with the given format. The format string cannot have the variable-length options ‘s‘ or ‘z‘ (see §6.4.2).

string.rep (s, n [, sep])

Returns a string that is the concatenation of n copies of the string s separated by the string sep. The default value for sep is the empty string (that is, no separator). Returns the empty string if n is not positive.

(Note that it is very easy to exhaust the memory of your machine with a single call to this function.)

string.reverse (s)

Returns a string that is the string s reversed.

string.sub (s, i [, j])

Returns the substring of s that starts at i and continues until j; i and j can be negative. If j is absent, then it is assumed to be equal to -1 (which is the same as the string length). In particular, the call string.sub(s,1,j) returns a prefix of s with length j, and string.sub(s, -i) (for a positive i) returns a suffix of s with length i.

If, after the translation of negative indices, i is less than 1, it is corrected to 1. If j is greater than the string length, it is corrected to that length. If, after these corrections, i is greater than j, the function returns the empty string.

string.unpack (fmt, s [, pos])

Returns the values packed in string s (see string.pack) according to the format string fmt (see §6.4.2). An optional pos marks where to start reading in s (default is 1). After the read values, this function also returns the index of the first unread byte in s.

string.upper (s)

Receives a string and returns a copy of this string with all lowercase letters changed to uppercase. All other characters are left unchanged. The definition of what a lowercase letter is depends on the current locale.

6.4.1 – Patterns

Patterns in Lua are described by regular strings, which are interpreted as patterns by the pattern-matching functions string.find, string.gmatch, string.gsub, and string.match. This section describes the syntax and the meaning (that is, what they match) of these strings.

Character Class:

A character class is used to represent a set of characters. The following combinations are allowed in describing a character class:

• x: (where x is not one of the magic characters ^$()%.[]*+-?) represents the character x itself.
• .: (a dot) represents all characters.
• %a: represents all letters.
• %c: represents all control characters.
• %d: represents all digits.
• %g: represents all printable characters except space.
• %l: represents all lowercase letters.
• %p: represents all punctuation characters.
• %s: represents all space characters.
• %u: represents all uppercase letters.
• %w: represents all alphanumeric characters.
• %x: represents all hexadecimal digits.
• %*x*: (where x is any non-alphanumeric character) represents the character x. This is the standard way to escape the magic characters. Any non-alphanumeric character (including all punctuation characters, even the non-magical) can be preceded by a ‘%‘ to represent itself in a pattern.

• [*set*]: represents the class which is the union of all characters in set. A range of characters can be specified by separating the end characters of the range, in ascending order, with a ‘-‘. All classes %x described above can also be used as components in set. All other characters in set represent themselves. For example, [%w_] (or [_%w]) represents all alphanumeric characters plus the underscore, [0-7] represents the octal digits, and [0-7%l%-] represents the octal digits plus the lowercase letters plus the ‘-‘ character.

  You can put a closing square bracket in a set by positioning it as the first character in the set. You can put a hyphen in a set by positioning it as the first or the last character in the set. (You can also use an escape for both cases.)

  The interaction between ranges and classes is not defined. Therefore, patterns like [%a-z] or [a-%%] have no meaning.

• [^*set*]: represents the complement of set, where set is interpreted as above.

For all classes represented by single letters (%a, %c, etc.), the corresponding uppercase letter represents the complement of the class. For instance, %S represents all non-space characters.

The definitions of letter, space, and other character groups depend on the current locale. In particular, the class [a-z] may not be equivalent to %l.

Pattern Item:

A pattern item can be

• a single character class, which matches any single character in the class;
• a single character class followed by ‘*‘, which matches sequences of zero or more characters in the class. These repetition items will always match the longest possible sequence;
• a single character class followed by ‘+‘, which matches sequences of one or more characters in the class. These repetition items will always match the longest possible sequence;
• a single character class followed by ‘-‘, which also matches sequences of zero or more characters in the class. Unlike ‘*‘, these repetition items will always match the shortest possible sequence;
• a single character class followed by ‘?‘, which matches zero or one occurrence of a character in the class. It always matches one occurrence if possible;
• %*n*, for n between 1 and 9; such item matches a substring equal to the n-th captured string (see below);
• %b*xy*, where x and y are two distinct characters; such item matches strings that start with x, end with y, and where the x and y are balanced. This means that, if one reads the string from left to right, counting +1 for an x and -1 for a y, the ending y is the first y where the count reaches 0. For instance, the item %b() matches expressions with balanced parentheses.
• %f[*set*], a frontier pattern; such item matches an empty string at any position such that the next character belongs to set and the previous character does not belong to set. The set set is interpreted as previously described. The beginning and the end of the subject are handled as if they were the character ‘\0‘.

Pattern:

A pattern is a sequence of pattern items. A caret ‘^‘ at the beginning of a pattern anchors the match at the beginning of the subject string. A ‘$‘ at the end of a pattern anchors the match at the end of the subject string. At other positions, ‘^‘ and ‘$‘ have no special meaning and represent themselves.

Captures:

A pattern can contain sub-patterns enclosed in parentheses; they describe captures. When a match succeeds, the substrings of the subject string that match captures are stored (captured) for future use. Captures are numbered according to their left parentheses. For instance, in the pattern "(a*(.)%w(%s*))", the part of the string matching "a*(.)%w(%s*)" is stored as the first capture, and therefore has number 1; the character matching “.“ is captured with number 2, and the part matching “%s*“ has number 3.

As a special case, the capture () captures the current string position (a number). For instance, if we apply the pattern "()aa()" on the string "flaaap", there will be two captures: 3 and 5.

Multiple matches:

The function string.gsub and the iterator string.gmatch match multiple occurrences of the given pattern in the subject. For these functions, a new match is considered valid only if it ends at least one byte after the end of the previous match. In other words, the pattern machine never accepts the empty string as a match immediately after another match. As an example, consider the results of the following code:

     > string.gsub("abc", "()a*()", print);
     --> 1   2
     --> 3   3
     --> 4   4

The second and third results come from Lua matching an empty string after ‘b‘ and another one after ‘c‘. Lua does not match an empty string after ‘a‘, because it would end at the same position of the previous match.

6.4.2 – Format Strings for Pack and Unpack

The first argument to string.pack, string.packsize, and string.unpack is a format string, which describes the layout of the structure being created or read.

A format string is a sequence of conversion options. The conversion options are as follows:

• <: sets little endian
• >: sets big endian
• =: sets native endian
• ![*n*]: sets maximum alignment to n (default is native alignment)
• b: a signed byte (char)
• B: an unsigned byte (char)
• h: a signed short (native size)
• H: an unsigned short (native size)
• l: a signed long (native size)
• L: an unsigned long (native size)
• j: a lua_Integer
• J: a lua_Unsigned
• T: a size_t (native size)
• i[*n*]: a signed int with n bytes (default is native size)
• I[*n*]: an unsigned int with n bytes (default is native size)
• f: a float (native size)
• d: a double (native size)
• n: a lua_Number
• c*n*: a fixed-sized string with n bytes
• z: a zero-terminated string
• s[*n*]: a string preceded by its length coded as an unsigned integer with n bytes (default is a size_t)
• x: one byte of padding
• X*op*: an empty item that aligns according to option op (which is otherwise ignored)
• ‘ ‘: (space) ignored

(A “[*n*]“ means an optional integral numeral.) Except for padding, spaces, and configurations (options “xX <=>!“), each option corresponds to an argument in string.pack or a result in string.unpack.

For options “!*n*“, “s*n*“, “i*n*“, and “I*n*“, n can be any integer between 1 and 16. All integral options check overflows; string.pack checks whether the given value fits in the given size; string.unpack checks whether the read value fits in a Lua integer. For the unsigned options, Lua integers are treated as unsigned values too.

Any format string starts as if prefixed by “!1=“, that is, with maximum alignment of 1 (no alignment) and native endianness.

Native endianness assumes that the whole system is either big or little endian. The packing functions will not emulate correctly the behavior of mixed-endian formats.

Alignment works as follows: For each option, the format gets extra padding until the data starts at an offset that is a multiple of the minimum between the option size and the maximum alignment; this minimum must be a power of 2. Options “c“ and “z“ are not aligned; option “s“ follows the alignment of its starting integer.

All padding is filled with zeros by string.pack and ignored by string.unpack.