Wildcard searching settings

Wildcard searching is a common text search type. In Manticore it is performed at dictionary level. By default, both plain and RT tables use a dictionary type called dict. In this mode words are stored as they are, so the size of the table is not affected by enabling wildcarding. When a wildcard search is performed, in the dictionary a lookup is made to find all possible expansions of the wildcarded word. This expansion can be problematic in terms of computation at query time in cases where the expanded word can provide lots of expansions or expansions that have huge hitlists. The penalties are higher in case of infixes, where wildcard is added at the start and end of the words. expansion_limit is to be used to avoid such problems.

min_prefix_len

  1. min_prefix_len = length

Minimum word prefix length to index and search. Optional, default is 0 (do not allow prefixes).

Prefixes allow to implement wildcard searching by wordstart* wildcards.

For instance, if you index word “example” with min_prefix_len=3 you will be able to find it by “exa”, “exam”, “examp”, “exampl” prefixes along with the word itself.

Be aware that in case of dict\=crc min_prefix_len will also affect full-text index size as each word expansion will be stored additionally.

Manticore can differentiate perfect word matches from prefix matches and rank the former higher if you conform the following conditions:

Note that either with the dict\=crc mode or with any of the above options disabled, there is no way to differentiate between the prefixes and full words, and thus perfect word matches can’t be ranked higher.

When minimum infix length is set to a positive number, minimum prefix length is always considered 1.

  • SQL
  • JSON
  • PHP
  • Python
  • javascript
  • Java
  • CONFIG

SQL JSON PHP Python javascript Java CONFIG

  1. CREATE TABLE products(title text, price float) min_prefix_len = '3'
  1. POST /cli -d "
  2. CREATE TABLE products(title text, price float) min_prefix_len = '3'"
  1. $index = new \Manticoresearch\Index($client);
  2. $index->setName('products');
  3. $index->create([
  4.             'title'=>['type'=>'text'],
  5.             'price'=>['type'=>'float']
  6.         ],[
  7.             'min_prefix_len' => '3'
  8.         ]);
  1. utilsApi.sql('CREATE TABLE products(title text, price float) min_prefix_len = \'3\'')
  1. res = await utilsApi.sql('CREATE TABLE products(title text, price float) min_prefix_len = \'3\'');
  1. utilsApi.sql("CREATE TABLE products(title text, price float) min_prefix_len = '3'");
  1. table products {
  2.   min_prefix_len = 3
  4.   type = rt
  5.   path = tbl
  6.   rt_field = title
  7.   rt_attr_uint = price
  8. }

min_infix_len

  1. min_infix_len = length

Minimum infix prefix length to index and search. Optional, default is 0 (do not allow infixes), and minimum allowed non-zero value is 2.

Infix length setting enables wildcard searches with term patterns like start*, *end, *middle*, and so on. It also lets you disable too short wildcards if those are too expensive to search for.

Manticore can differentiate perfect word matches from infix matches and rank the former higher if you conform the following conditions:

Note that either with the dict\=crc mode or with any of the above options disabled, there is no way to differentiate between the infixes and full words, and thus perfect word matches can’t be ranked higher.

Infix wildcard search query time can vary greatly, depending on how many keywords the substring will actually expand to. Short and frequent syllables like *in* or *ti* just might expand to way too many keywords, all of which would need to be matched and processed. Therefore, to generally enable substring searches you would set min_infix_len to 2; and to limit the impact from wildcard searches with too short wildcards, you might set it higher.

Infixes must be at least 2 characters long, wildcards like *a* are not allowed for performance reasons.

When minimum infix length is set to a positive number, minimum prefix length is considered 1. For dict word infixing and prefixing cannot be both enabled at the same. For dict and other fields to have prefixes declared with prefix_fields, but it’s forbidden to declare same field in the both lists.

In case of dict=keywords, beside the wildcard * two other wildcard characters can be used:

  • ? can match any(one) character: t?st will match test, but not teast
  • % can match zero or one character : tes% will match tes or test, but not testing
  • SQL
  • JSON
  • PHP
  • Python
  • javascript
  • Java
  • CONFIG

SQL JSON PHP Python javascript Java CONFIG

  1. CREATE TABLE products(title text, price float) min_infix_len = '3'
  1. POST /cli -d "
  2. CREATE TABLE products(title text, price float) min_infix_len = '3'"
  1. $index = new \Manticoresearch\Index($client);
  2. $index->setName('products');
  3. $index->create([
  4.             'title'=>['type'=>'text'],
  5.             'price'=>['type'=>'float']
  6.         ],[
  7.             'min_infix_len' => '3'
  8.         ]);
  1. utilsApi.sql('CREATE TABLE products(title text, price float) min_infix_len = \'3\'')
  1. res = await utilsApi.sql('CREATE TABLE products(title text, price float) min_infix_len = \'3\'');
  1. utilsApi.sql("CREATE TABLE products(title text, price float) min_infix_len = '3'");
  1. table products {
  2.   min_infix_len = 3
  4.   type = rt
  5.   path = tbl
  6.   rt_field = title
  7.   rt_attr_uint = price
  8. }

prefix_fields

  1. prefix_fields = field1[, field2, ...]

List of full-text fields to limit prefix indexing to. Applies to dict=crc only. Optional, default is empty (index all fields in prefix mode).

Because prefix indexing impacts both indexing and searching performance, it might be desired to limit it to specific full-text fields only: for instance, to provide prefix searching through URLs, but not through page contents. prefix_fields specifies what fields will be prefix-indexed; all other fields will be indexed in normal mode. The value format is a comma-separated list of field names.

  • CONFIG

CONFIG

  1. table products {
  2.   prefix_fields = title, name
  3.   min_prefix_len = 3
  4.   dict = crc

infix_fields

  1. infix_fields = field1[, field2, ...]

The list of full-text fields to limit infix indexing to. Applies to dict=crc only. Optional, default is empty (index all fields in infix mode).

Similar to prefix_fields, but lets you limit infix-indexing to given fields.

  • CONFIG

CONFIG

  1. table products {
  2.   infix_fields = title, name
  3.   min_infix_len = 3
  4.   dict = crc

max_substring_len

  1. max_substring_len = length

Maximum substring (either prefix or infix) length to index. Optional, default is 0 (do not limit indexed substrings). Applies to dict only.

By default, substring (either prefix or infix) indexing in the dict⛔ will index all possible substrings as separate keywords. That might result in an overly large full-text index. So this directive lets you limit the impact of substring indexing by skipping too-long substrings (which, chances are, will never get searched for anyway).

For example, a test table of 10,000 blog posts takes this much disk space depending on the settings:

  • 6.4 MB baseline (no substrings)
  • 24.3 MB (3.8x) with min_prefix_len = 3
  • 22.2 MB (3.5x) with min_prefix_len = 3, max_substring_len = 8
  • 19.3 MB (3.0x) with min_prefix_len = 3, max_substring_len = 6
  • 94.3 MB (14.7x) with min_infix_len = 3
  • 84.6 MB (13.2x) with min_infix_len = 3, max_substring_len = 8
  • 70.7 MB (11.0x) with min_infix_len = 3, max_substring_len = 6

So in this test limiting the max substring length saved us 10-15% on the table size.

There is no performance impact associated with substring length when using dict=keywords mode, so this directive is not applicable and intentionally forbidden in that case. If required, you can still limit the length of a substring that you search for in the application code.

  • CONFIG

CONFIG

  1. table products {
  2.   max_substring_len = 12
  3.   min_infix_len = 3
  4.   dict = crc

expand_keywords

  1. expand_keywords = {0|1|exact|star}

Expands keywords with their exact forms (i.e. the forms of the keywords before applying any morphological modifications) and/or stars when possible. The supported values are:

  • 1 - expand to both the exact form and the form with the stars. running will become (running | *running* | =running)
  • exact - augment the keyword with only its exact form. running will become (running | =running)
  • star - augment the keyword by adding * around it. running will become (running | *running*) Optional, default is 0 (do not expand keywords).

Queries against tables with expand_keywords feature enabled are internally expanded as follows: if the table was built with prefix or infix indexing enabled, every keyword gets internally replaced with a disjunction of the keyword itself and a respective prefix or infix (keyword with stars). If the table was built with both stemming and index_exact_words enabled, exact form is also added.

  • SQL
  • JSON
  • PHP
  • Python
  • javascript
  • Java
  • CONFIG

SQL JSON PHP Python javascript Java CONFIG

  1. CREATE TABLE products(title text, price float) expand_keywords = '1'
  1. POST /cli -d "
  2. CREATE TABLE products(title text, price float) expand_keywords = '1'"
  1. $index = new \Manticoresearch\Index($client);
  2. $index->setName('products');
  3. $index->create([
  4.             'title'=>['type'=>'text'],
  5.             'price'=>['type'=>'float']
  6.         ],[
  7.             'expand_keywords' => '1'
  8.         ]);
  1. utilsApi.sql('CREATE TABLE products(title text, price float) expand_keywords = \'1\'')
  1. res = await utilsApi.sql('CREATE TABLE products(title text, price float) expand_keywords = \'1\'');
  1. utilsApi.sql("CREATE TABLE products(title text, price float) expand_keywords = '1'");
  1. table products {
  2.   expand_keywords = 1
  4.   type = rt
  5.   path = tbl
  6.   rt_field = title
  7.   rt_attr_uint = price
  8. }

Expanded queries take naturally longer to complete, but can possibly improve the search quality, as the documents with exact form matches should be ranked generally higher than documents with stemmed or infix matches.

Note that the existing query syntax does not allow to emulate this kind of expansion, because internal expansion works on keyword level and expands keywords within phrase or quorum operators too (which is not possible through the query syntax). Take a look at the examples and how expand_keywords affects the search result weights and how “runsy” is found by “runs” w/o the need to add a star:

  • expand_keywords_enabled
  • expand_keywords_disabled

expand_keywords_enabled expand_keywords_disabled

  1. mysql> create table t(f text) min_infix_len='2' expand_keywords='1' morphology='stem_en';
  2. Query OK, 0 rows affected, 1 warning (0.00 sec)
  4. mysql> insert into t values(1,'running'),(2,'runs'),(3,'runsy');
  5. Query OK, 3 rows affected (0.00 sec)
  7. mysql> select *, weight() from t where match('runs');
  8. +------+---------+----------+
  9. | id   | f       | weight() |
  10. +------+---------+----------+
  11. |    2 | runs    |     1560 |
  12. |    1 | running |     1500 |
  13. |    3 | runsy   |     1500 |
  14. +------+---------+----------+
  15. 3 rows in set (0.01 sec)
  17. mysql> drop table t;
  18. Query OK, 0 rows affected (0.01 sec)
  20. mysql> create table t(f text) min_infix_len='2' expand_keywords='exact' morphology='stem_en';
  21. Query OK, 0 rows affected, 1 warning (0.00 sec)
  23. mysql> insert into t values(1,'running'),(2,'runs'),(3,'runsy');
  24. Query OK, 3 rows affected (0.00 sec)
  26. mysql> select *, weight() from t where match('running');
  27. +------+---------+----------+
  28. | id   | f       | weight() |
  29. +------+---------+----------+
  30. |    1 | running |     1590 |
  31. |    2 | runs    |     1500 |
  32. +------+---------+----------+
  33. 2 rows in set (0.00 sec)
  1. mysql> create table t(f text) min_infix_len='2' morphology='stem_en';
  2. Query OK, 0 rows affected, 1 warning (0.00 sec)
  4. mysql> insert into t values(1,'running'),(2,'runs'),(3,'runsy');
  5. Query OK, 3 rows affected (0.00 sec)
  7. mysql> select *, weight() from t where match('runs');
  8. +------+---------+----------+
  9. | id   | f       | weight() |
  10. +------+---------+----------+
  11. |    1 | running |     1500 |
  12. |    2 | runs    |     1500 |
  13. +------+---------+----------+
  14. 2 rows in set (0.00 sec)
  16. mysql> drop table t;
  17. Query OK, 0 rows affected (0.01 sec)
  19. mysql> create table t(f text) min_infix_len='2' morphology='stem_en';
  20. Query OK, 0 rows affected, 1 warning (0.00 sec)
  22. mysql> insert into t values(1,'running'),(2,'runs'),(3,'runsy');
  23. Query OK, 3 rows affected (0.00 sec)
  25. mysql> select *, weight() from t where match('running');
  26. +------+---------+----------+
  27. | id   | f       | weight() |
  28. +------+---------+----------+
  29. |    1 | running |     1500 |
  30. |    2 | runs    |     1500 |
  31. +------+---------+----------+
  32. 2 rows in set (0.00 sec)

This directive does not affect indexer in any way, it only affects searchd.

expansion_limit

  1. expansion_limit = number

Maximum number of expanded keywords for a single wildcard. Details are here.

Ignoring stop words

Stop words are the words that are skipped during indexing and searching. Typically you’d put most frequent words to the stop words list, because they do not add much value to search results but consume a lot of resources to process.

Stemming is by default applied when parsing stop words file. That might however lead to undesired results. You can turn that off with stopwords_unstemmed.

Small enough files are stored in the table header, see embedded_limit for details.

While stop words are not indexed, they still do affect the keyword positions. For instance, assume that “the” is a stop word, that document 1 contains the line “in office”, and that document 2 contains “in the office”. Searching for “in office” as for an exact phrase will only return the first document, as expected, even though “the” in the second one is skipped as a stop word. That behavior can be tweaked through the stopword_step directive.

stopwords

  1. stopwords=path/to/stopwords/file[ path/to/another/file ...]

Stop word files list (space separated). Optional, default is empty. You can specify several file names, separated by spaces. All the files will be loaded. In the RT mode only absolute paths are allowed.

Stop words file format is simple plain text. The encoding must be UTF-8. File data will be tokenized with respect to charset_table settings, so you can use the same separators as in the indexed data.

Stop word files can either be created manually, or semi-automatically. indexer provides a mode that creates a frequency dictionary of the table, sorted by the keyword frequency, see --buildstops and --buildfreqs switch for details. Top keywords from that dictionary can usually be used as stop words.

  • SQL
  • JSON
  • PHP
  • Python
  • javascript
  • Java
  • CONFIG

SQL JSON PHP Python javascript Java CONFIG

  1. CREATE TABLE products(title text, price float) stopwords = '/usr/local/manticore/data/stopwords.txt /usr/local/manticore/data/stopwords-ru.txt /usr/local/manticore/data/stopwords-en.txt'
  1. POST /cli -d "
  2. CREATE TABLE products(title text, price float) stopwords = '/usr/local/manticore/data/stopwords.txt stopwords-ru.txt stopwords-en.txt'"
  1. $index = new \Manticoresearch\Index($client);
  2. $index->setName('products');
  3. $index->create([
  4.             'title'=>['type'=>'text'],
  5.             'price'=>['type'=>'float']
  6.         ],[
  7.             'stopwords' => '/usr/local/manticore/data/stopwords.txt stopwords-ru.txt stopwords-en.txt'
  8.         ]);
  1. utilsApi.sql('CREATE TABLE products(title text, price float) stopwords = \'/usr/local/manticore/data/stopwords.txt /usr/local/manticore/data/stopwords-ru.txt /usr/local/manticore/data/stopwords-en.txt\'')
  1. res = await utilsApi.sql('CREATE TABLE products(title text, price float) stopwords = \'/usr/local/manticore/data/stopwords.txt /usr/local/manticore/data/stopwords-ru.txt /usr/local/manticore/data/stopwords-en.txt\'');
  1. utilsApi.sql("CREATE TABLE products(title text, price float) stopwords = '/usr/local/manticore/data/stopwords.txt /usr/local/manticore/data/stopwords-ru.txt /usr/local/manticore/data/stopwords-en.txt'");
  1. table products {
  2.   stopwords = /usr/local/manticore/data/stopwords.txt
  3.   stopwords = stopwords-ru.txt stopwords-en.txt
  5.   type = rt
  6.   path = tbl
  7.   rt_field = title
  8.   rt_attr_uint = price
  9. }

Alternatively you can use one of the default stop word files that come with Manticore. Currently stop words for 50 languages are available. Here is the full list of aliases for them:

  • af - Afrikaans
  • ar - Arabic
  • bg - Bulgarian
  • bn - Bengali
  • ca - Catalan
  • ckb- Kurdish
  • cz - Czech
  • da - Danish
  • de - German
  • el - Greek
  • en - English
  • eo - Esperanto
  • es - Spain
  • et - Estonian
  • eu - Basque
  • fa - Persian
  • fi - Finnish
  • fr - French
  • ga - Irish
  • gl - Galician
  • hi - Hindi
  • he - Hebrew
  • hr - Croatian
  • hu - Hungarian
  • hy - Armenian
  • id - Indonesian
  • it - Italian
  • ja - Japanese
  • ko - Korean
  • la - Latin
  • lt - Lithuanian
  • lv - Latvian
  • mr - Marathi
  • nl - Dutch
  • no - Norwegian
  • pl - Polish
  • pt - Portuguese
  • ro - Romanian
  • ru - Russian
  • sk - Slovak
  • sl - Slovenian
  • so - Somali
  • st - Sotho
  • sv - Swedish
  • sw - Swahili
  • th - Thai
  • tr - Turkish
  • yo - Yoruba
  • zh - Chinese
  • zu - Zulu

For example, to use stop words for Italian language just put the following line in your config file:

  • SQL
  • JSON
  • PHP
  • Python
  • javascript
  • Java
  • CONFIG

SQL JSON PHP Python javascript Java CONFIG

  1. CREATE TABLE products(title text, price float) stopwords = 'it'
  1. POST /cli -d "
  2. CREATE TABLE products(title text, price float) stopwords = 'it'"
  1. $index = new \Manticoresearch\Index($client);
  2. $index->setName('products');
  3. $index->create([
  4.             'title'=>['type'=>'text'],
  5.             'price'=>['type'=>'float']
  6.         ],[
  7.             'stopwords' => 'it'
  8.         ]);
  1. utilsApi.sql('CREATE TABLE products(title text, price float) stopwords = \'it\'')
  1. res = await utilsApi.sql('CREATE TABLE products(title text, price float) stopwords = \'it\'');
  1. utilsApi.sql("CREATE TABLE products(title text, price float) stopwords = 'it'");
  1. table products {
  2.   stopwords = it
  4.   type = rt
  5.   path = tbl
  6.   rt_field = title
  7.   rt_attr_uint = price
  8. }

If you need to use stop words for multiple languages you should list all their aliases, separated with commas (RT mode) or spaces (plain mode):

  • SQL
  • JSON
  • PHP
  • Python
  • javascript
  • Java
  • CONFIG

SQL JSON PHP Python javascript Java CONFIG

  1. CREATE TABLE products(title text, price float) stopwords = 'en, it, ru'
  1. POST /cli -d "
  2. CREATE TABLE products(title text, price float) stopwords = 'en, it, ru'"
  1. $index = new \Manticoresearch\Index($client);
  2. $index->setName('products');
  3. $index->create([
  4.             'title'=>['type'=>'text'],
  5.             'price'=>['type'=>'float']
  6.         ],[
  7.             'stopwords' => 'en, it, ru'
  8.         ]);
  1. utilsApi.sql('CREATE TABLE products(title text, price float) stopwords = \'en, it, ru\'')
  1. res = await utilsApi.sql('CREATE TABLE products(title text, price float) stopwords = \'en, it, ru\'');
  1. utilsApi.sql("CREATE TABLE products(title text, price float) stopwords = 'en, it, ru'");
  1. table products {
  2.   stopwords = en it ru
  4.   type = rt
  5.   path = tbl
  6.   rt_field = title
  7.   rt_attr_uint = price
  8. }

stopword_step

  1. stopword_step={0|1}

Position increment on stopwords. Optional, allowed values are 0 and 1, default is 1.

  • SQL
  • JSON
  • PHP
  • Python
  • javascript
  • Java
  • CONFIG

SQL JSON PHP Python javascript Java CONFIG

  1. CREATE TABLE products(title text, price float) stopwords = 'en' stopword_step = '1'
  1. POST /cli -d "
  2. CREATE TABLE products(title text, price float) stopwords = 'en' stopword_step = '1'"
  1. $index = new \Manticoresearch\Index($client);
  2. $index->setName('products');
  3. $index->create([
  4.             'title'=>['type'=>'text'],
  5.             'price'=>['type'=>'float']
  6.         ],[
  7.             'stopwords' => 'en, it, ru',
  8.             'stopword_step' => '1'
  9.         ]);
  1. utilsApi.sql('CREATE TABLE products(title text, price float) stopwords = \'en\' stopword_step = \'1\'')
  1. res = await utilsApi.sql('CREATE TABLE products(title text, price float) stopwords = \'en\' stopword_step = \'1\'');
  1. utilsApi.sql("CREATE TABLE products(title text, price float) stopwords = \'en\' stopword_step = \'1\'");
  1. table products {
  2.   stopwords = en
  3.   stopword_step = 1
  5.   type = rt
  6.   path = tbl
  7.   rt_field = title
  8.   rt_attr_uint = price
  9. }

stopwords_unstemmed

  1. stopwords_unstemmed={0|1}

Whether to apply stop words before or after stemming. Optional, default is 0 (apply stop word filter after stemming).

By default, stop words are stemmed themselves, and applied to tokens after stemming (or any other morphology processing). In other words, by default, a token is stopped when stem(token) is equal to stem(stopword). That can lead to unexpected results when a token gets (erroneously) stemmed to a stopped root. For example, ‘Andes’ might get stemmed to ‘and’, so when ‘and’ is a stopword, ‘Andes’ is also skipped.

stopwords_unstemmed directive changed this behaviour. When it’s enabled, stop words are applied before stemming (and therefore to the original word forms), and the tokens are skipped when token is equal to stopword.

  • SQL
  • JSON
  • PHP
  • Python
  • javascript
  • Java
  • CONFIG

SQL JSON PHP Python javascript Java CONFIG

  1. CREATE TABLE products(title text, price float) stopwords = 'en' stopwords_unstemmed = '1'
  1. POST /cli -d "
  2. CREATE TABLE products(title text, price float) stopwords = 'en' stopwords_unstemmed = '1'"
  1. $index = new \Manticoresearch\Index($client);
  2. $index->setName('products');
  3. $index->create([
  4.             'title'=>['type'=>'text'],
  5.             'price'=>['type'=>'float']
  6.         ],[
  7.             'stopwords' => 'en, it, ru',
  8.             'stopwords_unstemmed' => '1'
  9.         ]);
  1. utilsApi.sql('CREATE TABLE products(title text, price float) stopwords = \'en\' stopwords_unstemmed = \'1\'')
  1. res = await utilsApi.sql('CREATE TABLE products(title text, price float) stopwords = \'en\' stopwords_unstemmed = \'1\'');
  1. utilsApi.sql("CREATE TABLE products(title text, price float) stopwords = \'en\' stopwords_unstemmed = \'1\'");
  1. table products {
  2.   stopwords = en
  3.   stopwords_unstemmed = 1
  5.   type = rt
  6.   path = tbl
  7.   rt_field = title
  8.   rt_attr_uint = price
  9. }

Word forms

Word forms are applied after tokenizing incoming text by charset_table rules. They essentially let you replace one word with another. Normally, that would be used to bring different word forms to a single normal form (e.g. to normalize all the variants such as “walks”, “walked”, “walking” to the normal form “walk”). It can also be used to implement stemming exceptions, because stemming is not applied to words found in the forms list.

wordforms

  1. wordforms = path/to/wordforms.txt
  2. wordforms = path/to/alternateforms.txt
  3. wordforms = path/to/dict*.txt

Word forms dictionary. Optional, default is empty.

The dictionaries are used to normalize incoming words both during indexing and searching. Therefore, when it comes to a plain table, it’s required to rotate the table in order to pick up changes in the word forms file.

Word forms support in Manticore is designed to handle large dictionaries well. They moderately affect indexing speed; for example, a dictionary with 1 million entries slows down indexing by about 1.5 times. Searching speed is not affected at all. The additional RAM impact is roughly equal to the dictionary file size, and dictionaries are shared across tables. For instance, if the very same 50 MB word forms file is specified for 10 different tables, the additional searchd RAM usage will be about 50 MB.

The dictionary file should be in a simple plain text format. Each line should contain source and destination word forms, in UTF-8 encoding, separated by a “greater than” sign. Rules from the charset_table will be applied when the file is loaded, so if you are using built-in charset_table options, it is typically case-insensitive, just like your other full-text indexed data. Here is a sample file contents:

  1. walks > walk
  2. walked > walk
  3. walking > walk

There is a bundled utility called Spelldump that helps you create a dictionary file in a format that Manticore can read. The utility can read from source .dict and .aff dictionary files in the ispell or MySpell format, as bundled with OpenOffice.

You can map several source words to a single destination word. The process happens on tokens, not the source text, so differences in whitespace and markup are ignored.

You can use the => symbol instead of >. Comments (starting with #) are also allowed. Finally, if a line starts with a tilde (~), the wordform will be applied after morphology, instead of before (note that only a single source word is supported in this case).

  1. core 2 duo > c2d
  2. e6600 > c2d
  3. core 2duo => c2d # Some people write '2duo' together...
  4. ~run > walk # Along with stem_en morphology enabled replaces 'run', 'running', 'runs' (and any other words that stem to just 'run') to 'walk'

You can specify multiple destination tokens:

  1. s02e02 > season 2 episode 2
  2. s3 e3 > season 3 episode 3

You can specify multiple files, not just one. Masks can be used as a pattern, and all matching files will be processed in simple ascending order.

In the RT mode, only absolute paths are allowed.

If multi-byte codepages are used and file names include foreign characters, the resulting order may not be exactly alphabetic. If the same wordform definition is found in multiple files, the latter one is used and overrides previous definitions.

  • SQL
  • JSON
  • PHP
  • Python
  • javascript
  • Java
  • CONFIG

SQL JSON PHP Python javascript Java CONFIG

  1. CREATE TABLE products(title text, price float) wordforms = '/var/lib/manticore/wordforms.txt' wordforms = '/var/lib/manticore/alternateforms.txt /var/lib/manticore/dict*.txt'
  1. POST /cli -d "
  2. CREATE TABLE products(title text, price float) wordforms = '/var/lib/manticore/wordforms.txt' wordforms = '/var/lib/manticore/alternateforms.txt' wordforms = '/var/lib/manticore/dict*.txt'"
  1. $index = new \Manticoresearch\Index($client);
  2. $index->setName('products');
  3. $index->create([
  4.             'title'=>['type'=>'text'],
  5.             'price'=>['type'=>'float']
  6.         ],[
  7.             'wordforms' => [
  8.                 '/var/lib/manticore/wordforms.txt',
  9.                 '/var/lib/manticore/alternateforms.txt',
  10.                 '/var/lib/manticore/dict*.txt'
  11.             ]
  12.         ]);
  1. utilsApi.sql('CREATE TABLE products(title text, price float) wordforms = \'/var/lib/manticore/wordforms.txt\' wordforms = \'/var/lib/manticore/alternateforms.txt\' wordforms = \'/var/lib/manticore/dict*.txt\'')
  1. res = await utilsApi.sql('CREATE TABLE products(title text, price float)wordforms = \'/var/lib/manticore/wordforms.txt\' wordforms = \'/var/lib/manticore/alternateforms.txt\' wordforms = \'/var/lib/manticore/dict*.txt\'');
  1. utilsApi.sql("CREATE TABLE products(title text, price float) wordforms = '/var/lib/manticore/wordforms.txt' wordforms = '/var/lib/manticore/alternateforms.txt' wordforms = '/var/lib/manticore/dict*.txt'");
  1. table products {
  2.   wordforms = /var/lib/manticore/wordforms.txt
  3.   wordforms = /var/lib/manticore/alternateforms.txt
  4.   wordforms = /var/lib/manticore/dict*.txt
  6.   type = rt
  7.   path = tbl
  8.   rt_field = title
  9.   rt_attr_uint = price
  10. }