You're viewing version 3.8 of the OpenSearch documentation. This version is no longer maintained. For the latest version, see the current documentation. For information about OpenSearch version maintenance, see Release Schedule and Maintenance Policy.
Kuromoji stemmer token filter
The kuromoji_stemmer token filter normalizes katakana words by removing a trailing long vowel mark (ー) from words that meet a minimum length threshold. Many foreign loanwords in Japanese are written in katakana with a trailing ー that denotes a lengthened final vowel (for example, コンピューター, プリンター). In practice, Japanese speakers often drop the trailing ー in informal or technical writing, resulting in two surface forms for the same word. This filter collapses those variants to a single canonical form.
The filter makes conversions such as the following:
- コンピューター (computer) becomes コンピュータ.
- プリンター (printer) becomes プリンタ.
Installation
The kuromoji_stemmer token filter requires the analysis-kuromoji plugin. For installation instructions, see Kuromoji analyzer.
Parameters
The following table lists the parameters for the kuromoji_stemmer token filter.
| Parameter | Data type | Description |
|---|---|---|
minimum_length | Integer | The minimum number of characters a token must have for the trailing long vowel mark to be removed. Tokens shorter than this threshold are passed through unchanged. Default is 4. |
The default minimum length of 4 prevents short words such as カー (car, two characters) from being incorrectly stemmed to カ.
Example
The following example creates an index with a custom analyzer that uses kuromoji_stemmer:
PUT /kuromoji-stemmer-index
{
"settings": {
"analysis": {
"filter": {
"katakana_stemmer": {
"type": "kuromoji_stemmer",
"minimum_length": 4
}
},
"analyzer": {
"stemmer_analyzer": {
"type": "custom",
"tokenizer": "kuromoji_tokenizer",
"filter": ["katakana_stemmer"]
}
}
}
}
}
Test the analyzer with katakana loanwords (meaning “I use a computer and printer”):
POST /kuromoji-stemmer-index/_analyze
{
"analyzer": "stemmer_analyzer",
"text": "コンピューターとプリンターを使う"
}
The response shows the trailing long vowel marks removed from both words:
{
"tokens": [
{
"token": "コンピュータ",
"start_offset": 0,
"end_offset": 7,
"type": "word",
"position": 0
},
{
"token": "と",
"start_offset": 7,
"end_offset": 8,
"type": "word",
"position": 1
},
{
"token": "プリンタ",
"start_offset": 8,
"end_offset": 13,
"type": "word",
"position": 2
},
{
"token": "を",
"start_offset": 13,
"end_offset": 14,
"type": "word",
"position": 3
},
{
"token": "使う",
"start_offset": 14,
"end_offset": 16,
"type": "word",
"position": 4
}
]
}